Spice4Cars

Establish an ML data management which supports

ML data management activities,
relevant sources of ML data,
ML data lifecycle including a status model, and
interfaces to affected parties.

Note 1: Supported ML data management activities may include data collection, labeling/annotation, and structuring.

Linked Knowledge Nuggets:
arrow_forward "Examples of tools used for managing ML data"

person Author: Process Fellows
These tools are typical examples for supporting a ML project.

CVAT — annotation:

CVAT is used to label data, especially images and videos. For example, if you want to train an object detection model, people can use CVAT to draw bounding boxes around cars, pedestrians, traffic signs, defects, or other objects.
In the ML pipeline, CVAT answers the question:
“What is shown in the data, and what should the model learn?”

Example:
A camera image contains three cars. In CVAT, an annotator marks each car with a bounding box and the label `car`. The model later learns from these annotations.

DVC — dataset versioning:

DVC stands for "Data Version Control". It is like Git, but for large datasets and machine learning artifacts.
In machine learning, datasets often change: new images are added, bad labels are corrected, or test data is separated from training data. DVC helps track exactly which dataset version was used for a specific experiment.
DVC answers the question:
“Which version of the data was used to train this model?”

Example:

Model A was trained on dataset version 1.0.
Model B was trained on dataset version 1.1 with 500 additional annotated images.

DVC helps reproduce both experiments later.

MLflow — metadata and experiment tracking:

MLflow is used to track machine learning experiments. It records information such as parameters, metrics, models, and results.
For example, when training a neural network, MLflow can store:
training accuracy, validation loss, learning rate, batch size, model version, training time, used dataset version, and output model file.
MLflow answers the question:
“Which experiment produced which result?”

Example:

Experiment 1 used learning rate `0.001` and reached 87% accuracy.
Experiment 2 used learning rate `0.0001` and reached 91% accuracy.

MLflow makes these experiments comparable.

S3 / MinIO — storage:

S3 is a cloud storage system, and MinIO is an S3-compatible storage solution that can be self-hosted. In machine learning, they are often used to store large files such as datasets, images, trained models, annotations, and logs.
S3 or MinIO answer the question:
“Where do we store large ML files reliably?”

Example:

Raw images are stored in S3 or MinIO.
DVC tracks which files belong to which dataset version.

MLflow stores model artifacts and experiment outputs there as well.

arrow_forward "Important aspects for collecting data"

person Author: Process Fellows

Select data sources
- Where does the data come from?
  - Own databases
  - External APIs
  - Web scraping (if permitted)
  - Public data sets, e.g. Kaggle: online platform for data science and machine learning by Google; UCI ML repository: One of the oldest collection of public accessible data sets for machine learning by University of California, Irvine
- Your own measurements
- Tip: Check whether you have access to the data and whether it may be used under license law.
Ensure data quality: Good data is the key to successful machine learning! Garbage in means Garbage out means Bad Model!
- Completeness: Are important values missing? (e.g. missing prices in sales data)
- Consistency: Do units and formats match? (e.g. date in the same format)
- Correctness: Are there obvious errors? (e.g. negative prices)
- Representativeness: Does the data cover all possible cases? (Avoidance of bias!)
- Balance: Are there unbalanced data classes? (e.g. highly uneven distribution in a classification)
- Tip: Clean and normalize data before modeling! “Normalize” means: Pre-processing step: numerical values are transformed into a specific range (e.g. between 0 and 1 or -1 and 1). Aim is to eliminate scaling differences between features and thus improve the performance of ML algorithms. Example: Min-Max-scaling (but there are other approaches too)
Determine amount of data: How much data do you need?
- Supervised Learning: The more complex the model, the more data is required.
- Unsupervised Learning: Clustering can often work with less data.
- Deep Learning: Often requires a lot of data (e.g. millions of images for image recognition).
- Tip: If data is scarce, use “data augmentation” (e.g. rotate images, vary texts).
Data format & structure: What structure does the data have?
- Tabular data: (CSV, Excel, SQL databases) → good for classic ML models
- Text data: (chat histories, articles) → NLP models (Natural Language Processing)
- Image and video data: (JPG, PNG, MP4) → Convolutional Neural Networks (CNNs)
- Time series: (sensor values, share prices) → Recurrent Neural Networks
- Tip: Standardized formats facilitate processing!
Organize data storage & access
- Databases: SQL (structured) vs. NoSQL (flexible, e.g. MongoDB for JSON data)
- Cloud storage: AWS S3, Google Drive, Azure
- Local: CSVs, pickle files for fast processing
- Tip: Version data to be able to track changes!
Consider data protection & ethics:
- Anonymization & pseudonymization: (e.g. for personal data)
- Compliance with GDPR/DSGVO: If user data is processed
- Avoid bias: Is the data fairly distributed or does it favor certain groups?
- Tip: If sensitive data is used, check whether it can be aggregated or anonymized.
Conclusion:
- Data is the foundation of machine learning
- Bad data → bad models!
- A good data acquisition workflow saves a lot of time and significantly improves prediction quality.

Develop an approach to ensure that the quality of ML data are analyzed based on the defined ML data quality criteria and that activities are performed to support the avoidance of biases of data.
Note 2: Examples of ML data quality criteria are relevant data sources, reliability and consistency of labelling, completeness against ML data requirements.
Note 3: The ML data management should support the quality criteria and activities of the ML data quality approach.
Note 4: Biases to avoid may include sampling bias (e.g., gender, age) and feedback loop bias.
Note 5: For the creation of ML data sets see MLE.3.BP2 and MLE.4.BP2.

Linked Knowledge Nuggets:
arrow_forward "Feedback loop bias explained"

arrow_forward "Important aspects for collecting data"

person Author: Process Fellows

Select data sources
- Where does the data come from?
  - Own databases
  - External APIs
  - Web scraping (if permitted)
  - Public data sets, e.g. Kaggle: online platform for data science and machine learning by Google; UCI ML repository: One of the oldest collection of public accessible data sets for machine learning by University of California, Irvine
- Your own measurements
- Tip: Check whether you have access to the data and whether it may be used under license law.
Ensure data quality: Good data is the key to successful machine learning! Garbage in means Garbage out means Bad Model!
- Completeness: Are important values missing? (e.g. missing prices in sales data)
- Consistency: Do units and formats match? (e.g. date in the same format)
- Correctness: Are there obvious errors? (e.g. negative prices)
- Representativeness: Does the data cover all possible cases? (Avoidance of bias!)
- Balance: Are there unbalanced data classes? (e.g. highly uneven distribution in a classification)
- Tip: Clean and normalize data before modeling! “Normalize” means: Pre-processing step: numerical values are transformed into a specific range (e.g. between 0 and 1 or -1 and 1). Aim is to eliminate scaling differences between features and thus improve the performance of ML algorithms. Example: Min-Max-scaling (but there are other approaches too)
Determine amount of data: How much data do you need?
- Supervised Learning: The more complex the model, the more data is required.
- Unsupervised Learning: Clustering can often work with less data.
- Deep Learning: Often requires a lot of data (e.g. millions of images for image recognition).
- Tip: If data is scarce, use “data augmentation” (e.g. rotate images, vary texts).
Data format & structure: What structure does the data have?
- Tabular data: (CSV, Excel, SQL databases) → good for classic ML models
- Text data: (chat histories, articles) → NLP models (Natural Language Processing)
- Image and video data: (JPG, PNG, MP4) → Convolutional Neural Networks (CNNs)
- Time series: (sensor values, share prices) → Recurrent Neural Networks
- Tip: Standardized formats facilitate processing!
Organize data storage & access
- Databases: SQL (structured) vs. NoSQL (flexible, e.g. MongoDB for JSON data)
- Cloud storage: AWS S3, Google Drive, Azure
- Local: CSVs, pickle files for fast processing
- Tip: Version data to be able to track changes!
Consider data protection & ethics:
- Anonymization & pseudonymization: (e.g. for personal data)
- Compliance with GDPR/DSGVO: If user data is processed
- Avoid bias: Is the data fairly distributed or does it favor certain groups?
- Tip: If sensitive data is used, check whether it can be aggregated or anonymized.
Conclusion:
- Data is the foundation of machine learning
- Bad data → bad models!
- A good data acquisition workflow saves a lot of time and significantly improves prediction quality.

Relevant sources for the raw data are identified and continuously monitored for changes. The raw data are collected according to the ML data requirements.
Note 6: The identification and collection of ML data might be an organizational responsibility.
Note 7: Continuous monitoring should include the ODD and may lead to changes of the ML requirements.

Linked Knowledge Nuggets:
arrow_forward "Important aspects for collecting data"

person Author: Process Fellows

Select data sources
- Where does the data come from?
  - Own databases
  - External APIs
  - Web scraping (if permitted)
  - Public data sets, e.g. Kaggle: online platform for data science and machine learning by Google; UCI ML repository: One of the oldest collection of public accessible data sets for machine learning by University of California, Irvine
- Your own measurements
- Tip: Check whether you have access to the data and whether it may be used under license law.
Ensure data quality: Good data is the key to successful machine learning! Garbage in means Garbage out means Bad Model!
- Completeness: Are important values missing? (e.g. missing prices in sales data)
- Consistency: Do units and formats match? (e.g. date in the same format)
- Correctness: Are there obvious errors? (e.g. negative prices)
- Representativeness: Does the data cover all possible cases? (Avoidance of bias!)
- Balance: Are there unbalanced data classes? (e.g. highly uneven distribution in a classification)
- Tip: Clean and normalize data before modeling! “Normalize” means: Pre-processing step: numerical values are transformed into a specific range (e.g. between 0 and 1 or -1 and 1). Aim is to eliminate scaling differences between features and thus improve the performance of ML algorithms. Example: Min-Max-scaling (but there are other approaches too)
Determine amount of data: How much data do you need?
- Supervised Learning: The more complex the model, the more data is required.
- Unsupervised Learning: Clustering can often work with less data.
- Deep Learning: Often requires a lot of data (e.g. millions of images for image recognition).
- Tip: If data is scarce, use “data augmentation” (e.g. rotate images, vary texts).
Data format & structure: What structure does the data have?
- Tabular data: (CSV, Excel, SQL databases) → good for classic ML models
- Text data: (chat histories, articles) → NLP models (Natural Language Processing)
- Image and video data: (JPG, PNG, MP4) → Convolutional Neural Networks (CNNs)
- Time series: (sensor values, share prices) → Recurrent Neural Networks
- Tip: Standardized formats facilitate processing!
Organize data storage & access
- Databases: SQL (structured) vs. NoSQL (flexible, e.g. MongoDB for JSON data)
- Cloud storage: AWS S3, Google Drive, Azure
- Local: CSVs, pickle files for fast processing
- Tip: Version data to be able to track changes!
Consider data protection & ethics:
- Anonymization & pseudonymization: (e.g. for personal data)
- Compliance with GDPR/DSGVO: If user data is processed
- Avoid bias: Is the data fairly distributed or does it favor certain groups?
- Tip: If sensitive data is used, check whether it can be aggregated or anonymized.
Conclusion:
- Data is the foundation of machine learning
- Bad data → bad models!
- A good data acquisition workflow saves a lot of time and significantly improves prediction quality.

arrow_forward "What is a "hybrid dataset"?"

The raw data are processed (annotated, analyzed, and structured) according to the ML data requirements.

Perform the activities according to the ML data quality approach to ensure that the ML data meet the defined ML data quality criteria.
Note 8: These activities may include sample-based reviews or statistical methods.

Linked Knowledge Nuggets:
arrow_forward "Important aspects for collecting data"

person Author: Process Fellows

Select data sources
- Where does the data come from?
  - Own databases
  - External APIs
  - Web scraping (if permitted)
  - Public data sets, e.g. Kaggle: online platform for data science and machine learning by Google; UCI ML repository: One of the oldest collection of public accessible data sets for machine learning by University of California, Irvine
- Your own measurements
- Tip: Check whether you have access to the data and whether it may be used under license law.
Ensure data quality: Good data is the key to successful machine learning! Garbage in means Garbage out means Bad Model!
- Completeness: Are important values missing? (e.g. missing prices in sales data)
- Consistency: Do units and formats match? (e.g. date in the same format)
- Correctness: Are there obvious errors? (e.g. negative prices)
- Representativeness: Does the data cover all possible cases? (Avoidance of bias!)
- Balance: Are there unbalanced data classes? (e.g. highly uneven distribution in a classification)
- Tip: Clean and normalize data before modeling! “Normalize” means: Pre-processing step: numerical values are transformed into a specific range (e.g. between 0 and 1 or -1 and 1). Aim is to eliminate scaling differences between features and thus improve the performance of ML algorithms. Example: Min-Max-scaling (but there are other approaches too)
Determine amount of data: How much data do you need?
- Supervised Learning: The more complex the model, the more data is required.
- Unsupervised Learning: Clustering can often work with less data.
- Deep Learning: Often requires a lot of data (e.g. millions of images for image recognition).
- Tip: If data is scarce, use “data augmentation” (e.g. rotate images, vary texts).
Data format & structure: What structure does the data have?
- Tabular data: (CSV, Excel, SQL databases) → good for classic ML models
- Text data: (chat histories, articles) → NLP models (Natural Language Processing)
- Image and video data: (JPG, PNG, MP4) → Convolutional Neural Networks (CNNs)
- Time series: (sensor values, share prices) → Recurrent Neural Networks
- Tip: Standardized formats facilitate processing!
Organize data storage & access
- Databases: SQL (structured) vs. NoSQL (flexible, e.g. MongoDB for JSON data)
- Cloud storage: AWS S3, Google Drive, Azure
- Local: CSVs, pickle files for fast processing
- Tip: Version data to be able to track changes!
Consider data protection & ethics:
- Anonymization & pseudonymization: (e.g. for personal data)
- Compliance with GDPR/DSGVO: If user data is processed
- Avoid bias: Is the data fairly distributed or does it favor certain groups?
- Tip: If sensitive data is used, check whether it can be aggregated or anonymized.
Conclusion:
- Data is the foundation of machine learning
- Bad data → bad models!
- A good data acquisition workflow saves a lot of time and significantly improves prediction quality.

Inform all affected parties about the agreed processed ML data and provide them to the affected parties.

Evidence of interpersonal communication.

Identifies:

Scope of information
Need for feedback, for example an expected confirmation within one week
Meta data, for example time when communication was done or how information was distributed.

Includes:

Personal information
Work-flows, for example within tools

Examples and References:

E-mails and other forms of memos
Verbal statements
Meeting minutes, for example in standups
Electronic media, for example webcasts, blog posts intranet forum
Chat protocols
Wiki pages
Photo protocol

Used by these processes:

ACQ.4 Supplier Monitoring
HWE.1 Hardware Requirements Analysis
HWE.2 Hardware Design
HWE.3 Verification against Hardware Design
HWE.4 Verification against Hardware Requirements
MAN.3 Project Management
MLE.1 Machine Learning Requirements Analysis
MLE.2 Machine Learning Architecture
MLE.3 Machine Learning Training
MLE.4 Machine Learning Model Testing
PIM.3 Process Improvement
REU.2 Reuse of Products
SUP.1 Quality Assurance
SUP.11 Machine Learning Data Management
SWE.1 Software Requirements Analysis
SWE.2 Software Architectural Design
SWE.3 Software Detailed Design and Unit Construction
SWE.4 Software Unit Verification
SWE.5 Software Component Verification and Integration Verification
SWE.6 Software Verification
SYS.1 Requirements Elicitation
SYS.2 System Requirements Analysis
SYS.3 System Architectural Design
SYS.4 System Integration and Integration Verification
SYS.5 System Verification
VAL.1 Validation

Used by these process attributes:

PA2.1 Process performance management process attribute

Data used for machine learning, attributed by metadata.

Identifies:

Unique ID
Attributes
Data characteristics, for example audio recordings sensor data, pictures and videos
Data created by an algorithm

Includes:

Data created by processing of other data, for example added noise, changed colors or merged pictures

Used by these processes:

SUP.11 Machine Learning Data Management

Data management supporting machine learning engineering (MLE).

Identifies:

Data management activities
Data collection workflow
Labeling, Annotations and description
Ingestion
Exploration and profiling workflows
Procedure of structuring and cleansing
Related phases of engineering, for example ML training

Includes:

References to configuration management
Information of data sources

Examples and References:

Supports the relevant sources of ML data

Used by these processes:

SUP.11 Machine Learning Data Management

Approach describing criteria and activities to satisfy ML data quality goals.

Identifies:

Quality criteria for example relevant data sources.
Analysis activities of the data

Includes:

Reliability and consistency of labeling
Completeness against ML data requirements

Examples and References:

Quality criteria to avoid data bias

Used by these processes:

SUP.11 Machine Learning Data Management

SUP.11 Machine Learning Data Management

intacs® Certified Automotive SPICE®​ Hardware Engineering

intacs^® Certified Automotive SPICE^® Hardware Engineering