personAuthor: Process Fellows
Test Management ensures that testing activities are strategically planned, monitored, and evaluated across all development phases. From unit tests to system-level integration, this cross-cutting discipline defines methods, tools, documents, and roles to ensure traceable and efficient verification and validation.
school
PF_Testmanagement_Extract.pdf Short Overview of Test Management (related to all Automotive SPICE® verification processes)
# PROCESS PURPOSE
The purpose is to ensure that the production data compliant (Hardware = Assembled and interconnected electrical or electronic hardware components or parts which perform analog or digital functions or operations.) is verified to provide evidence for compliance with the (Hardware = Assembled and interconnected electrical or electronic hardware components or parts which perform analog or digital functions or operations.) design.
# PROCESS OUTCOMES
O1 (Verification measure = Verification measure can be:
Test cases
Measurements
Calculations
Simulations
Reviews
Analyses
Note, that in particular domains certain verification measures may not be applicable, e.g., software units generally cannot be verified by means of calculations or analyses.) are specified for the verification of the (Hardware = Assembled and interconnected electrical or electronic hardware components or parts which perform analog or digital functions or operations.) against the (Hardware = Assembled and interconnected electrical or electronic hardware components or parts which perform analog or digital functions or operations.) design, including the interfaces between the (Hardware element = Generic term; can represent a hardware component, a hardware part, a hardware interface, or the hardware.) and the dynamic aspects.
O2 (Verification measure = Verification measure can be:
Test cases
Measurements
Calculations
Simulations
Reviews
Analyses
Note, that in particular domains certain verification measures may not be applicable, e.g., software units generally cannot be verified by means of calculations or analyses.) are selected according to the (Release = A product delivered to a customer, including a defined set of functionalities and properties.) scope considering criteria, including criteria for (Regression verification = Selective re-verification of elements to verify that modifications have not caused unintended effects.).
O3
The verification is performed on production data compliant samples using the selected (Verification measure = Verification measure can be:
Test cases
Measurements
Calculations
Simulations
Reviews
Analyses
Note, that in particular domains certain verification measures may not be applicable, e.g., software units generally cannot be verified by means of calculations or analyses.), and the verification results are recorded.
O4
Consistency and bidirectional traceability are established between the (Hardware element = Generic term; can represent a hardware component, a hardware part, a hardware interface, or the hardware.) and the (Verification measure = Verification measure can be:
Test cases
Measurements
Calculations
Simulations
Reviews
Analyses
Note, that in particular domains certain verification measures may not be applicable, e.g., software units generally cannot be verified by means of calculations or analyses.).
O5
Bidirectional traceability is established between the (Verification measure = Verification measure can be:
Test cases
Measurements
Calculations
Simulations
Reviews
Analyses
Note, that in particular domains certain verification measures may not be applicable, e.g., software units generally cannot be verified by means of calculations or analyses.) and the verification results.
O6
The verification results are summarized and communicated to all affected parties.
# BASE PRACTICES
BP1
Specify verification measures for the verification against the hardware design. (
O1 )
Specify the (Verification measure = Verification measure can be:
Test cases
Measurements
Calculations
Simulations
Reviews
Analyses
Note, that in particular domains certain verification measures may not be applicable, e.g., software units generally cannot be verified by means of calculations or analyses.) suitable to provide evidence for compliance of the (Hardware = Assembled and interconnected electrical or electronic hardware components or parts which perform analog or digital functions or operations.) with the (Hardware = Assembled and interconnected electrical or electronic hardware components or parts which perform analog or digital functions or operations.) design and its dynamic aspects. This includes
techniques for the (Verification measure = Verification measure can be:
Test cases
Measurements
Calculations
Simulations
Reviews
Analyses
Note, that in particular domains certain verification measures may not be applicable, e.g., software units generally cannot be verified by means of calculations or analyses.),
pass/fail criteria for the (Verification measure = Verification measure can be:
Test cases
Measurements
Calculations
Simulations
Reviews
Analyses
Note, that in particular domains certain verification measures may not be applicable, e.g., software units generally cannot be verified by means of calculations or analyses.),
a definition of entry and exit criteria for the (Verification measure = Verification measure can be:
Test cases
Measurements
Calculations
Simulations
Reviews
Analyses
Note, that in particular domains certain verification measures may not be applicable, e.g., software units generally cannot be verified by means of calculations or analyses.),
necessary sequence of the (Verification measure = Verification measure can be:
Test cases
Measurements
Calculations
Simulations
Reviews
Analyses
Note, that in particular domains certain verification measures may not be applicable, e.g., software units generally cannot be verified by means of calculations or analyses.), and
the required verification infrastructure and environment setup.
Note 1: Examples on what a (Verification measure = Verification measure can be:
Test cases
Measurements
Calculations
Simulations
Reviews
Analyses
Note, that in particular domains certain verification measures may not be applicable, e.g., software units generally cannot be verified by means of calculations or analyses.) may focus on are the timeliness and timing dependencies of the correct signal flow between interfacing (Hardware element = Generic term; can represent a hardware component, a hardware part, a hardware interface, or the hardware.), or interactions between (Hardware component = Logical (e.g., functional block) or physical group of hardware parts realizing a functionality, which
cannot be realized by any of its hardware parts alone, e.g. voltage monitoring, power supply.
may be organized hierarchically, i.e., a hardware component can contain lower-level hardware components.
Note: Depending on the application, e.g., the populated PCB, a system-on-chip, a microcontroller, or an SBC can be considered a HW component.). Note 2: Measuring points can be used for stepwise testing of (Hardware element = Generic term; can represent a hardware component, a hardware part, a hardware interface, or the hardware.).
BP2
Ensure the use of compliant samples. (
O3 )
Ensure that the samples used for the verification against the (Hardware = Assembled and interconnected electrical or electronic hardware components or parts which perform analog or digital functions or operations.) design are compliant with the corresponding production data, including the special characteristics. Ensure that deviations are documented and that they do not alter the verification results. Note 3: Examples of compliance are sample reports, record of visual inspection, ICT report.
BP3
Select verification measures. (
O2 )
Document the selection of (Verification measure = Verification measure can be:
Test cases
Measurements
Calculations
Simulations
Reviews
Analyses
Note, that in particular domains certain verification measures may not be applicable, e.g., software units generally cannot be verified by means of calculations or analyses.) considering the selection criteria including regression criteria. The documented selection of (Verification measure = Verification measure can be:
Test cases
Measurements
Calculations
Simulations
Reviews
Analyses
Note, that in particular domains certain verification measures may not be applicable, e.g., software units generally cannot be verified by means of calculations or analyses.) shall have sufficient coverage according to the (Release = A product delivered to a customer, including a defined set of functionalities and properties.) scope. Note 4: Examples for selection criteria can be prioritization of requirements, the need for regression due to changes to the (Hardware = Assembled and interconnected electrical or electronic hardware components or parts which perform analog or digital functions or operations.) design, or the intended use of the delivered (Hardware = Assembled and interconnected electrical or electronic hardware components or parts which perform analog or digital functions or operations.) (Release = A product delivered to a customer, including a defined set of functionalities and properties.) (e.g., test bench, test track, public road etc.).
BP4
Verify against the hardware design. (
O3 )
Verify the (Hardware = Assembled and interconnected electrical or electronic hardware components or parts which perform analog or digital functions or operations.) design using the selected (Verification measure = Verification measure can be:
Test cases
Measurements
Calculations
Simulations
Reviews
Analyses
Note, that in particular domains certain verification measures may not be applicable, e.g., software units generally cannot be verified by means of calculations or analyses.). Record the verification results including pass/fail status and the corresponding (Verification measure = Verification measure can be:
Test cases
Measurements
Calculations
Simulations
Reviews
Analyses
Note, that in particular domains certain verification measures may not be applicable, e.g., software units generally cannot be verified by means of calculations or analyses.) output data. Note 5: See SUP.9 for handling the verification results which deviate from expected results.
Ensure consistency and establish bidirectional traceability between the (Hardware element = Generic term; can represent a hardware component, a hardware part, a hardware interface, or the hardware.) and the (Verification measure = Verification measure can be:
Test cases
Measurements
Calculations
Simulations
Reviews
Analyses
Note, that in particular domains certain verification measures may not be applicable, e.g., software units generally cannot be verified by means of calculations or analyses.). Establish bidirectional traceability between the (Verification measure = Verification measure can be:
Test cases
Measurements
Calculations
Simulations
Reviews
Analyses
Note, that in particular domains certain verification measures may not be applicable, e.g., software units generally cannot be verified by means of calculations or analyses.) and the verification results. Note 6: Bidirectional traceability supports consistency, and facilitates impact analysis of change requests, and demonstration of coverage. Traceability alone, e.g., the existence of links, does not necessarily mean that the information is consistent.
Linked Knowledge Nuggets: arrow_forward "Consistency vs. Traceability – What’s the Difference?"
personAuthor: Process Fellows
Consistency ensures that related content doesn’t contradict itself – e.g., requirements align with architecture and test. Traceability, in contrast, is about links: can you follow a requirement through to implementation and verification? Both are needed – consistency builds trust, traceability enables control. Typically, traceability strongly supports consistency review.
arrow_forward "The role of traceability in risk control"
personAuthor: Process Fellows
Traceability isn’t just about completeness — it’s about managing impact. When a requirement changes, trace links tell you what’s affected. That’s your early-warning system.
arrow_forward "The true benefit of traceability
"
personAuthor: Process Fellows
Sometimes the creation of traceability is seen as an additional expense, the benefits are not recognized.
Traceability should be set up at the same time as the derived elements are created. Both work products are open in front of us and the creation of the trace often only takes a few moments.
In the aftermath, the effort increases noticeably and the risk of gaps is high.
If the traceability is complete and consistent, the discovery of dependencies is unbeatably fast and reliable compared to searching for dependencies at a later stage, when there may also be time pressure.
It also enables proof of complete coverage of the derived elements and allows the complete consistency check.
BP6
Summarize and communicate the results. (
O6 )
Summarize the verification results and communicate them to all affected parties. Note 7: Providing all necessary information from the test case execution in a summary enables other parties to judge the consequences.
# OUTPUT INFORMATION ITEMS
13-52
Communication evidence (
O6 )
Evidence of interpersonal communication.Identifies:
Scope of information
Need for feedback, for example an expected confirmation within one week
Meta data, for example time when communication was done or how information was distributed.
Includes:
Personal information
Work-flows, for example within tools
Examples and References:
E-mails and other forms of memos
Verbal statements
Meeting minutes, for example in standups
Electronic media, for example webcasts, blog posts intranet forum
Chat protocols
Wiki pages
Photo protocol
Used by these processes:
ACQ.4 Supplier Monitoring
HWE.1 Hardware Requirements Analysis
HWE.2 Hardware Design
HWE.3 Verification against Hardware Design
HWE.4 Verification against Hardware Requirements
MAN.3 Project Management
MLE.1 Machine Learning Requirements Analysis
MLE.2 Machine Learning Architecture
MLE.3 Machine Learning Training
MLE.4 Machine Learning Model Testing
PIM.3 Process Improvement
REU.2 Reuse of Products
SUP.1 Quality Assurance
SUP.11 Machine Learning Data Management
SWE.1 Software Requirements Analysis
SWE.2 Software Architectural Design
SWE.3 Software Detailed Design and Unit Construction
SWE.4 Software Unit Verification
SWE.5 Software Component Verification and Integration Verification
SWE.6 Software Verification
SYS.1 Requirements Elicitation
SYS.2 System Requirements Analysis
SYS.3 System Architectural Design
SYS.4 System Integration and Integration Verification
SYS.5 System Verification
VAL.1 Validation
Used by these process attributes:
PA2.1 Process performance management process attribute
13-51
Consistency evidence (
O4, O5 )
Evidence of information to be semantically coherent alongrelevant artifacts, ensuring completeness, purpose maturity of processes, (Task = A definition, but not the execution, of a coherent set of atomic actions.) and products throughout their lifecycle.Identifies:
Traceability information, for example hyperlinks, repository location or editorial references.
Naming conventions
Relevant artifacts
Revision and revision history information
Change documentation and analysis information
Includes:
Meta-information, for example database identifiers notes in Git commits comments
Examples and References:
Evidence of Definition of Done (DoD) adherence.
Used by these processes:
HWE.1 Hardware Requirements Analysis
HWE.2 Hardware Design
HWE.3 Verification against Hardware Design
HWE.4 Verification against Hardware Requirements
MAN.3 Project Management
MLE.1 Machine Learning Requirements Analysis
MLE.2 Machine Learning Architecture
MLE.3 Machine Learning Training
MLE.4 Machine Learning Model Testing
SUP.8 Configuration Management
SUP.10 Change Request Management
SWE.1 Software Requirements Analysis
SWE.2 Software Architectural Design
SWE.3 Software Detailed Design and Unit Construction
SWE.4 Software Unit Verification
SWE.5 Software Component Verification and Integration Verification
SWE.6 Software Verification
SYS.2 System Requirements Analysis
SYS.3 System Architectural Design
SYS.4 System Integration and Integration Verification
SYS.5 System Verification
VAL.1 Validation
08-60
Verification Measure (
O1 )
Tool or method used to ensure the accuracy and reliability of a (Measurement = The activity to find the size, quantity or degree of something.) or test under specified conditions to verify specified functionality.Identifies:
Description of test scope, for example fault injection safety case
Classifications, for example black-box or grey-box test
(Measurement = The activity to find the size, quantity or degree of something.) to be performed during the test, for example an optical inspection
Test steps description
Pass/fail criteria, for example from end user survey
Includes:
Simulation scenarios
Emulation
Entry criteria
Equivalence classes and boundary values
Calculations and mathematical functions
Techniques and resources, for example review verification
Verification environment setup and configuration
Sequences
Used by these processes:
HWE.3 Verification against Hardware Design
HWE.4 Verification against Hardware Requirements
SWE.4 Software Unit Verification
SWE.5 Software Component Verification and Integration Verification
SWE.6 Software Verification
SYS.4 System Integration and Integration Verification
SYS.5 System Verification
03-50
Verification Measure Data (
O3 )
Data recorded during the execution of a (Verification measure = Verification measure can be:
Test cases
Measurements
Calculations
Simulations
Reviews
Analyses
Note, that in particular domains certain verification measures may not be applicable, e.g., software units generally cannot be verified by means of calculations or analyses.).Identifies:
Logging data, for example raw data or tool generated output
Values of (Measurement = The activity to find the size, quantity or degree of something.) and calculations
Protocols of simulations
Review data
Includes:
Finding records
Used by these processes:
HWE.3 Verification against Hardware Design
HWE.4 Verification against Hardware Requirements
SWE.4 Software Unit Verification
SWE.5 Software Component Verification and Integration Verification
SWE.6 Software Verification
SYS.4 System Integration and Integration Verification
SYS.5 System Verification
08-58
Verification Measure Selection Set (
O2 )
A set of (Verification measure = Verification measure can be:
Test cases
Measurements
Calculations
Simulations
Reviews
Analyses
Note, that in particular domains certain verification measures may not be applicable, e.g., software units generally cannot be verified by means of calculations or analyses.) selected for a specific purpose.Identifies:
(Verification measure = Verification measure can be:
Test cases
Measurements
Calculations
Simulations
Reviews
Analyses
Note, that in particular domains certain verification measures may not be applicable, e.g., software units generally cannot be verified by means of calculations or analyses.)
Scope of their application, for example a product variant, regression or a subcomponent
Includes:
Identification of the (Verification measure = Verification measure can be:
Test cases
Measurements
Calculations
Simulations
Reviews
Analyses
Note, that in particular domains certain verification measures may not be applicable, e.g., software units generally cannot be verified by means of calculations or analyses.)
Criteria for re-verification, for example in the case of changes
Dependencies
Used by these processes:
HWE.3 Verification against Hardware Design
HWE.4 Verification against Hardware Requirements
SWE.4 Software Unit Verification
SWE.5 Software Component Verification and Integration Verification
SWE.6 Software Verification
SYS.4 System Integration and Integration Verification
SYS.5 System Verification
15-52
Verification Results (
O3 )
Results of verification activities.Identifies:
Identification information
Log data
Passed and failed results
Not executed/blocked verification activities
Test execution details, for example tester name, role.
Timing information
Includes:
Summary of verification results
References, for example to problem resolution actions.
Used by these processes:
HWE.3 Verification against Hardware Design
HWE.4 Verification against Hardware Requirements
SWE.4 Software Unit Verification
SWE.5 Software Component Verification and Integration Verification
SWE.6 Software Verification
SYS.4 System Integration and Integration Verification
HWE.3
Verification against Hardware Design
