[Testmanagement] TRG Test Automation Proposal

[ds-hzimmer]

Overview

Providing a Tractus-X Release Guideline (TRG) for Test Automation using Open Source methods and tools, in order to give product teams a template and best-practice example to start with or improve

Focus will be on API tests at the integration test level (end-to-end). Potentially UI tests at the integration test level can also be covered. Test automation with Unit Tests at lower test levels are presumably out of scope of this TRG, as those are already covered elsewhere and standard for implementation.

Explain the topic in 2 sentences

Creation of a TRG for test automation Open Source methods and tools. Providing code examples for an implementation focused on Cucumber (Gherkin).

What's the benefit?

Providing a guideline on how to implement a closed test automation cycle from feature user story via test case creation to implementation, test execution and test reporting.

Ultimately this should result in increased quality and speed in test execution.

What are the Risks/Dependencies ?

• TRG would be defined within 25.03 release cycle, but presumably only go into effect for subsequent releases.

• Other testing tools and methods than described in the TRG could be used instead by product teams to achieve the intended effect, in particular also manual testing. Thus not an immediate requirement.

• To be determined if integration with test management system to download feature files and upload results should be described as-is, omitted, or described generically.

Detailed explanation

TRG contents:

• Test case creation using Gherkin syntax, DSL feature files
• Use Cucumber with Maven to generate test classes and implement them.
• Integrate GitHub automation to execute tests
• (Potentially additional example code to import test run results into Jira Xray test management system (not open source))

Current implementation

Compare to existing implementations for products Trace-X and IRS as examples that can be provided https://github.com/eclipse-tractusx/item-relationship-service/tree/main/irs-cucumber-tests https://github.com/eclipse-tractusx/traceability-foss/tree/main/tx-cucumber-tests

Proposed improvements

Feature Team

Catena-X e.V. test management (doubleSlash, on order of the association)

Potentially involvement of additional teams. Interviews for current test automation status have already been conducted with first teams.

Contributor

• ds-meberle
• ds-hzimmer
• ds-asmierzchalski

Additional contributors to be determined

Committer

• ds-lcapellino

Additional committers to be determined

User Stories

• Issue 1, linked to specific repository
• Issue 2, linked to another specific repository

Acceptance Criteria

• [ ] TRG for Test Automation proposal created
• [ ] TRG for Test Automation discussed with the respective expert groups and committees
• [ ] TRG for Test Automation published

Out of scope: Implementation of Test Automation TRG by a new product team within the release.

Test Cases

Note: Not directly applicable as this feature suggestion describes a testing improvement itself.

Test Case 1: Test Automation TRG describes definition of test cases (structure, syntax, etc.) with Cucumber

Test Case 2: Test Automation TRG includes suggestions for implementation, including code examples.

(Though exact implementation of scenarios of course depends on the used programming language.)

Test Case 3: Test Automation TRG describes test case handling:

• Download of feature files, test execution of the feature files, generation of test resuls

Test Case 3: Test Automation should describe scenarios for different REST API calls

• for a GET API call
• for a POST/PUT API call
• for a DELETE API call (if applicable in the product context)

Steps and Expected Results to be determined

Architectural Relevance

Note: Some checks potentially not directly applicable as this feature suggestion describes a testing/test management improvement and not a feature change in the functionality of a product itself.

The following items are ensured (answer: yes) after this issue is implemented:

• [x] This feature aligns with our current architectural guidelines
  • Data Sovereignty: All data sharing activities across company boundaries follow the Catena-X Regulatory Framework, in particular the Data Exchange Governance, and the Dataspace Protocol via a compliant Connector (like the tractusx-edc or similar, see Connector KIT)
  • [x] CX-0010 Business Partner Number
  • [x] CX-0013 Identity of Member Companies
  • [x] CX-0018 Data Space Connectivity (EDC)
  • [x] CX-0049 DID Document Schema
  • [x] CX-0050 Framework Agreement Credential
  • [x] CX-0149 Verified Company Identity
  • Interoperability: Digital Twins are used (compliant to the Digital Twin KIT and the Industry Core KIT)
  • [x] CX-0001 EDC Discovery API
  • [x] CX-0002 Digital Twins in Catena-X
  • [x] CX-0018 Data Space Connectivity (EDC)
  • Data Format:
  • [x] The data model is based on a published Semantic Model
• [x] The impact on the overall system architecture has been assessed. The Feature does not require changes to the architecture or any existing standard? Please have a look here on the overarching architecture
• [x] Potential risks or conflicts with existing architecture has been assessed

Justification: (Fill this out, if at least one of the checkboxes above cannot be ticked. Contact the Architecture Management Committee to get an approval for the justification)

Additional information

• [x] I am aware that my request may not be developed if no developer can be found for it. I'll try to contribute a developer (bring your own developer)

[stephanbcbauer]

Some hints from Release Management (@ther3sa) and Tractus-X Project Lead (@stephanbcbauer)

• Status currently in Inbox. ⇾ Only features with status backlog are considered in open planning
• Please add missing sections from the feature template, or fill them out