E2E Tests (Release 24.08)

[drcgjung]

Please open and fill in this issue in your product repository to document the status of E2E tests.

Show success/failure by attaching screenshots and/or result documents/failure messages.

Close this issue once the status of the E2E tests has been documented

Testers(s) and Role(s):

• Oghuzan Balandi (T-Systems, CONSUMER, SUPPLIER)
• Christoph Jung (T-Systems, OEM, PROVIDER, PRODUCER)

Deployment/Umbrella Chart Coordinates:

• https://github.com/big-data-spaces/knowledge-agents-deployment/tree/release/cx-2408/infrastructure

Tested Sub-Charts/Versions:

• tractusx-connector-azure-vault:0.7.3
• agent-plane-azure-vault:1.13.21-SNAPSHOT
• conforming-agent:1.13.21-SNAPSHOT
• matchmaking-agent:1.13.21-SNAPSHOT
• remoting-agent:1.13.21-SNAPSHOT
• provisioning-agent:1.13.21-SNAPSHOT
• aas-bridge:1.13.7-SNAPSHOT

Retest with:

• agent-plane-azure-vault:1.13.22-SNAPSHOT
• matchmaking-agent:1.13.22-SNAPSHOT

Release Management Reference Issue:

https://github.com/eclipse-tractusx/sig-release/issues/823

Preconditions / Integration Tests

• For Release 2408 Integration Test various scenarios need to be performed by the testing parties depending on their role (CONSUMER, OEM, PROVIDER, PRODUCER, SUPPLIER).
• Example performances of the integration steps can be found in the postman collection

CONSUMER of a sample OEM and PROVIDER:

• [x] Deploy Agent-Enabled Connector
• [x] Perform Remote Catalogue Request (OEM and PROVIDER)
• [x] Check Federated Catalogue (need to list OEM and PROVIDER assets)

OEM for a sample SUPPLIER, access Provisioning Agents and register a graph

• [x] Deploy Agent-Enabled Connector
• [x] Perform Remote Catalogue Request (against SUPPLIER)
• [x] Check Federated Catalogue (needs to see SUPPLIER and PROVIDER assets)
• [x] Deploy Provisioning Agent
• [x] Access Provisioning Agent (Reliability)
• [x] Access Provisioning Agent (Supply Chain)
• [x] Register Graph Asset(s)
• [x] Register Graph Policy
• [x] Register Graph Contract

SUPPLIER for a sample PRODUCER and OEM, access Remoting Agent and register a graph

• [x] Deploy Agent-Enabled Connector
• [x] Perform Remote Catalogue Request (against PRODUCER and OEM)
• [x] Check Federated Catalogue (of PRODUCER and OEM)
• [x] Deploy Remoting Agent with RUL binding
• [x] Access Remoting Agent (Remaining Useful Life)
• [x] Register Graph Asset
• [x] Upload Graph Asset (Supply Chain)
• [x] Register Graph Policy
• [x] Register Graph Contract

PRODUCER for a sample SUPPLIER and register a graph

• [x] Deploy Agent-Enabled Connector
• [x] Perform Remote Catalogue Request
• [x] Check Federated Catalogue (needs to see SUPPLIER and PROVIDER)
• [x] Upload Graph Asset (Supply Chain)
• [x] Register Graph Policy
• [x] Register Graph Contract

PROVIDER for sample OEM, SUPPLIER and PRODUCER and register a Skill

• [x] Deploy Agent-Enabled Connector
• [x] Perform Remote Catalogue Request (against OEM, SUPPLIER and PRODUCER)
• [x] Register Skill Asset
• [x] Register Skill Policy
• [x] Register Skill Contract

Test Plan

• The Test Steps are documented as API interactions in this postman collection.
• The concrete Test environment is referred in this postman environment. The necessary secrets will be communicated by the test management in private.

PROVIDER Test

• [x] E2E_02_00_01_OEM_GRAPH_POLICY OEM Creates a Policy for Providing Graphs and Skills to the E2E participants

• [x] E2E_02_00_02_OEM_GRAPH_CONTRACT OEM Creates a Contract Definition for Providing Graphs and Skills to the E2E participants

• [x] E2E_02_00_03_OEM_RELIABILITY_GRAPH OEM Creates a Graph Asset with Telematics Data

• [x] E2E_02_01_00_SUPPLIER_GRAPH_POLICY SUPPLIER Creates a Policy for Providing Graphs to the E2E participants

• [x] E2E_02_01_01_SUPPLIER_GRAPH_CONTRACT SUPPLIER Creates a Contract Definition for Providing Graphs to the E2E participants

• [x] E2E_02_01_02_SUPPLIER_BEHAVIOUR_GRAPH SUPPLIER Creates a Graph Asset with RUL Prognosis

• [x] E2E_02_01_03_SUPPLIER_BEHAVIOUR_GRAPH_VARIANT SUPPLIER Creates a Graph Asset with HI Prognosis

• [x] E2E_02_02_01_OPEN_POLICY PROVIDER Creates a Permissive Policy

• [x] E2E_02_02_02_OPEN_CONTRACT PROVIDER Creates a Permissive Contract Definition

• [x] E2E_02_02_03_REGISTER_RUL_SKILL_REMOTE PROVIDER Creates a RUL Skill in PROVIDER Mode

• [x] E2E_02_02_04_REGISTER_RUL_SKILL_LOCAL PROVIDER Creates a RUL Skill in CONSUMER Mode

• [x] E2E_02_02_06_REGISTER_BEH_SKILL_PROVIDER OEM Creates a Behaviour Skill in PROVIDER Mode

• [x] E2E_02_03_01_OEM_FEDERATED_CATALOGUE OEM Checks SUPPLIER Information in Federated Catalogue

• [x] E2E_02_03_02_OEM_CALL_REMOTE OEM Checks Successful Calling SUPPLIER Prognosis Logic

• [x] E2E_02_03_03_OEM_CALL_REMOTE_FAIL_INVALID_GRAPH OEM Checks Successful Unsuccessful Calling SUPPLIER Prognosis Logic (Wrong Address)

• [x] E2E_02_03_04_OEM_CALL_REMOTE_FAIL_INVALID_SYNTAX OEM Checks Successful Unsuccessful Calling SUPPLIER Prognosis Logic (Invalid Query)

• [x] E2E_02_03_05_OEM_CALL_REMOTE_FAIL_NS OEM Checks Successful Unsuccessful Calling SUPPLIER Prognosis Logic (Non-Matched Semantics)

• [x] E2E_02_04_01_PROVIDER_FEDERATED_CATALOGUE PROVIDER Checks OEM Information in Federated Catalogue

• [x] E2E_02_04_02_PROVIDER_CALL_REMOTE PROVIDER Checks Successful Calling OEM Telematics Data

• [x] E2E_02_04_03_PROVIDER_CALL_REMOTE_FAIL_INVALID_GRAPH PROVIDER Checks Successful Unsuccessful Calling OEM Telematics Data (Wrong Address)

• [x] E2E_02_04_04_PROVIDER_CALL_REMOTE_FAIL_INVALID_SYNTAX PROVIDER Checks Successful Unsuccessful Calling OEM Telematics Data (Invalid Query)

• [x] E2E_02_04_05_PROVIDER_CALL_REMOTE_FAIL_NS PROVIDER Checks Successful Unsuccessful Calling OEM Telematics Data (Non-Matched Semantics)

• [x] E2E_02_06_01_LIST_SKILLS PROVIDER lists all required skills for the E2E Test

• [x] E2E_02_06_02_AAS_DESCRIPTION PROVIDER successfully lists AAS capabilties

• [x] [E2E_02_06_03_AAS_ALL_SHELLS](https://www.postman.com/catena-x/workspace/catena-x-knowledge-agents/request/2757771-e5ffecc4-e3bf-4561-b512-4c535afa5e38?action=share&source=copy-link&creator=2757771&ctx=documentation PROVIDER successfully lists all twins inside Traceability Graph

• [x] E2E_02_06_04_AAS_ALL_SUBMODELS PROVIDER successfully lists all submodels inside Traceability Graph

• [x] E2E_02_06_05_ONE_SHELL PROVIDER successfully looks up single twin from the Traceability Graph

• [x] E2E_02_06_06_NOTEXISTS_SHELL PROVIDER fails looking up non-existing twin

• [x] E2E_02_06_07_ONE_SUBMODEL PROVIDER successfully looks up single submodel from the Traceability Graph

• [x] E2E_02_06_07_NOTEXISTS_SUBMODEL PROVIDER fails looking up non-existing submodel

CONSUMER Test

• [x] E2E_02_02_05_REGISTER_RUL_SKILL_CONSUMER CONSUMER Creates a Non-Federated RUL Skill
• [x] E2E_02_05_01_RUL_SKILL_ADHOC CONSUMER Calls an Ad-hoc Skill Over OEM and SUPPLIER
• [x] E2E_02_05_02_RUL_SKILL CONSUMER Calls Own Predefined RUL Skill
• [x] E2E_02_05_03_RUL_SKILL_WRONG_VEHICLES CONSUMER Calls Own Predefined RUL Skill with an invalid parameterization
• [x] E2E_02_05_04_RUL_SKILL_REMOTE_IMPLICIT CONSUMER Calls PROVIDER Skill (Implicit Mode)
• [x] E2E_02_05_05_RUL_SKILL_REMOTE_EXPLICIT CONSUMER Calls PROVIDER Skill (Explicit Mode)
• [x] E2E_02_05_06_RUL_SKILL_REMOTE_WRONG_MODE CONSUMER Calls PROVIDER Skill (Wrong Mode)
• [x] E2E_02_05_07_RUL_SKILL_LOCAL_IMPLICIT CONSUMER Calls PROVIDER Skill (Implicit Mode)
• [x] E2E_02_05_08_RUL_SKILL_LOCAL_EXPLICIT CONSUMER Calls PROVIDER Skill (Explicit Mode)
• [x] E2E_02_05_09_RUL_SKILL_LOCAL_WRONG_MODE CONSUMER Calls PROVIDER Skill (Wrong Mode)
• [x] E2E_02_05_10_BT_SKILL_RUL CONSUMER Calls Polymorphic OEM Skill (Variant 1)
• [x] E2E_02_05_11_BT_SKILL_HI CONSUMER Calls Polymorphic OEM Skill (Variant 2)

Hints

Information Sharing

[drcgjung]

PROVIDER Test Execution (3 Tests Failed)

• [x] E2E_02_00_01_OEM_GRAPH_POLICY OEM Creates a Policy for Providing Graphs and Skills to the E2E participants

• [x] E2E_02_00_02_OEM_GRAPH_CONTRACT OEM Creates a Contract Definition for Providing Graphs and Skills to the E2E participants

• [x] E2E_02_00_03_OEM_RELIABILITY_GRAPH OEM Creates a Graph Asset with Telematics Data

• [x] E2E_02_01_00_SUPPLIER_GRAPH_POLICY SUPPLIER Creates a Policy for Providing Graphs to the E2E participants

• [x] E2E_02_01_01_SUPPLIER_GRAPH_CONTRACT SUPPLIER Creates a Contract Definition for Providing Graphs to the E2E participants

• [x] E2E_02_01_02_SUPPLIER_BEHAVIOUR_GRAPH SUPPLIER Creates a Graph Asset with RUL Prognosis

• [x] E2E_02_01_03_SUPPLIER_BEHAVIOUR_GRAPH_VARIANT SUPPLIER Creates a Graph Asset with HI Prognosis

• [x] E2E_02_02_01_OPEN_POLICY PROVIDER Creates a Permissive Policy

• [x] E2E_02_02_02_OPEN_CONTRACT PROVIDER Creates a Permissive Contract Definition

• [ ] E2E_02_02_03_REGISTER_RUL_SKILL_REMOTE PROVIDER Creates a RUL Skill in PROVIDER Mode

• [ ] E2E_02_02_04_REGISTER_RUL_SKILL_LOCAL PROVIDER Creates a RUL Skill in CONSUMER Mode

• [ ] E2E_02_02_06_REGISTER_BEH_SKILL_PROVIDER OEM Creates a Behaviour Skill in PROVIDER Mode

• [x] E2E_02_03_01_OEM_FEDERATED_CATALOGUE OEM Checks SUPPLIER Information in Federated Catalogue

• [x] E2E_02_03_02_OEM_CALL_REMOTE OEM Checks Successful Calling SUPPLIER Prognosis Logic

• [x] E2E_02_03_03_OEM_CALL_REMOTE_FAIL_INVALID_GRAPH OEM Checks Successful Unsuccessful Calling SUPPLIER Prognosis Logic (Wrong Address)

• [x] E2E_02_03_04_OEM_CALL_REMOTE_FAIL_INVALID_SYNTAX OEM Checks Successful Unsuccessful Calling SUPPLIER Prognosis Logic (Invalid Query)

• [x] E2E_02_03_05_OEM_CALL_REMOTE_FAIL_NS OEM Checks Successful Unsuccessful Calling SUPPLIER Prognosis Logic (Non-Matched Semantics)

• [x] E2E_02_04_01_PROVIDER_FEDERATED_CATALOGUE PROVIDER Checks OEM Information in Federated Catalogue

• [x] E2E_02_04_02_PROVIDER_CALL_REMOTE PROVIDER Checks Successful Calling OEM Telematics Data

• [x] E2E_02_04_03_PROVIDER_CALL_REMOTE_FAIL_INVALID_GRAPH PROVIDER Checks Successful Unsuccessful Calling OEM Telematics Data (Wrong Address)

• [x] E2E_02_04_04_PROVIDER_CALL_REMOTE_FAIL_INVALID_SYNTAX PROVIDER Checks Successful Unsuccessful Calling OEM Telematics Data (Invalid Query)

• [x] E2E_02_04_05_PROVIDER_CALL_REMOTE_FAIL_NS PROVIDER Checks Successful Unsuccessful Calling OEM Telematics Data (Non-Matched Semantics)

• [x] E2E_02_06_01_LIST_SKILLS PROVIDER lists all required skills for the E2E Test

• [x] E2E_02_06_02_AAS_DESCRIPTION PROVIDER successfully lists AAS capabilties

• [x] [E2E_02_06_03_AAS_ALL_SHELLS](https://www.postman.com/catena-x/workspace/catena-x-knowledge-agents/request/2757771-e5ffecc4-e3bf-4561-b512-4c535afa5e38?action=share&source=copy-link&creator=2757771&ctx=documentation PROVIDER successfully lists all twins inside Traceability Graph

• [x] E2E_02_06_04_AAS_ALL_SUBMODELS PROVIDER successfully lists all submodels inside Traceability Graph

• [x] E2E_02_06_05_ONE_SHELL PROVIDER successfully looks up single twin from the Traceability Graph

• [x] E2E_02_06_06_NOTEXISTS_SHELL PROVIDER fails looking up non-existing twin

• [x] E2E_02_06_07_ONE_SUBMODEL PROVIDER successfully looks up single submodel from the Traceability Graph

• [x] E2E_02_06_07_NOTEXISTS_SUBMODEL PROVIDER fails looking up non-existing submodel

[drcgjung]

CONSUMER Test Execution (1 Test failed)

• [ ] E2E_02_02_05_REGISTER_RUL_SKILL_CONSUMER CONSUMER Creates a Non-Federated RUL Skill

• [X] E2E_02_05_01_RUL_SKILL_ADHOC CONSUMER Calls an Ad-hoc Skill Over OEM and SUPPLIER

• [x] E2E_02_05_02_RUL_SKILL CONSUMER Calls Own Predefined RUL Skill

• [x] E2E_02_05_03_RUL_SKILL_WRONG_VEHICLES CONSUMER Calls Own Predefined RUL Skill with an invalid parameterization

• [x] E2E_02_05_04_RUL_SKILL_REMOTE_IMPLICIT CONSUMER Calls PROVIDER Skill (Implicit Mode)

• [x] E2E_02_05_05_RUL_SKILL_REMOTE_EXPLICIT CONSUMER Calls PROVIDER Skill (Explicit Mode)

• [x] E2E_02_05_06_RUL_SKILL_REMOTE_WRONG_MODE CONSUMER Calls PROVIDER Skill (Wrong Mode)

• [x] E2E_02_05_07_RUL_SKILL_LOCAL_IMPLICIT CONSUMER Calls PROVIDER Skill (Implicit Mode)

• [x] E2E_02_05_08_RUL_SKILL_LOCAL_EXPLICIT CONSUMER Calls PROVIDER Skill (Explicit Mode)

• [x] E2E_02_05_09_RUL_SKILL_LOCAL_WRONG_MODE CONSUMER Calls PROVIDER Skill (Wrong Mode)

• [x] E2E_02_05_10_BT_SKILL_RUL CONSUMER Calls Polymorphic OEM Skill (Variant 1)

• [x] E2E_02_05_11_BT_SKILL_HI CONSUMER Calls Polymorphic OEM Skill (Variant 2)

[drcgjung]

PROVIDER Retest

• [x] E2E_02_02_03_REGISTER_RUL_SKILL_REMOTE PROVIDER Creates a RUL Skill in PROVIDER Mode

• [x] E2E_02_02_04_REGISTER_RUL_SKILL_LOCAL PROVIDER Creates a RUL Skill in CONSUMER Mode

• [x] E2E_02_02_06_REGISTER_BEH_SKILL_PROVIDER OEM Creates a Behaviour Skill in PROVIDER Mode

[drcgjung]

CONSUMER Retest

• [x] E2E_02_02_05_REGISTER_RUL_SKILL_CONSUMER CONSUMER Creates a Non-Federated RUL Skill

• [x] E2E_02_05_07_RUL_SKILL_LOCAL_IMPLICIT CONSUMER Calls PROVIDER Skill (Implicit Mode)

• [x] E2E_02_05_08_RUL_SKILL_LOCAL_EXPLICIT CONSUMER Calls PROVIDER Skill (Explicit Mode)