Kyma with Compass test function should be reachable through secured API Rule

[Sawthis]

Description

The periodic kyma-integration-k3d-compass-dev job is constantly failing.

Expected result

The job is green again and we have 0 failures in last 24 hours.

Actual result

It failed 24 times in last 24 hours.

Steps to reproduce

Run the periodic kyma-integration-k3d-compass-dev job.

Troubleshooting

[Disper]

There is an issue on the compass side that is being tracked on some internal compass tooling that will not link here. If anyone is interested in more information, please reach either me or @franpog859.

[franpog859]

The Error when updating label definition with key scenarios: Error: Error calling Director API: 422 Unprocessable Entity Error problem I was working on is fixed now. I'm unassigning myself from this issue for now, @Sawthis

[Disper]

Pasting additional info that @franpog859 added on the current issue on Slack:

function should be reachable through secured API Rule: Kyma with Compass test function should be reachable through secured API Rule

{expected { Object (event, podName) } to have nested property 'order.totalPriceWithTax.value' AssertionError AssertionError: expected { Object (event, podName) } to have nested property 'order.totalPriceWithTax.value'
    at checkFunctionResponse (test/fixtures/commerce-mock/index.js:137:35)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at async Context.<anonymous> (compass-test/compass-test.js:46:5)}

I will edit the title to reflect the current problem.

[Disper]

As the function is not reachable and api gateway exposes it (thx @nachtmaar for help with narrowing the issue down), can it be something that @wookiee or @goat would be able to know more about?

[kwiatekus]

events are triggering the last.order function

    ✓ order.created.v1 legacy event should trigger the lastorder function
    ✓ order.created.v1 cloud event in structured mode should trigger the lastorder function
    ✓ order.created.v1 cloud event in binary mode should trigger the lastorder function

That means function is up and running. Looks like its not properly exposed via API rule. Or its just hiding another problem. Assertion error says also:

AssertionError: expected { Object (event, podName) } to have nested property 'order.totalPriceWithTax.value'

Could have the spec of the event changed?

[nachtmaar]

I have a cluster dump from a failed execution, please ping me and I will be happy to share it :)

Hint for me where I find it

~/SAPDevelop/tickets/c-8aca7a6/

[nachtmaar]

This is the code line: https://github.com/kyma-project/kyma/blob/main/tests/fast-integration/test/fixtures/commerce-mock/index.js#L136 I will try to log out the actual value in res.data. That might help

[nachtmaar]

Logged out the error and it maches the output from above (AssertionError: expected { Object (event, podName) } to have nested property 'order.totalPriceWithTax.value'). Looks like the return value of the function is not correct.

console.log(JSON.stringify(res.data, null, 2))
{
  "event": {},
  "podName": "lastorder-n8r82-556666bfc5-kxvmf"
}

[kwiatekus]

For some reasons the last-order function in this scenario does not return proper order data in the response and hence the assertion fails (not consistently - its flaky)

Please refactor the test so it throes better errors. To be honest, the compass-test doesnt tell a story. It hides a lot of details and creates false impression that api-gateway or serverless doesnt work. What we read is an error "function should be reachable through secured API Rule" - but the function does respond via API rule, but the content of the response doesnt meet the criteria of the assertion. The response data is built by the function logic which depends on many 3rd party components (function sends an event towards a commerce mock application that is registered via compass)

IMO @kyma-project/framefrog or the compass team should investigate the flakiness

[kwiatekus]

As discussed with @pbochynski and @janmedrek this test could be dropped. Each team should have solid test of their integrations and be accountable for them. This test is a js based integration test that suffers from lack of ownership and flakiness

[janmedrek]

The pipeline has been removed.