search

adoptium / aqa-tests

Home of test infrastructure for Adoptium builds
https://adoptium.net/aqavit
Apache License 2.0
131 stars 314 forks source link

CryptoTests are reporting GssApiMechanismTests and SaslServerFactoryTests as passed, where it should be skipped #5246

Open judovana opened 6 months ago

judovana commented 6 months ago

Those two tests - SaslServerFactoryTests and GssApiMechanismTests are relying on remote kerberos server, which is disabled in Aqa-tests by SKIP_AGENT_TESTS/cryptotests.skipAgentTests

However setting up this property/variable is making the tests silently pass, instead of ignoring them. I will be fixing this in upstream today. It thsould not affect the aqavit runs, onluy those two tests will move from passed to ignored.

However I would like to enable those two tests. The kerberos server it needs, have accidentaly same setup as kerberos server for tck. Is there any available ?

In both cases, I would like to improve a bit the aqavit crypto test targets, so they allow the passing of both AGENT_HOSTNAME and SKIP_AGENT_TESTS inside, so anybody who have kerbeors server can point Cryptotest targets to it.

In addition, this server can be setup locally by morevoer simple script - if all depndences are satisfied. Thus I was thinking, about creating an external aqa-tests test, which would run only those two tests, and will prepare the local kerberos server inside the container. WDYT @smlambert @llxia @zzambers @annaibm @pshipton?

judovana commented 6 months ago

The https://github.com/rh-openjdk/CryptoTest/pull/68 is merged, so the two tests should now disapear fro all runs which do have agent enabled.

smlambert commented 6 months ago

Due to legal requirements, our TCK tests run in private and the kerberos server is also not publicly available.

We had previously talked about adding a feature to run a server in a container and made available for the duration of tests that require it.

Let's design an approach that can be used for many scenarios where any test target that needs a containerized server can use the same method.

judovana commented 6 months ago

The five tests now shoudl be properly ignored. Will think about the kerberos localhost a bit