search

rh-openjdk / CryptoTest

Complete example of java crypto API
MIT License
3 stars 9 forks source link

cryptotest/CryptoTest.java fails by openjdk main-line repo #72

Open sendaoYan opened 2 weeks ago

sendaoYan commented 2 weeks ago

The test log snippet:

Total checks: 1, failed: 1
Some algorithms missed! Checked 704 from 707
FAILED: cryptotest.CryptoTest$NoAlgorithmMissed
----------------------------------
FAILED: cryptotest.CryptoTest$NoAlgorithmMissed
Test runs: 35; failed: 1
STDERR:
java.lang.RuntimeException: Some tests failed: 1
    at cryptotest.CryptoTest.main(CryptoTest.java:172)

CryptoTest.java.log

judovana commented 2 weeks ago

hi!

My localo runs are passing. from log I can see it is custom build of some jdk/ What jdk? How it was built? Your jtreg-7.4.1 is what origin? How did you treated AGENT variables? Have you run trhough run.sh or differently? The verbosity of output in xml depends on jtreg verison/origin because of recent changes. to jtreg/jtharness projects. But you have not specified xml output, which suggests you run via your custom command. What was your workdir? The run.sh have workday in test.173... , which should contain .jtr file of major importance.

NoAlgorithmMissed.jtr or CryptoTest.jtr or generally .*jtr cotnains info from single run(s) you should be bale to narrow the check:

AGENT_HOSTNAME=dead.dead sh run.sh /usr/lib/jvm/java-17-openjdk 6666666
~/git/CryptoTest ~/git/CryptoTest
Non cygwin system!
/home/jvanek/git/CryptoTest
treating jdk as: 17
Running with /usr/lib/jvm/java-17-openjdk...
XML output  to /home/jvanek/git/CryptoTest/test.1730882596/jdk/work
Passed: cryptotest/tests/TestServices.java
FAILED: cryptotest/CryptoTest.java
Test results: passed: 1; failed: 1
Report written to /home/jvanek/git/CryptoTest/test.1730882596/jdk/report/html/report.html
Results written to /home/jvanek/git/CryptoTest/test.1730882596/jdk/work
Error: Some tests failed or other problems occurred.
~/git/CryptoTest

or

AGENT_HOSTNAME=some.existing.one sh run.sh /usr/lib/jvm/java-17-openjdk 6666666
~/git/CryptoTest ~/git/CryptoTest
Non cygwin system!
/home/jvanek/git/CryptoTest
treating jdk as: 17
Running with /usr/lib/jvm/java-17-openjdk...
XML output  to /home/jvanek/git/CryptoTest/test.1730882747/jdk/work
Passed: cryptotest/tests/TestServices.java
Passed: cryptotest/CryptoTest.java
Test results: passed: 2
Report written to /home/jvanek/git/CryptoTest/test.1730882747/jdk/report/html/report.html
Results written to /home/jvanek/git/CryptoTest/test.1730882747/jdk/work
~/git/CryptoTest

or

SKIP_AGENT_TESTS=true sh run.sh /usr/lib/jvm/java-17-openjdk 6666666
~/git/CryptoTest ~/git/CryptoTest
Non cygwin system!
/home/jvanek/git/CryptoTest
treating jdk as: 17
Running with /usr/lib/jvm/java-17-openjdk...
XML output  to /home/jvanek/git/CryptoTest/test.1730882901/jdk/work
Passed: cryptotest/tests/TestServices.java
Passed: cryptotest/CryptoTest.java
Test results: passed: 2
Report written to /home/jvanek/git/CryptoTest/test.1730882901/jdk/report/html/report.html
Results written to /home/jvanek/git/CryptoTest/test.1730882901/jdk/work
~/git/CryptoTest

Note the difference in agent handling (but you have -Dcryptotests.skipAgentTests=1, so you shodl be ok). I see it passing also on jdk is 23 but had not tried current head.

The NoAlgorithmMissed is slightly special, as it test, that any algorithm found iun jdk, was tested by all other tests. From info above I posted, do you think you will be able to dig up what algorithms were missed? jdk24 or your own crypto impl for your custom jdk may have new algortihms which woudl require new tests to be added.

Please try to dig it out and ask here, I willt ry to navigate. the minimal output may be improvement of README.md so those issues are prevented in future.

Especially the cryptotest.CryptoTest$NoAlgorithmMissed, when manually added, is definitely candidate to produce much better output. However in correct hadnling (https://github.com/rh-openjdk/CryptoTest/blob/11d731ec6682bcf90dd4a871359ed623ac3ad8a2/cryptotest/CryptoTest.java#L148) it prints at lesat how much were missed.

Thanx!

sendaoYan commented 2 weeks ago

Hi, I think this test fails since below jdk main-line repo commit:

commit 2a1ae0ff89a8ac364206b09059d9dc884adcc5ac
Author: Kevin Driver <kdriver@openjdk.org>
Date:   Tue Nov 5 21:07:52 2024 +0000

    8331008: Implement JEP 478: Key Derivation Function API (Preview)
judovana commented 2 weeks ago

Oh I see. Ok. Thanx a lot! I will try head. This will most likely need new test-case to be added. This will have a bit longer iteration as I need to find time to dive into it. In case you are familiar with the api, feel free to elaborate. Again thanx a lot for report.