search

ruby / openssl

Provides SSL, TLS and general purpose cryptography.
Other
240 stars 167 forks source link

Rakefile: Manage test files by excluding test files in the test_fips task. #820

Closed junaruga closed 7 hours ago

junaruga commented 9 hours ago

Until now, we have added 18 test files as target files executed in the test_fips task to the Rakefile. As there are total 33 test files, more than 50% (= 18/33) of the test files have been added.

Now I want to manage the test files in the test_fips task by excluding test files rather than adding the target test files in the test_fips task. About the excluded test files, I mentioned at https://github.com/ruby/openssl/issues/603#issuecomment-2491878176.

My motivations for this change are

  1. We can test FIPS cases for the test files to be added in the future.
  2. We can clarify the remaining failing test files that we need to fix.
junaruga commented 9 hours ago

As a test for this PR, below is the CI test result in OpenSSL master fips on the latest master branch.

https://github.com/ruby/openssl/actions/runs/11959473510/job/33341351372#step:11:453

249 tests, 1523 assertions, 0 failures, 0 errors, 2 pendings, 22 omissions, 0 notifications

And below is the CI test result in OpenSSL master fips on this PR. And both results are the same, and it is expected.

https://github.com/ruby/openssl/actions/runs/11973989540/job/33384106630?pr=820#step:11:453

249 tests, 1523 assertions, 0 failures, 0 errors, 2 pendings, 22 omissions, 0 notifications
junaruga commented 6 hours ago

That sounds like a good way forward, and as far as I understand the plan is to eventually make rake test compatible with it. Thanks!

Yes. Your understanding is right. Eventually we will make the rake test compatible with FIPS cases, or the rake test and rake test_fips calls an internal common logic.

Thanks for your reviewing!