search

open-quantum-safe / liboqs

C library for prototyping and experimenting with quantum-resistant cryptography
https://openquantumsafe.org/
Other
1.87k stars 459 forks source link

Expand weekly test runs to platforms other than x86_64 / Linux #1674

Open SWilson4 opened 9 months ago

SWilson4 commented 9 months ago

After https://github.com/open-quantum-safe/liboqs/pull/1560 lands, the full KAT tests will be run on a weekly basis. Our weekly jobs currently (the KAT tests and the constant-time tests) only run on x86_64 / Linux at the moment. It would be nice to also run the tests on aarch64, Windows, MacOS, etc.

planetf1 commented 8 months ago

I'm not familar with circleci. So I think only x86_64 is used for the current github workflows? They do provide runners for some of these options.

Some platforms I don't see include

baentsch commented 8 months ago

So I think only x86_64 is used for the current github workflows?

Correct. And that (desire to test on other HW) is the primary reason why we still have to support CircleCI.

need to find someone to provide a suitable pool?

Correct. That was one of the prime motivations for OQS to join LF (presumably easier access to more platforms for CI). Right now, those IBM platforms are tested via Travis. And we'd love to streamline our CI to one system (incl. getting rid of CCI), so any contribution very welcome.

bhess commented 8 months ago

Some platforms I don't see include

  • ubuntu on arm (aarch64)
  • Z (s390x)

And we'd love to streamline our CI to one system

Moving everything to one CI would be great. @planetf1, as Michael says we currently have basic s390x and ppc64le CI with travis, which is sometimes a bit flaky (I have a ticket open with Travis about the intermittent errors). There is work to allow self-hosted runners on linux-s390x that might be worth to follow: https://github.com/actions/runner/issues/2263.

.. or https://github.com/marketplace/actions/run-on-architecture which seems to support both s390x, ppc64le (and more).

planetf1 commented 6 months ago

When the pqca was forming we did get a post from a github employee suggesting we might be able to get some extra help with github

SWilson4 commented 6 months ago

I have talked to GitHub about getting into the Arm private beta, but it was pending some administrative work to get our account upgraded to Enterprise. Pinging @ryjones for any updates on that front.

bhess commented 6 months ago
  • There is an unofficial version of github action runner as mentioned above, which seems current, and now supports (experimental...!) s390x. There is hope this could get integrated by github. maybe we can point them to this ? (@bhess presumably we could find some s390x to actually run on?)

This would be great @planetf1 and IMO the way to go when consolidating to Github CI. I also tried an experimental build using run-on-architecture and s390x.. it works but it's terribly slow because it's emulated. So not really an option, also in light of the project's goal for responsible CI use.

ryjones commented 6 months ago

@SWilson4 no follow up, yet. I'm as annoyed as you are :) @bhess if your s390x vms/etc are available to travis, it should be easy enough to hook them into github.

SWilson4 commented 6 months ago

I have talked to GitHub about getting into the Arm private beta, but it was pending some administrative work to get our account upgraded to Enterprise. Pinging @ryjones for any updates on that front.

GitHub folks contacted me to say that PQCA is now on GitHub Enterprise, so I'm hopeful that Arm runners are coming soon.

ryjones commented 6 months ago

@SWilson4 working on it :) Right now we have large runners, but not arm (yet)