search

compiler-research / CppInterOp

A Clang-based C++ Interoperability Library
Other
37 stars 20 forks source link

Add CI for OSX and Windows #138

Open vgvassilev opened 10 months ago

vgvassilev commented 10 months ago

Update our build infrastructure to include OSX and Windows builds.

mcbarton commented 7 months ago

My suggested improvements that could be made to the Github ci workflow for macos are

I know this issue is about ci for Macos and Windows, but my suggested improvements that could be made to the Ubuntu Github ci workflow are

vgvassilev commented 7 months ago

My suggested improvements that could be made to the Github ci workflow for macos are

* Currently making use macOS version 12 runner. Make use of macos-13 github runner (currently in beta). Latest version of macOS is version 14 for comparison.
  https://github.com/actions/runner-images/tree/main - list of runners available
  https://github.com/actions/runner-images/blob/main/images/macos/macos-13-Readme.md - Readme on runner

That makes sense to me.

* By default the macos github runners use 3 cores. There is a 12 core version which came out this year
  https://github.blog/2023-03-01-github-actions-introducing-faster-github-hosted-x64-macos-runners/ - blog post about release of 12 core x86 macos runners
  https://github.blog/2023-10-02-introducing-the-new-apple-silicon-powered-m1-macos-larger-runner-for-github-actions/ - blog post about release of 12 core arm64 macos runners

We do not need a many core machine for that project. We rely on extensive caching of our llvm builds and that works mostly well.

I know this issue is about ci for Macos and Windows, but my suggested improvements that could be made to the Ubuntu Github ci workflow are

* ubu22-gcc9-clang-repl-16 workflow currently fails to load lubcudart.so . My assumption is that is because it is not running a machine with a nvidia gpu. Github recently started a limited beta on GPU enabled Github runners. It is not clear whether these are nvidia or amd based gpus, but something to consider.

Do you mean that fail to load as part of the tests? If so, this is our way to check if there was cuda runtime (eg. if there is a cuda gpu).

  [Actions: GPU enabled GitHub-hosted runners (Limited Beta) github/roadmap#505](https://github.com/github/roadmap/issues/505) - Roadmap gpu enabled Github runners, with link to sign up to beta

* Github runners currently run on x86 machines I believe. May be worth making a workflow which runs on Arm based machines. See link below on how to join waiting list
  https://resources.github.com/devops/accelerate-your-cicd-with-arm-and-gpu-runners-in-github-actions/ - Link to where you can sign up to arm based Github runners

I joined the waiting list - let's see what happens...

mcbarton commented 7 months ago

Do you mean that fail to load as part of the tests? If so, this is our way to check if there was cuda runtime (eg. if there is a cuda gpu).

I was referring to the Github CI, job ubu22-gcc9-clang-repl-16 under 'Build and Test/Install CppInterOp on Unix systems' it fails to load libcudart.so . This is to be expected since they won't have a cuda gpu. I was trying to suggest looking into whether the GPU enabled Github-hosted runners which are in a limited beta have cuda gpus, so that any cuda tests can be run on the Github CI in future.

mcbarton commented 5 months ago

I think that once the PR https://github.com/compiler-research/CppInterOp/pull/181 is merged that this issue should be closed, as its about updating the build infrastructure for Windows and osx. Separate issues regarding the ci could be raised instead, such as activate arm builds on ci, adding cppyy Windows builds, etc.

mcbarton commented 5 months ago

@vgvassilev According to this PR raised in the runner-images repo https://github.com/actions/runner-images/pull/9179 a Github runner using macos-14 will become available very soon (the latest version). Would you like me to replace the osx13 jobs with an osx14 ones after it becomes available, or add them as separate jobs alongside osx13?

vgvassilev commented 5 months ago

Probably that’s a good idea.

mcbarton commented 4 months ago

I joined the waiting list - let's see what happens...

@vgvassilev Did this project get access to the non osx arm Github runners in the end after you joined the waiting list?

Also the CI is currently failing on the main branch after the wasm CI merge, as it's trying to make use of the llvm build which was made on the main branch, before the merge occurred (this didn't have certain options disable which are incompatible with the wasm build). If you clear the cache it will pass doing the next scheduled CI run.

vgvassilev commented 4 months ago

I joined the waiting list - let's see what happens...

@vgvassilev Did this project get access to the non osx arm Github runners in the end after you joined the waiting list?

I have not heard back from them.

Also the CI is currently failing on the main branch after the wasm CI merge, as it's trying to make use of the llvm build which was made on the main branch, before the merge occurred (this didn't have certain options disable which are incompatible with the wasm build). If you clear the cache it will pass doing the next scheduled CI run.

Done!