HaxeFoundation / hxcpp

Runtime files for c++ backend for haxe
Other
295 stars 184 forks source link

Mac CI takes an age to run #1121

Closed Aidan63 closed 2 months ago

Aidan63 commented 2 months ago

Compiling the haxe test suite on macos runner is very slow compared to Windows and Linux.

This seems to be a github issue (which they are very quiet about) so I don't know if there's anything that can be done.

https://github.com/actions/runner-images/issues/1336

Probably not too much of an issue in normal operation, but it in situations like we have now where the CI queue resembles Clapham Junction, that task appears to be a bottleneck.

tobil4sk commented 2 months ago

Could this be because it's emulating x86_64 on arm64 (because we're using macos-latest)? We could check if it's faster if we go back to macos-13 which should still be x86_64 by default.

Aidan63 commented 2 months ago

I just had a look at the results from the commit from before I moved it over to macos-latest. On macos-13 the haxe test suite takes around 15mins to run.

https://github.com/HaxeFoundation/hxcpp/actions/runs/9553228673/job/26331610744

The slow part is compiling all the C++, is the clang executable going through the emulation layer as well when it gets spawned from the hxcpp build tool (assuming apple clang is native arm in the first place)?

tobil4sk commented 2 months ago

Setup haxe installs x86_64 versions of haxe and neko, so neko is running in the x86_64 environment and I guess that means any of the processes it calls will also be x86_64.

We could fix this by installing arm64 versions of haxe and neko from homebrew, or by adding arm64 support for setup-haxe but the haxelib binary is not yet built properly for arm64 (see https://github.com/HaxeFoundation/haxe/pull/11663). Or we could revert back to macos-13 for now.

Aidan63 commented 2 months ago

Might be best to switch back to macos-13 for now, and once all the links in the chain have arm versions try again and see what the performance is like.

tobil4sk commented 2 months ago

The nightly builds are now fully universal (https://github.com/HaxeFoundation/haxe/pull/11663 was merged)! So hopefully that might give us a speed up.

Aidan63 commented 2 months ago

Quickly bodge setup-haxe to use the new neko release candidates and building the haxe test suite has now gone done to 6mins, which is a nice speed up. I'm reasonably confident this is all using native arm versions as the cffi test is failing with an architecture mismatch error.

https://github.com/Aidan63/hxcpp/actions/runs/9799095232

Full run results for reference.

So whats the correct version matchups? Can this new neko release be used for all haxe 4.x releases and going forward? The 4.3.4 building was still very slow as I'm guessing thats not a universal binary? I've used lix for years now so all knowledge of how these things work together has long since leaked out of my head.

tobil4sk commented 2 months ago

I'm reasonably confident this is all using native arm versions as the cffi test is failing with an architecture mismatch error.

https://github.com/Aidan63/hxcpp/blob/d4e8d40f2894515617146b5703a8a02efa8cdf41/.github/workflows/test.yml#L55

I think HXCPP_M64 is forcing x86 here. Passing HXCPP_ARM64 (or nothing) instead should build the correct ndll.

Can this new neko release be used for all haxe 4.x releases and going forward?

Yes, there shouldn't be issues with that.

The 4.3.4 building was still very slow as I'm guessing thats not a universal binary?

It's not, but the next release (4.3.5) should come with universal binaries.