bytedeco / javacpp-presets

The missing Java distribution of native C++ libraries
Other
2.68k stars 744 forks source link

Missing PyTorch MPS support in CI builds due to older SDK #1464

Open sbrunk opened 10 months ago

sbrunk commented 10 months ago

I just gave the macosx-arm64 PyTorch from the CI build a try just to realize that GPU support via MPS is missing. The reason seems that we need a sdk version >= 12.3 but the build is using 12.1:

https://github.com/pytorch/pytorch/blob/af8f37c2b6f9f670ee8154e81ebd9fce7843f60e/CMakeLists.txt#L117C1-L118C46

        if(_macosx_sdk_version VERSION_GREATER_EQUAL 12.3)
            set(_MPS_supported_os_version ON)

https://github.com/bytedeco/javacpp-presets/actions/runs/7650509878/job/20880141743#step:2:2541

-- sdk version: 12.1, mps supported: OFF

It also seems like I'm just in time to have missed the release deadline. 🙈

saudet commented 10 months ago

That new version of the SDK is probably available only in macos 12.x right? I'm not going to do that for this release anyway

sbrunk commented 10 months ago

Yeah it looks like 12.1 is the latest version available in the 11 runner: https://github.com/actions/runner-images/blob/main/images/macos/macos-11-Readme.md#installed-sdks Higher versions up to 13.1 will need 12: https://github.com/actions/runner-images/blob/main/images/macos/macos-12-Readme.md#installed-sdks

sbrunk commented 10 months ago

I've created a branch from current master & the zlib 1.3.1 commit from the release branch and did an update to the macOS 12 runner image to get a first impression of how many presets are breaking.

Looking through the results, we get 4 presets that are failing on macOS 12 (without failing on all other platforms).

Other builds either fail on all platforms (skia and mxnet) or only on linux-arm (seems like a mirror is down) so I'm assuming these are unrelated.


I have a hunch for scipy: gfortran-10 is part for the macOS 11 image, but the macOS 12 image starts from gfortran-11. ~I'll try to add gfortran-10 back in the deploy-macosx workflow.~ It's part of gcc so it will still be installed already with gcc@10

Perhaps I should also remove gcc-13?

saudet commented 10 months ago

No, please use GCC 11. Newer versions of GCC tend to be too buggy to be usable. We'll need to update all these lines too: https://github.com/bytedeco/javacpp-presets/blob/master/.github/actions/deploy-macosx/action.yml#L44-L60

saudet commented 10 months ago

Ah, yes, please remove GCC 13. Are you saving GCC 10 is still available? If that still works that's OK, yes

sbrunk commented 10 months ago

Yes it looks like GCC 10 just not installed by default anymore in the new image but it can still be installed via Homebrew like it's already done. I'm doing another run with GCC 13 removed.