search

actions / runner-images

GitHub Actions runner images
MIT License
10.24k stars 3.08k forks source link

[macOS] OpenSSL 1.1 will be removed and OpenSSL 3 will be the default for all macOS images once [date not determined] #10817

Open sarathrajsrinivasan opened 1 month ago

sarathrajsrinivasan commented 1 month ago

Breaking changes

OpenSSL 1.1 will be removed from all macOS images and the default version will be switched to OpenSSL 3.

Target date

⚠️ We will notify you of planned changes when exact dates are set. ⚠️

The motivation for the changes

OpenSSL 1.1 has reached its end-of-life (EOL) and is no longer supported. Due to its deprecation, Homebrew has removed it from its available packages.

Possible impact

If your library/project is not compatible with OpenSSL 3 and you build them based on default OpenSSL version - your workflow will be broken.

Platforms affected

Runner images affected

Mitigation ways

Use the below step in your workflow to download and install OpenSSL 1.1 manually

    - name : Install  OpenSSL 1.1 manually
      run : |
        wget https://www.openssl.org/source/openssl-1.1.1w.tar.gz
        tar -xvf openssl-1.1.1w.tar.gz
        cd openssl-1.1.1w
        ./config --prefix=/usr/local/openssl
        make
        sudo make install
        export PATH=/usr/local/openssl/bin:$PATH
eregon commented 2 weeks ago

This means currently some runs on macos-14 have openssl@1.1 and some don't: https://github.com/eregon/setup-ruby/actions/runs/11724213556/job/32657603152

This is quite problematic because if something is built on macos-14 on a run with openssl@1.1 (and the software links to that) it can fail when that's later used in another run as openssl@1.1 /opt/homebrew/opt/openssl@1.1/lib/libssl.1.1.dylib is gone.

Basically, anything that ever linked to openssl@1.1 on macos-* will be broken by this change (until it's recompiled against openssl@3).

jorupp commented 2 weeks ago

Is there a way to configure our build so we only run on the OpenSSL@3-equipped images? We can't add the -legacy flag we need for OpenSSL@3 (called within InstallAppleCertificate@2's pre-job) because it breaks when the build runs on an openssl@1.1.1-equipped image, and we can't manually install openssl@1.1.1 via a script because the call to openssl is in InstallAppleCertificate@2's pre-job.

jorupp commented 2 weeks ago

We worked around it with InstallAppleCertificate@2 by putting two copies of the task in our pipeline - one with opensslPkcsArgs: -legacy and one without, and with continueWithError: true on both so the one that fails won't fail our build.

mikeharder commented 2 weeks ago

Could this change be breaking actions/setup-python on Mac?

https://github.com/actions/runner-images/issues/10812#issuecomment-2463898503

altendky commented 2 weeks ago

In this test run https://github.com/Chia-Network/chia-blockchain/actions/runs/11745259139?pr=18844 with 30x each of macOS ARM/Intel and Python version 3.9/3.10/3.11/3.12, only 3.10 on Intel failed. The passing jobs were run on either 20241008.186 or 20241023.237. The failing jobs were on 20241107.313.

mikeharder commented 2 weeks ago

In this test run https://github.com/Chia-Network/chia-blockchain/actions/runs/11745259139?pr=18844 with 30x each of macOS ARM/Intel and Python version 3.9/3.10/3.11/3.12, only 3.10 on Intel failed. The passing jobs were run on either 20241008.186 or 20241023.237. The failing jobs were on 20241107.313.

I think we might be seeing a combination of both #10817 and #10812. In your test run, Python 3.10 was found in the cache on macOS ARM64, so it passes. But if you land on an agent without Python 3.10 in the cache, it will try to download and install, but fails with the SSL error.

fengga commented 2 weeks ago

Is there a way to configure our build so we only run on the OpenSSL@3-equipped images? We can't add the -legacy flag we need for OpenSSL@3 (called within InstallAppleCertificate@2's pre-job) because it breaks when the build runs on an openssl@1.1.1-equipped image, and we can't manually install openssl@1.1.1 via a script because the call to openssl is in InstallAppleCertificate@2's pre-job.

I'm facing the same problem and if rollout to openssl@3.0 all the InstallAppleCertificate@2 task will be broken. Even if I use a cert signed by openssl@3.0 there is still a problem:

InstallAppleCertificate@2 will generate one command:

security import /MSALCPPAppleBuildCertificate_fgh.p12 -P -A -t cert -f pkcs12 -k ***/ios_signing_temp.keychain

which requires the MSALCPPAppleBuildCertificate_fgh.p12 signed by openssl@1.1

https://identitydivision.visualstudio.com/IDDP/_build/results?buildId=1385795&view=logs&j=9c650f88-aede-5522-025a-32fa3c2465b6&t=a74cb428-1f82-4e39-8054-c06bdb9b0377&l=20

sarathrajsrinivasan commented 2 weeks ago

Hi All,

We are currently reverting the change and keeping OpenSSL 1.1 in our images. It will be the default version for now. Will keep you posted with the updates.

dalemyers commented 2 weeks ago

@sarathrajsrinivasan Why was this reverted? After having spent time adjusting our flows to make it work with v3, we now need to undo that work, causing more disruption. This is likely the case for many others.

eregon commented 2 weeks ago

@dalemyers There is no need to undo that work, openssl@3 is available before & after that change.

dalemyers commented 2 weeks ago

@dalemyers There is no need to undo that work, openssl@3 is available before & after that change.

It is, but it's not the default, and various tasks expect one version or the other, so they need to be adjusted. Anything we control calling openssl directly is fine.

gabsamples6 commented 1 week ago

We are still facing this issues and we are really stuck, is the fix/revert going to be applied soon? thanks

jorupp commented 1 week ago

We are still facing this issues and we are really stuck, is the fix/revert going to be applied soon? thanks

Are you seeing v1.1.1 or v3 right now? We've been running ours with both versions of the task that's having an issue and using continueWithError: true to ignore the failing one. Every run this week has had the v1.1.1-style call succeed and the v3-style call fail.

surendrapatel-rak commented 6 days ago

We are still facing this issues and we are really stuck, is the fix/revert going to be applied soon? thanks

Are you seeing v1.1.1 or v3 right now? We've been running ours with both versions of the task that's having an issue and using continueWithError: true to ignore the failing one. Every run this week has had the v1.1.1-style call succeed and the v3-style call fail.

Is it not possible to skip the task (not needed) based on the agent macOS version check?

gabsamples6 commented 6 days ago

I have just tried on using both and both failed. I added "continueOnError" and legacy, This is just mad now - what are we going to do? Can someone post a full workaround if any? We cannot deploy an app to prod because of this

    - task: InstallAppleCertificate@2
      displayName: 'Install an Apple certificate ${{ parameters.appleCertificateP12 }} '
      inputs:
        certSecureFile: ${{ parameters.appleCertificateP12 }}
        certPwd: '${{ parameters.p12password }}'
        keychain: 'temp'
        opensslPkcsArgs: '-legacy'       USING LEGACY HERE
     continueOnError: true

    - task: InstallAppleCertificate@2
      displayName: 'Install an Apple certificate ${{ parameters.appleCertificateP12 }} '
      inputs:
        certSecureFile: ${{ parameters.appleCertificateP12 }}
        certPwd: '${{ parameters.p12password }}'
        keychain: 'temp'
      continueOnError: true
d-philipson commented 6 days ago

It's currently potluck if you get a runner image that actually works. 20241022.244 works while 20241106.316 fails. I just have to keep running the build until it eventually gets the older image and completes. It's absolutely ridiculous.

jorupp commented 6 days ago

I'm seeing both 20241022.254 and 20241118.366, but both are having the task with -legacy fail and the one without succeed (which works for our build thanks to continueOnError: true). /usr/local/bin/openssl version is saying OpenSSL 1.1.1w 11 Sep 2023 for both.

bwburch commented 6 days ago

@d-philipson yes I 100% concur this is ridiculous

erik-bershel commented 8 hours ago

For the near future, the issue is removed from the agenda due to the unpreparedness of some critical services for this change. We will notify you of planned changes when exact dates are set.