`/__e/node20/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /__e/node20/bin/node)`

[nicowilliams]

Using container: image: debian:bullseye to build the OpenJDK for an older release of a distro (Debian in this case, but this isn't specific to Debian) causes a failure due to node in the actions/checkout@v4 docker image to fail to run due to the container having an older glibc:

/usr/bin/docker exec  ebe297a6b5716c387c7333d466eb082069a2f0b0c97379a95e810de4f4fa67a5 sh -c "cat /etc/*release | grep ^ID"
/__e/node20/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /__e/node20/bin/node)

(Using build tools that depend on newer glibc versions seems likely to be a common pitfall for GitHub Actions.)

[briandfoy]

I also have this problem on Ubuntu 22.04 (see https://github.com/briandfoy/data-constraint/actions/runs/7645971689/job/20833756885 while it is around).

Run actions/checkout@v4
  with:
    repository: briandfoy/data-constraint
    token: ***
    ssh-strict: true
    persist-credentials: true
    clean: true
    sparse-checkout-cone-mode: true
    fetch-depth: 1
    fetch-tags: false
    show-progress: true
    lfs: false
    submodules: false
    set-safe-directory: true
/usr/bin/docker exec  55707bba4641f79c81786bd2c50d454ad4ce5589862509caadcaf0d83af05524 sh -c "cat /etc/*release | grep ^ID"
/__e/node20/bin/node: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.27' not found (required by /__e/node20/bin/node)
/__e/node20/bin/node: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /__e/node20/bin/node)
/__e/node20/bin/node: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /__e/node20/bin/node)
/__e/node20/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /__e/node20/bin/node)
/__e/node20/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by /__e/node20/bin/node)

[falkTX]

I have the same issue when trying to use ubuntu:18.04 image https://github.com/moddevices/mod-desktop-app/actions/runs/7697857575/job/20975801114

[falkTX]

Similarly the actions/cache@v4 also fails https://github.com/moddevices/mod-desktop-app/actions/runs/7698414056/job/20977633093

[Lastique]

I have the same problem with checkout@v4 with running CI on ubuntu:16.04 and ubuntu:18.04 docker images:

/usr/bin/docker exec  96e66dce4d560bc83ea53e3885a0f7a370efc33ab78c57efc31e036d6ef7b0c3 sh -c "cat /etc/*release | grep ^ID"
/__e/node20/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /__e/node20/bin/node)

https://github.com/Lastique/scope/actions/runs/7754629439/job/21148329132

/usr/bin/docker exec  25c6cfe8c9e7550b5d65e58ee1567ae81f08d6b5b6b6868be7c4026ac201e195 sh -c "cat /etc/*release | grep ^ID"
/__e/node20/bin/node: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.27' not found (required by /__e/node20/bin/node)
/__e/node20/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /__e/node20/bin/node)
/__e/node20/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by /__e/node20/bin/node)

https://github.com/Lastique/scope/actions/runs/7754629439/job/21148329685

These older Ubuntu images are needed to be able to test older compiler versions.

Until this issue is resolved, please remove the deprecation notice for checkout@v3, as there simply is no alternative.

[falkTX]

Until this issue is resolved, please remove the deprecation notice for checkout@v3, as there simply is no alternative.

note that it is not just the checkout being broken, anything that uses node is broken which includes cache and artifacts. even the node-setup action (which could potentially be a workaround) is broken.

basically all the v4 actions from github are broken for older distros due to glibc requirements

[nicowilliams]

basically all the v4 actions from github are broken for older distros due to glibc requirements

Yes. I've noticed. Obviously this was the first one I diagnosed, so it got the issue :)

[Dlazder]

Same problem in ubuntu 18

[evitiello]

Anyone found a workaround for this? Or know why it started happening?

[falkTX]

the workaround is to keep using v3 actions. and it happens because github devs updated the base system where their node stack is built from.

seeing the way github has handled other issues, I expect this one to just be ignored completely.

[Dlazder]

I wrote to hosting support, but it turned out that the newer version is not supported. I ended up renting vds, where I installed the version I needed.

[sameeraksc]

We also have this issue. Github is saying v3 is deprecated. But no one can use v4

[skyzyx]

For people asking, Node.js 20 started requiring a newer version of glibc than what these older distros are shipping. Node.js 16 reached end-of-life in September 2023. GitHub Actions skipped Node 18 and went straight to Node 20.

Ubuntu 18.04 went end-of-life in May 2023. Ubuntu 20.04 and newer LTS releases are still supported (as of this writing), but it does not mean that they have a new-enough version of glibc. This also affects things like CentOS 7 (reaching end-of-life in a few days) and Amazon Linux 2 (supported for 1 more year) who support neither Node.js 18 nor Node.js 20[1][2].

In languages like C/C++, and Linux distributions that are supported for many years, everything boils down to the versions of the shared C libraries (.so files) that are installed and "certified" for that release of that OS. So it's challenging to upgrade something as foundational as glibc in older Linux distributions and provide any kind of guarantee or support without upgrading the whole OS.

The core of the issue is continued use of end-of-life (unsupported) software. Node.js 20 requires newer releases of Linux distributions. The specific configuration of software you're trying to use includes end-of-life software in the mix.

This is why you are receiving these error messages. Since the @v3 of this action depends on an end-of-life version of Node.js, I think it would be wrong of the maintainers to "remove warnings". You are genuinely running unsupported software.

[1] https://github.com/nodejs/node/commit/3671cc0432 [2] https://github.com/nodejs/node/pull/42659

[vadz]

The core of the issue is making it very difficult to impossible for the projects that have nothing to do with JS on their own and that need to support the older platforms to not be able to test the builds on or create distributions for them any more.

Dropping support for the older Node versions even on the platforms where the new ones are not available is an interesting decision, to put it mildly.

[mmomtchev]

It is possible to build Node.js 20 for older glibc - for example AWS who usually have a much longer support lifecycle than Github do this. However the resulting binary will be a Frankenstein version and many binary modules won't work with it, unless these are also rebuilt from source. I maintain a number of binary Node.js addons and some of them fail in their environment.

I have seen one very hackish solution in which such a custom built Node.js 20 distribution is mounted in /__e/node20 read-only so that the Github Action cannot overwrite it: https://github.com/actions/runner/issues/2906

[nchao-ciitizen]

the workaround is to keep using v3 actions. and it happens because github devs updated the base system where their node stack is built from.

seeing the way github has handled other issues, I expect this one to just be ignored completely.

the workaround is to keep using v3 actions. and it happens because github devs updated the base system where their node stack is built from.

seeing the way github has handled other issues, I expect this one to just be ignored completely.

We used this workaround since the issue appeared, however it seems today v3 is hitting this issue as well unfortunately.

[mmomtchev]

I got around it by using:

env:
  ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true

in my workflow: https://github.com/mmomtchev/hadron/blob/main/.github/workflows/os_comp.yml#L32-L33

But this too, it will stop working at some point.

[tassiocred]

I got around it by using:

env:
  ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true

in my workflow: https://github.com/mmomtchev/hadron/blob/main/.github/workflows/os_comp.yml#L32-L33

But this too, it will stop working at some point.

This solution works fine, but be careful while using the v4 of the github action runners, they bypass this flag.

[chiragms]

I got around it by using:

env:
  ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true

in my workflow: https://github.com/mmomtchev/hadron/blob/main/.github/workflows/os_comp.yml#L32-L33 But this too, it will stop working at some point.

This solution works fine, but be careful while using the v4 of the github action runners, they bypass this flag.

Man! You saved the day. Thanks so much. I wonder how you figured it out.

[bohana]

The workaround worked for us too. Thanks!

Noting that you can also set this as an env var on the self-hosted runner (what we did). We added it to the runner startup script, so no changes to action yaml were needed.

[vamarkopoulos]

@bohana how did you made it without changing the yml file. Doesnt the action need the below env line ?

 steps:
      - name: Checkout
        uses: actions/checkout@v3
        env:
          ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: ${{ vars.ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION }}

[bohana]

@bohana how did you made it without changing the yml file. Doesnt the action need the below env line ?

ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION is also an OS level environment variable, It can be changed on the runner host directly (in the script that starts up the runner for example).

This worked without having to change any yml configuration (but obviously will apply globally to everything the runner does, in case)

For reference see accepted answer here: https://github.com/actions/runner/discussions/2604

[csukuangfj]

I got around it by using:

env:
  ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true

in my workflow: https://github.com/mmomtchev/hadron/blob/main/.github/workflows/os_comp.yml#L32-L33

But this too, it will stop working at some point.

Thank you! You saved my life!

[tonyarnold]

This is happening to me with checkout v4, on ubuntu-latest.

The v3 workaround isn't working for me, either.

[csukuangfj]

This is happening to me with checkout v4, on ubuntu-latest.

The v3 workaround isn't working for me, either.

Have you tried

env:
  ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true

Please see https://github.com/mmomtchev/hadron/blob/main/.github/workflows/os_comp.yml#L32-L33

[jlevon]

Now also broken for checkouts v3 with centos:7 container (working until very recently):

Run actions/checkout@v3
/usr/bin/docker exec  02e2cf2da72963d76b12a421e48bc0790138ee97bf567c8afcdf792dbc2093d8 sh -c "cat /etc/*release | grep ^ID"
/__e/node20/bin/node: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by /__e/node20/bin/node)
/__e/node20/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /__e/node20/bin/node)
/__e/node20/bin/node: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /__e/node20/bin/node)
/__e/node20/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /__e/node20/bin/node)
/__e/node20/bin/node: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by /__e/node20/bin/node)
/__e/node20/bin/node: /lib64/libc.so.6: version `GLIBC_2.25' not found (required by /__e/node20/bin/node)

https://github.com/nutanix/libvfio-user/actions/runs/10417971087/job/28853465005?pr=802

[jlevon]

ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION workaround apparently no longer works.

[backend-sungjin]

Same problem OS & Version: Centos 7 Runtime: nodejs v20 Error messages:

/root/actions-runner/externals/node20/bin/node: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by /root/actions-runner/externals/node20/bin/node)
/root/actions-runner/externals/node20/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/actions-runner/externals/node20/bin/node)
/root/actions-runner/externals/node20/bin/node: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /root/actions-runner/externals/node20/bin/node)
/root/actions-runner/externals/node20/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /root/actions-runner/externals/node20/bin/node)
/root/actions-runner/externals/node[20](https://github.com/cashwalk/WristdoctorAPIServer/actions/runs/10470028746/job/28994292915#step:2:21)/bin/node: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by /root/actions-runner/externals/node20/bin/node)
/root/actions-runner/externals/node20/bin/node: /lib64/libc.so.6: version `GLIBC_2.25' not found (required by /root/actions-runner/externals/node20/bin/node)

Has anyone solved this?

[jlevon]

@backend-sungjin as it's clear they're not planning to fix this, I'd recommend dropping use of this action altogether.

https://github.com/nutanix/libvfio-user/pull/803/files#diff-a0fe23534b616d51ce686d2a1bcd1a78bc75074aef1a2f6ee96c9469991e1a4cR35

[skyzyx]

Also, CentOS 7 has reached end-of-life.

[LuWang1983]

We are on v3 and encountering this error as well. I may have to pin down the specific v3 version that does not introduce this breaking change. No mention of breaking changes in the change logs either.

[LuWang1983]

FYI, pinning down the version didn't seem to work (I pinned down to v3.5.0), so assuming node20 is required to build the container in checkout, can possibly set the node version in VM to 20 first then to an older version if the rest of the jobs depend on an older node version.

[LuWang1983]

@backend-sungjin as it's clear they're not planning to fix this, I'd recommend dropping use of this action altogether.

https://github.com/nutanix/libvfio-user/pull/803/files#diff-a0fe23534b616d51ce686d2a1bcd1a78bc75074aef1a2f6ee96c9469991e1a4cR35

The thing is it may not always checkout the latest or the commit before a feature branch commit?

[caiocsgomes]

hey team, any updates on this? It's being 9 months and this issue has been referenced in hundreds of issues/prs

[mmomtchev]

There are no updates and there won't be any. They are deprecating these releases. They allowed people to use them for a while through a special flag that was supposed to give you one last deadline extension.

I do not agree with this decision, but this is how it works. They have a legitimate need to keep the number of supported configurations down to a reasonable number.

The only compromise I see possible is for Github to allow actions to use an existing Node.js runtime in the container through a flag - just as they can use an existing git client. This would allow anyone who really needs to try running their own action in an unsupported mode - or at least to create his own more limited action that works everywhere. Because at the moment, even this is not possible.

[alecrajeev]

I ran into this bug for some CI tests and I implemented a workaround.

We basically built (but did not push) a docker image and ran the CI tests inside of the docker build. If the CI tests passed, then the docker build was successful. If the CI tests failed, then the docker build failed.

In this way we got around the node js version limitations for actions/checkout.

[mcarbonneaux]

i use container to test build on old compiler, but when use checkout action i take the error about nodejs. it's possible to execute checkout outside the container ?