yarn: tests do not work on Alpine linux

felixfontein commented 2 years ago

Summary

Currently the tests for yarn do not work on Alpine linux, which is supported in CI since #4222.

Issue Type

Bug Report

Component Name

yarn

Ansible Version

unrelated

ansibullbot commented 2 years ago

Files identified in the description:

plugins/modules/packaging/language/yarn.py

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

ansibullbot commented 2 years ago

cc @chrishoffman @verkaufer click here for bot help

russoz commented 2 years ago

Apparently the problem is related to the fact that the integration test is downloading the official nodejs binaries and simply trying to run them, but the official bins are compiled against glibc whereas Alpine uses musl.

According to the top-rated answer in: https://stackoverflow.com/questions/64243684/install-node-js-from-source-on-alpine

Possible solutions would be:

Use node's official image instead of trying to install it from a alpine base image: that's because there are different dependencies and things to setup before having a working node image (certificates, tls libraries, etc). This is the most recommended.

Unlikely that we will generate a new test container based on node's image

Installing node via apk: node is available at alpine's official package manager apk, and you can install it by simply running apk add nodejs. The only problem here is that the version that's available in the repository is the LTS (12.18.4 as of 2020-10-07).

That could work but we would no longer be able to maintain the same level of compatibility in the module (going back to nodejs 4.8.0), and also possibly hard to test agains multiple versions of node.

Installing/building a compability layer for glibc in alpine: this is not recommended at all, since alpine is built over musl and running glibc is not a good practice and can lead to things breaking. Even installing the official libc6-compat may lead to problems: (...)

That also seems an unlikely solution as it would hugely increase the complexity of the test (having to build nodejs, multiple versions)

Another answer in that thread points to the use of unofficial binaries compiled against musl: https://unofficial-builds.nodejs.org/download/release/v14.4.0/node-v14.4.0-linux-x64-musl.tar.xz

But that solution has two major problems:

The oldest version available in that download url is 8.16.0, which fails to keep the compatibility with the current test
The repo for generating those unofficial images is not very reassuring when it comes to reliability. As of the time of this writing (Apr 03, 2022):

This project is experimental: its output is not guaranteed to remain consistent and its existence is not guaranteed into the future. This project is in need of a community of maintainers to keep it viable. If you would like to join, please submit pull requests to improve the work here.

I even started, for kicks, to check how complicated and how long it would take to build NodeJS 4.8.0 on that test-container. The build process itself relies on Python 2, which is no longer available, so that was a dead-end.

My personal opinion on the whole matter is:

We need to raise the minimum compatibility level for NodeJS. That would be a breaking change but trying to keep the bar that low (4.8.0 is from Feb 2017) is holding us back here.
We should consider deciding not to support Alpine Linux for this module at all, and then close this ticket, or
We should consider building nodejs ourselves (for musl support), but in that case item 1 above would become mandatory (unless we want to continue supporting Python 2), but also it could put quite a strain in our CI pipeline.

Personally I would vote for #2.

felixfontein commented 2 years ago

How about simply using a different approach on Alpine, and keep the current on the other systems? I don't see why we should drop compatibility with some versions just because we cannot test them everywhere?

Also we're only testing really old node versions right now (latest are 8.0.0 for yarn and 8.2.0 for npm), while the current version range is 17.x.y (latest stable is 16.x.y). That's not really helpful anyway, and we should definitely also test newer versions.

I guess the npm and yarn tests need a general rewrite.

russoz commented 2 years ago

Not supporting Alpine is a different approach on Alpine ;-)

But I can easily add newer node versions to the test pool.

russoz commented 2 years ago

I thought I could. sigh The tests are not pulling nodejs from the official node website, they are being pulled from:

src: 'https://ansible-ci-files.s3.amazonaws.com/test/integration/targets/yarn/{{ nodejs_path }}.tar.gz'
src: 'https://ansible-ci-files.s3.amazonaws.com/test/integration/targets/yarn/yarn-v{{yarn_version}}.tar.gz'

and newer versions (at least the few I tried) are not there. We should probably change this to the official URL and add these newer versions to it.

ansibullbot commented 1 year ago

Files identified in the description:

plugins/modules/yarn.py

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

ansible-collections / community.general