flaky installation on ci server

[geyuqiu]

$ ./mvnw com.github.eirslett:frontend-maven-plugin:install-node-and-npm -DnodeVersion=v12.13.1 -DnpmVersion=6.13.4 -Dmaven.repo.local=$MAVEN_USER_HOME
[ERROR] The archive file .../.maven/com/github/eirslett/node/12.13.1/node-12.13.1-linux-x64.tar.gz
is corrupted and will be deleted. Please try the build again.
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.8.0:
install-node-and-npm (default-cli) on project cbkop-gui-unified: Could not extract the Node archive:
Could not extract archive: '.../.maven/com/github/eirslett/node/12.13.1/node-12.13.1-linux-x64.tar.gz': EOFException -> [Help 1]

Do you want to request a feature or report a bug? bug What is the current behavior? because of a "failure" ci-setup, every push is built under the same directory flaky behavior: most of the time successfully suggested fix the archive maybe corrupted before and reinstallation should be clearing the old outdated / corrupted files? question: try rm -rf .maven before installation ? If the current behavior is a bug, please provide the steps to reproduce.

What is the expected behavior? runs smoothly every time Please mention your frontend-maven-plugin and operating system version. Red hat linux: 7 frontend-maven-plugin: 1.8.0

[SystemOutPrint]

+1

[grafvonb]

I have a similar problem when jobs run in parallel on the same agent:

08:11:47  [ERROR] The archive file /home/jenkins/.m2/repository/com/github/eirslett/node/12.22.7/node-12.22.7-linux-x64.tar.gz is corrupted and will be deleted. Please try the build again.
08:11:47  [INFO] ------------------------------------------------------------------------
08:11:47  [INFO] BUILD FAILURE
08:11:47  [INFO] ------------------------------------------------------------------------
08:11:47  [INFO] Total time:  5.177 s
08:11:47  [INFO] Finished at: 2021-10-22T06:11:46Z
08:11:47  [INFO] ------------------------------------------------------------------------
08:11:47  [ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.12.0:install-node-and-npm (install node and npm) on project mvn-node: Could not extract the Node archive: Could not extract archive: '/home/jenkins/.m2/repository/com/github/eirslett/node/12.22.7/node-12.22.7-linux-x64.tar.gz': EOFException -> [Help 1]

It seems to be a sort of a race condition. Is there an option to change the place where .tar.gz is saved before it gets extracted? If no we definitely need one.

[simontunnat]

I have the same issue in our GitLab CI environment.

It only happens from time to time: 09:11:11:813 [ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.12.0:install-node-and-npm (install node and npm) on project one-ebmpapst.ui.frontend: Could not extract the Node archive: Could not extract archive: '/builds/it.ics.epm/one-intranet/one-ebmpapst/.m2/repository/com/github/eirslett/node/16.14.2/node-16.14.2-linux-x64.tar.gz': EOFException -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.12.0:install-node-and-npm (install node and npm) on project one-ebmpapst.ui.frontend: Could not extract the Node archive: Could not extract archive: '/builds/it.ics.epm/one-intranet/one-ebmpapst/.m2/repository/com/github/eirslett/node/16.14.2/node-16.14.2-linux-x64.tar.gz'

[adamenveil]

Any fix for this?

[ryanrupp]

Relates to https://github.com/eirslett/frontend-maven-plugin/pull/807 which looks like when this scenario happens it will at least auto-delete the corrupted download, you can see this in the original post above

[ERROR] The archive file .../.maven/com/github/eirslett/node/12.13.1/node-12.13.1-linux-x64.tar.gz is corrupted and will be deleted. Please try the build again.

That said, it doesn't actually automatically retry the download then (maybe it should?) - there's a comment on the PR actually about this here.

[ryanrupp]

I think possibly the download logic here should be changed so that it downloads to a temp directory and then uses move (atomic) once the download has completed. Currently, it's just writing the response to the destination directly. If writing to a temp location you don't have to worry about the corrupt download really as much.

[seregamorph]

@ryanrupp you are right, such enhancement makes sense. And yes, initially not to complicate the change too much, the simple strategy with deletion of corrupted archive was used, the simple retry should eventually solve the problem. This solution is not perfect, also it can be not an option in some CI/CD environments, but for my team it was enough at the moment. I guess the best what you can do here is to submit the PR with proposed change.

[melloware]

Same happening here on GitHub Actions started happening constantly after latest 1.14.0 release.

Edit: Had to delete all my GitHub Caches to get it to start working again.

[pranavbhole]

@eirslett any update on this issue?

[uebelack]

+1

[Shahard2]

+1

[IevgenRagulin]

+1

[malaskowski]

This issue also happens when you try to manually download packages from https://nodejs.org/dist/. It seems that this repository is quite unstable (sometimes it takes less than 1 second to download package, sometimes over 5 min, sometimes it times out).

Retry option might mitigate this behaviour.

Related issue: https://github.com/nodejs/nodejs.org/issues/4495

[melloware]

There is an open PR: https://github.com/eirslett/frontend-maven-plugin/pull/1098

Just need to be merged and released.

[melloware]

@eirslett did a release yesterday but does not seem very active in this project. I wonder if it is time to fork it if he is no longer merging PR's or active in this plugin?

[pranavbhole]

@eirslett Can you please merge #1098 and release ? Thank you

[seregamorph]

The fix with several retries can be a simple and short-term solution, but it does not look like a good idea in a long-term. Private projects mostly should not rely on public repositories like maven central or nodejs.org. E.g. hub.docker.com already two years reject requests to fetch images if there is too many requests from one IP. I believe sooner of later the nodejs can start doing the same - and it's reasonable - probably they have huge costs for the distribution infrastructure. What is recommended is to setup internal proxy repository - for maven, for nodejs, for docker images. Also you can use it to publish internal artifacts - like custom frontend-maven-plugin build :) So you will not rely on code owners. This can sound complicated, but basic setup of nexus is pretty simple, it's even possible to run it via docker container.

[melloware]

@seregamorph I agree but what about GitHub Actions? We have automated builds running on our open source project and they probably all look like they are coming from GitHub?

[seregamorph]

That's a good question 😅 One of the options is to make the nexus repository available on public internet, just require authentication. As an option expose if via ngrok, example here https://github.com/marketplace/actions/nexus-repository-publisher-for-github-actions Also (probably for paid plans) GitHub actions may access private VPNs, but probably it's not a good choice for an open source project.

[jonkoops]

Perhaps #1118 has resolved this issue?

[melloware]

Unfortunately it does not it just pushes the problem I get this now on GitHub Actions.

Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.14.2:install-node-and-npm (install node and npm) on project primefaces: 
Could not download Node.js: 
Could not download https://nodejs.org/dist/v18.8.0/node-v18.8.0-linux-x64.tar.gz: 
Premature end of Content-Length delimited message body (expected: 43,238,752; received: 11,943,636) ->

[eirslett]

Unfortunately it does not it just pushes the problem I get this now on GitHub Actions.

Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.14.2:install-node-and-npm (install node and npm) on project primefaces: 
Could not download Node.js: 
Could not download https://nodejs.org/dist/v18.8.0/node-v18.8.0-linux-x64.tar.gz: 
Premature end of Content-Length delimited message body (expected: 43,238,752; received: 11,943,636) ->

Is it an option for you to host a Nexus instance for Primefaces? For example something like

https://www.primefaces.org/downloads/node/v18.8.0/node-v18.8.0-linux-x64.tar.gz

just simply a mirror of all the files in https://nodejs.org/dist/v18.8.0/?

[jonkoops]

I wonder what the cause of this issue is, is this just GitHub actions rate limiting kicking in?

[uebelack]

Workaround for github actions:

- name: Set Node Version from eirslett plugin
  run: |
    pom_files=$(find . -type f -name "pom.xml")

    for file in $pom_files; do
      node_version=$(grep -m 1 '<nodeVersion>' "$file" | sed -n 's/.*<nodeVersion>\(.*\)<\/nodeVersion>.*/\1/p')
      if [ -n "$node_version" ]; then
        echo "node version is $node_version"
        echo "NODE_VERSION=$node_version" >>${GITHUB_ENV}
        exit 0
      fi
    done

- name: Set up Node
  if: ${{ vars.NODE_VERSION != '' || env.NODE_VERSION != '' }}
  uses: actions/setup-node@v3
  with:
    node-version: ${{ vars.NODE_VERSION || env.NODE_VERSION }}

- name: Use github installed node for eirslett plugin
  run: |
    pom_files=$(find . -type f -name "pom.xml")
    node=$(which node)
    for file in $pom_files; do
      node_version=$(grep -m 1 '<nodeVersion>' "$file" | sed -n 's/.*<nodeVersion>\(.*\)<\/nodeVersion>.*/\1/p')
      if [ -n "$node_version" ]; then
        directory=$(dirname "$file")
        rm -rf $directory/node
        mkdir $directory/node
        ln -s $node $directory/node/node
      fi
    done

https://gist.github.com/uebelack/3b61f59a7a792e917c4fd4c37e4bea5d

[melloware]

@uebelack does this make sure Node is on the system already so when FontEnd goes to check its already installed by the GitHub Action?

[uebelack]

@uebelack does this make sure Node is on the system already so when FontEnd goes to check its already installed by the GitHub Action?

sure

[sebastianfrey]

Unfortunately it does not it just pushes the problem I get this now on GitHub Actions.

Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.14.2:install-node-and-npm (install node and npm) on project primefaces: 
Could not download Node.js: 
Could not download https://nodejs.org/dist/v18.8.0/node-v18.8.0-linux-x64.tar.gz: 
Premature end of Content-Length delimited message body (expected: 43,238,752; received: 11,943,636) ->

Is it an option for you to host a Nexus instance for Primefaces? For example something like

https://www.primefaces.org/downloads/node/v18.8.0/node-v18.8.0-linux-x64.tar.gz

just simply a mirror of all the files in https://nodejs.org/dist/v18.8.0/?

@eirslett Followed your approach, which works very well.

But instead of Nexus/Primefaces we use JFrog Artifactory as company wide maven repository. In our case we set up an Generic-Repository, which points to https://nodejs.org/dist.

After configuring frontend-maven-plugin to use the mirror by setting <nodeDownloadRoot /> to https://company.maven.host/artifactory/name-of-your-node-mirror our CI download problems disappeared.