search

eirslett / frontend-maven-plugin

"Maven-node-grunt-gulp-npm-node-plugin to end all maven-node-grunt-gulp-npm-plugins." A Maven plugin that downloads/installs Node and NPM locally, runs NPM install, Grunt, Gulp and/or Karma.
Apache License 2.0
4.2k stars 867 forks source link

Add configurable number of retries and interval between retries #1098

Open snuyanzin opened 1 year ago

snuyanzin commented 1 year ago

The PR adds retry option of download and install for node, npm, yarn.

We (Flink community) sometimes faced an issue like 

[ERROR] The archive file /__w/1/.m2/repository/com/github/eirslett/node/16.13.2/node-16.13.2-linux-x64.tar.gz is corrupted and will be deleted. Please try the build again.

so archive was corrupted and removed. However it would be nice to have an autoretry for this case i guess it could also partially address #882

melloware commented 8 months ago

@eirslett can we get this bumped or merged. GitHub Actions is constantly failing with this error?

uebelack commented 8 months ago

Pleaaaase 😜

eirslett commented 8 months ago

I see the problem! It's just that I'm a bit unsure what the right solution is. There are trade-offs.

There are already so many configuration options on this plugin - and if you look at the PRs that have been opened, so many proposed configuration options - in the end it will just be even more unmaintainable than it already is. It's hard to write automated tests for everything without a major refactor of the codebase, and it's not uncommon that unrelated things break because of that.

Maybe adding dynamic retries is an option, I'm not sure it has to be configurable though. Could just go with a simple default, for example 3 retries, 10 seconds between each. But retrying against a CDN (nodejs.org in this case) that is overloaded usually won't fix the problem anyways, it could make the problem even worse.

There was a suggestion that we could stop streaming the downloaded file - instead just keep it in memory until it's fully downloaded. All or nothing. That will probably solve the issue with corrupt files.

As for Flink - Maybe Flink could host a mirror of the node.js CDN, for its use? This plugin has specific configuration for using other download roots. It's made specifically to be able to use a more reliable CDN than nodejs.org.

melloware commented 8 months ago

I am going to keep my eye on 1.14.2 to see if it alleviates our GitHub Actions failures thanks for looking into this @eirslett

melloware commented 8 months ago

OK getting this error now...

Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.14.2:install-node-and-npm (install node and npm) on project primefaces: 
Could not download Node.js: 
Could not download https://nodejs.org/dist/v18.8.0/node-v18.8.0-linux-x64.tar.gz: 
Premature end of Content-Length delimited message body (expected: 43,238,752; received: 11,943,636) ->
eirslett commented 8 months ago

OK getting this error now...

How often is this occurring - 100 % of the time, or is it only when the Node.js CDN is unstable?

melloware commented 8 months ago

Its seems really random. So it must just be when Node.js CDN is busy?

dennisoelkers commented 4 months ago

@eirslett: are there any objections against merging this? We are also facing the premature EOF issue on github actions every day and think this could help us improving it.