search

honeybadger-io / honeybadger-webpack

A webpack plugin to send sourcemaps to Honeybadger
MIT License
29 stars 7 forks source link

Timeout when uploading source-map files with Docker #299

Closed Pierre-Do closed 3 years ago

Pierre-Do commented 3 years ago

What are the steps to reproduce this issue?

  1. Build a large project (200+ source-maps) with Docker
  2. Use the plugin to upload the files to Honeybadger

What happens?

From time to time (once every 5 builds, give or take), the Webpack build is hanging (nothing's happening) or socket hang up when uploading the source-maps.

What were you expecting to happen?

Source-maps being uploaded consistently

Any logs, error output, etc?

None, the build appears stuck

Any other comments?

These kind of issues are difficult to debug. However, I believe it might be related to this: https://github.com/honeybadger-io/honeybadger-webpack/blob/master/src/HoneybadgerSourceMapPlugin.js#L191

Since 1.3.0, the source-maps are uploaded using fetchRetry, in a promise. As shown above, all promises for all source-maps are created at the same time, hence uploading every file at roughly the same time. This isn't an issue for small projects, but my team has been experiencing some issues on with main project (200+ source-maps).

One solution could be to upload each file one after the other. This would however be a performance drag. I suggest to have a compromise and upload files 3 or 5 at the time with the following solution: https://github.com/qoqa/honeybadger-webpack/commit/0078ee289937ea3aa37a437216cf072749a72e80

We've been running this fork with the fix linked above. After a week of using this version, we haven't seen any build hanging anymore.

I'm happy to make a PR with this fix to this repo if you deem the solution viable for a broader audience.

What versions are you using?

Operating System: Docker image (ruby-2.7.2-buster) Package Version: 1.3.0 Node Version: 14

joshuap commented 3 years ago

@Pierre-Do great writeup! Yes, please submit a PR. 🙏 It might be useful to make the batch number configurable, like retries. I'm not sure what name would make the most sense for the config option—maybe workerCount?

joshuap commented 3 years ago

@ParamagicDev I wonder if this could help with #276

Pierre-Do commented 3 years ago

@Pierre-Do great writeup! Yes, please submit a PR. pray It might be useful to make the batch number configurable, like retries. I'm not sure what name would make the most sense for the config option—maybe workerCount?

Thanks for your feedback @joshuap. workerCount sounds good to me, it would indeed by great to configure that. I've tried with 3 in my fork, but I feel we could safely default to 5.

I'll drop a PR as soon as I figure a way to write some pertinent tests! Probably early next week.

Pierre-Do commented 3 years ago

Suggested pull request: https://github.com/honeybadger-io/honeybadger-webpack/pull/300