Esbuild plugin does up inject/upload all source and map files on build

[Fwang36]

Is there an existing issue for this?

• [X] I have checked for existing issues https://github.com/getsentry/sentry-javascript/issues
• [X] I have reviewed the documentation https://docs.sentry.io/
• [X] I am using the latest SDK release https://github.com/getsentry/sentry-javascript/releases

How do you use Sentry?

Sentry Saas (sentry.io)

Which SDK are you using?

@sentry/browser

SDK Version

@sentry/esbuild-plugin 2.4.0

Framework Version

react 18.2

Link to Sentry event

No response

SDK Setup

No response

Steps to Reproduce

1. Enable splitting in esbuild config with splitting: true
2. Create a dynamic import
3. Add the sentryEsbuildPlugin()
4. build

My esbuild config -

My output files -

Upload logs where only app.js and app.js.map being uploaded -

Expected Result

All the files outputted by the bundler should be injected and uploaded

Actual Result

Not all files outputted by the bundler are being injected and uploaded.

[mydea]

Thanks for reporting this! Moving this to https://github.com/getsentry/sentry-javascript-bundler-plugins, as this is related to the plugin, not the SDK.

[lforst]

Took a look at this. Unfortunately, due to the restrictive nature of the esbuild plugin API we will probably not get debug ID injection working in combination with splitting: true. In these cases we will probably need to use options.uploadLegacySourcemaps.

[lforst]

Putting in backlog and keeping an eye on the esbuild API. Maybe in the future this will be possible.

[muuvmuuv]

Hey, I don't know if this is related or not, but when using the new Angular with esbuild and inject sentry esbuild plugin I see source maps in buildArtifactPaths but after glob(globAssets) all source maps get lost. Is that because the artifacts are not yet written?

[lforst]

@muuvmuuv did you provide an assets option?

[muuvmuuv]

@lforst yeah, ofc. I also created my own plugin just to print out all build artifacts to proove my assets glob is correct but sadly does not work. Since Angular does not support plugins atm I hacked around my own "cli" but it works perfectly for anything else: https://hastebin.com/share/lawidivura.javascript

[lforst]

Can you share what you defined in your assets option? I think it might be wrong.

[muuvmuuv]

I tried ./**/*.js.map, *.js.map and ./www/browser/*.js.map

[lforst]

You could add debug: true - maybe the plugin will tell you what is wrong. Generally I believe you should set assets to **/*.js.map (without dot in front) and that should work.

[muuvmuuv]

Nope, this also does not work. I guess this has something to do with the files not beeing actually created but used virtual until Angular finished all post tasks so Sentry wont find the files via "glob" function. The path of every build artifact is /project/chunk.xxxx.js.map but this file only exist virtually.

⠸ Building...[sentry-esbuild-plugin] Warning: Didn't find any matching sources for debug ID upload. Please check the `sourcemaps.assets` option.

I created this plugin to see the build artifacts. Got it from here which sound related but idk https://github.com/floydspace/serverless-esbuild/issues/145#issuecomment-861749419:

{
    name: 'plugin-sm',
    setup: (build) => {
      build.onEnd((args) => {
        for (const file of args.outputFiles) {
          if (/\.map$/.test(file.path)) {
            console.log(file)
          }
        }
      })
    },
  }

It prints:

{
  path: '/Users/marvin/Developer/PROJECT/chunk-IVCW2ILW.js.map',
  contents: <Buffer 7b 22 76 63 ... 3634 more bytes>,
  hash: 'LYLq5nfs65g',
  text: [Getter]
}
{
  path: '/Users/marvin/Developer/PROJECT/chunk-OZZSOK5O.js.map',
  contents: <Buffer 7b 22 76 6e 69 63 ... 1101 more bytes>,
  hash: 'noq8VdjXkkc',
  text: [Getter]
}

[muuvmuuv]

Ok when I writeSync the js and map file in my plugin before sentry I get a new error. And I had to set assets to "*/.js" since it searches by itself for map files and needs the source assets.

[sentry-esbuild-plugin] Debug: Could not determine debug ID from bundle. This can happen if you did not clean your output folder before installing the Sentry plugin. File will not be source mapped: /Users/marvin/Developer/M8FINDER/M8FINDER/www/browser/chunk-27CYYZI4.js

[lforst]

Well this is not an error it is more like a warning and I would suggest following what it says. Did you clean your output folder before running the plugin with the **/*.js option? In general, I recommend completely omitting the assets option since the plugin should find output files by itself.

[wSedlacek]

I am running into this issue too when using Angular and Nx. It seems like the glob is running before all the files have been saved to disk and thus not finding them. https://github.com/getsentry/sentry-javascript-bundler-plugins/blob/a097e3249129d30a058d10f41a66960f17ca1647/packages/bundler-plugin-core/src/index.ts#L494-L501

I have confirmed that checking my outputFiles in onEnd does have the source map files. Angular disable the esbuild writing and does some post processing before writing the files itself. https://github.com/angular/angular-cli/blob/125fb779ff394f388c2d027c1dda4a33bd8caa62/packages/angular_devkit/build_angular/src/tools/esbuild/bundler-context.ts#L95

Perhaps there is a way to move the source map upload to after everything has been written? Maybe the outputFiles could be used directly for uploading?

[muuvmuuv]

That is exactly what I discovered @wSedlacek I think the esbuild plugin should work with the Buffer instead of file output on filesystem in the end they should be the same. Or we need an Angular first-level support for Sentry (like with Tailwind), Astro also has announced this lately.

[lforst]

Interesting. Seems like Angular is doing unorthodox things here. I am not a fan of this. We might need to build something special for Angular. In the meanwhile you can use Sentry CLI to upload sourcemaps. That should still work.

[wSedlacek]

So I played around with the changeset #454 which used the outputFiles which does get the content of the files to the upload block. However, the files seem to be missing sentry-dbid- which prevents them from getting past this check. https://github.com/wSedlacek/sentry-javascript-bundler-plugins/blob/a097e3249129d30a058d10f41a66960f17ca1647/packages/bundler-plugin-core/src/debug-id-upload.ts#L250-L256

Reverting the changes and looking at the outputs I am not seeing sentry-dbid- any of the files that are written to disk. It seems like thus may be a multipart issue to get an esbuild plugin working with Angular.

In the meanwhile you can use Sentry CLI to upload sourcemaps. That should still work.

I was able to use the CLI to inject and upload. I will do this in the mean time. Thank you!

[lforst]

@wSedlacek It's really cool that you tried to fix this. I had a suspicion that things are harder than they seem. Our biggest issue is frameworks doing shady stuff with bundlers.

[joewIST]

Also experiencing similar issues with sentry esbuild plugin, Angular and Nx combination. With some tweaking I can get rid of the errors, but mine just hangs on the "Building" step seemingly indefinitely...

[kevireilly]

Just wanted to provide the additional data point that I'm experiencing this issue with @sentry/esbuild-plugin and splitting: true specifically (without Angular). Does work appropriately for the main esm bundle.mjs and iife bundle.js.

[kevireilly]

By hacking around in node_modules/@sentry/esbuild-plugin/dist/esm/index.mjs I've gained some additional insight in to the issue and part of a potential solution. One problem is that only files of kind === 'entry-point' are being evaluated and looks like dynamic-import needs to be allowed as well.

https://github.com/getsentry/sentry-javascript-bundler-plugins/blob/81e3de95ad1b3d1777818a404bf0c57c65842497/packages/esbuild-plugin/src/index.ts#L54

if (args.kind !== "entry-point" && args.kind !== "dynamic-import") {
  return;
} else {
  ...various setup to later inject a Debug ID...
}

With this change, I'm observing named chunks getting uploaded appropriately. However, it does look like it is missing chunk-[hash].[m]js files presumably because they're not explicitly dynamically imported (at least in my scenario) as opposed to a byproduct of splitting that I don't understand just yet. At any rate, some good output where previously it was only just esm-module.mjs.map being identified and uploaded:

[sentry-esbuild-plugin] Debug: Could not determine debug ID from bundle. This can happen if you did not clean your output folder before installing the Sentry plugin. File will not be source mapped: /Users/kevin.reilly/development/project/dist/chunk-G5VFEN4J.mjs
[sentry-esbuild-plugin] Debug: Could not determine debug ID from bundle. This can happen if you did not clean your output folder before installing the Sentry plugin. File will not be source mapped: /Users/kevin.reilly/development/project/dist/chunk-YHQW4PJU.mjs

Source Map Upload Report
  Minified Scripts
    ~/0ae57e80-90ef-40a9-b1d0-5e522b79665a-4.js (sourcemap at middleware-XWW3I7LS.mjs.map, debug id 0ae57e80-90ef-40a9-b1d0-5e522b79665a)
    ~/49a9e9b6-6eda-47bb-9cd1-3a6dfc3427ea-3.js (sourcemap at esm-module.mjs.map, debug id 49a9e9b6-6eda-47bb-9cd1-3a6dfc3427ea)
    ~/5bec62e2-a022-4935-b999-9424fa2f28fc-0.js (sourcemap at goodies-2AHNUHGU.mjs.map, debug id 5bec62e2-a022-4935-b999-9424fa2f28fc)
  Source Maps
    ~/0ae57e80-90ef-40a9-b1d0-5e522b79665a-4.js.map (debug id 0ae57e80-90ef-40a9-b1d0-5e522b79665a)
    ~/49a9e9b6-6eda-47bb-9cd1-3a6dfc3427ea-3.js.map (debug id 49a9e9b6-6eda-47bb-9cd1-3a6dfc3427ea)
    ~/5bec62e2-a022-4935-b999-9424fa2f28fc-0.js.map (debug id 5bec62e2-a022-4935-b999-9424fa2f28fc)

[lforst]

@kevireilly Thanks for doing the research. I'll investigate whether also checking for dynamic-import could work. As it stands splitting: true is not supported.

[jahusa02]

Are there any updates?

[lforst]

@jahusa02 if there were you would see it here.

[attilacsanyi]

@lforst @kevireilly is there any update on this? We are using angular esbuild with sourceMaps: true which generates proper sourcemaps, but it seems @sentry/esbuild-plugin not upload it. Thanks for any update on this issue.

[attilacsanyi]

@lforst what do you think, will be an update soon? We are using PWA and Angular with esbuild bundler and @sentry/esbuild-plugin. Until this issue the manual upload won't work as sentry-cli would update sources with debug-id and service worker will break.

I want to avoid switching back to webpack as esbuild is more performant.

Grateful for any feedback. 🙇

[Lms24]

Hey @attilacsanyi! We're aware of the limitation around PWA/service workers and the CLI breaking the integrity of the generated artifacts. Sadly, as far as I'm aware, there's nothing we can do about that for the CLI flow as we'll never get around the timing problem here.

but it seems @sentry/esbuild-plugin not upload it

Just to confirm: Is the reason why our plugin doesn't upload the source maps the same that is discussed in this issue?

Unfortunately, generally, there are no updates at the moment. We've all been very busy with the new SDK major version.

My 2 cents: Angular makes it extremely tricky for us to provide proper ways to upload source maps, due to them not allowing customized build setups. Having talked a bit about this with Angular team members recently, I'm afraid this won't change anytime soon :( I understand their reasoning but it sucks for tools like ours.

[muuvmuuv]

The main issue is still there because Angular does not create the files physically until build-ends so Sentry is not able to link them properly. As I state a few comments earlier it could work if Sentry uses the Buffer during the build and hash as ID. But I am not deeply enough into that process, just did some digging.

[lforst]

@attilacsanyi @Lms24 sorry for my ignorance, I might have forgotten. What is the problem with Sentry CLI + service workers again?

[attilacsanyi]

@attilacsanyi @Lms24 sorry for my ignorance, I might have forgotten. What is the problem with Sentry CLI + service workers again?

No problem @lforst . We use Nx monorepo with Angular and use esbuild executor. Afaik after the build the bundle hash calculated and service worker script generated by Angular based on js bundles content.

The manual Sentry CLI script generated by npx @sentry/wizard@latest -i sourcemaps wizard I can manually link inject-ed debug-ids into sourcemaps and JS bundles.

"sentry:sourcemaps:inject": "sentry-cli sourcemaps inject --org MyOrg --project MyApp ./dist/apps/my-app/browser",
"sentry:sourcemaps:upload": "sentry-cli sourcemaps upload --org MyOrg --project MyApp ./dist/apps/my-app/browser --dist my-dist --release my-release",
"sentry:sourcemaps": "pnpm sentry:sourcemaps:inject && pnpm sentry:sourcemaps:upload"

But here is the problem the bundles content has changed as this debug ids were injected after Angular generated the hash for them hence will be a hash mismatch. (The order of service worker hash generation and build order is managed by Angular builder so cannot hook into any process in the middle :( )

Thank you if you have any feedback on this.

[lforst]

@attilacsanyi Ah thanks for explaining. Yeah this is a bit of a chicken and egg problem. I unfortunately don't see a clear path forward for us. In theory we are already operating and injecting debug IDs on the "virtual" level but it seems like angular is circumventing our plugin somehow.

In the meanwhile you can use the "legacy" (really should not be named as such because we cannot and will not remove it) way of uploading source maps: https://docs.sentry.io/platforms/javascript/sourcemaps/troubleshooting_js/legacy-uploading-methods/

[attilacsanyi]

@attilacsanyi Ah thanks for explaining. Yeah this is a bit of a chicken and egg problem. I unfortunately don't see a clear path forward for us. In theory we are already operating and injecting debug IDs on the "virtual" level but it seems like angular is circumventing our plugin somehow.

In the meanwhile you can use the "legacy" (really should not be named as such because we cannot and will not remove it) way of uploading source maps: https://docs.sentry.io/platforms/javascript/sourcemaps/troubleshooting_js/legacy-uploading-methods/

Thanks for the idea @lforst, but based on the CI build logs the uploadLegacySourcemaps + debug approach has the same problem: it runs too early before Angular Esbuild finished generating the sourcemaps and not find anything to upload :(

...
# Build started here
2024-05-15T09:32:40.3550299Z > nx run my-app:build:production
2024-05-15T09:32:40.3550676Z 
2024-05-15T09:32:41.8097156Z - Building...
...
2024-05-15T09:33:28.3976701Z   chunk-PJQECVUI.js   139.3kb
2024-05-15T09:33:28.3977214Z   chunk-AHHMLF6Q.js   138.9kb
2024-05-15T09:33:28.3977626Z   chunk-X5XUYWHG.js   132.2kb
2024-05-15T09:33:28.3978152Z   ...and 358 more output files...
# Build finished here, but source maps not here yet
2024-05-15T09:33:28.3978615Z ⚡ Done in 46299ms
2024-05-15T09:33:29.7071376Z   INFO    2024-05-15 09:33:29.706584419 +00:00 Loaded config from /runner/_work/org/my_app/.sentryclirc
2024-05-15T09:33:29.7072988Z   DEBUG   2024-05-15 09:33:29.706628420 +00:00 sentry-cli version: 2.31.2, platform: "linux", architecture: "x86_64"
# Sentry CLI called to upload sourcemaps
2024-05-15T09:33:29.7076089Z   INFO    2024-05-15 09:33:29.706645140 +00:00 sentry-cli was invoked with the following command line: "/runner/_work/1inch-v2/1inch-v2/node_modules/@sentry/cli-linux-x64/bin/sentry-cli" "releases" "files" ....
...
# Sentry CLI did not find anything (too early execution)
2024-05-15T09:33:30.1550532Z > Nothing to upload
2024-05-15T09:33:31.1702563Z [sentry-esbuild-plugin] Warning: Didnt find any matching sources for debug ID upload. Please check the `sourcemaps.assets` option.
...
# Finally only here is the point when Angular Application bundle generation completes
2024-05-15T09:33:42.4374075Z  [37mApplication bundle generation complete. [60.584 seconds] [39m...
...

:(

[Lms24]

I'm realizing this is not ideal but you should probably get away with using the CLI in the legacy way. This you can call really do after the build is finished (hence, the assets should be "physically" present) and the CLI won't touch your JS files to inject anything, meaning the hashes should stay valid

[pboling]

Not sure how helpful, or even if related, but I'm getting the same error with latest @sveltejs/kit (2.5.9) and none of the troubleshooting things are helping. It's almost as if the config settings are not relevant to the error / behavior at all. The defaults from the sveltekit wizard did not work.

[sentry-vite-plugin] Warning: Didn't find any matching sources for debug ID upload. Please check the `sourcemaps.assets` option.

[pboling]

Let me know if I should create a new issue... finding some hints that all is not well in the library.

I get similar typescript errors if I try to use any of these options:

1. url
2. build
3. setCommits
4. cleanArtifacts
5. rewrite

The documentation (typescript version) says we should be able to use cleanArtifacts: true, and in other places has examples with the other options I tried.

But it gives me an error:

I'm on the latest version of all packages as of today.

Worth noting also, that running my build script locally "works" with no errors, but also does not appear to build SourceMaps at all. Somehow building SourceMaps is triggered in CI, despite it being the same build script that runs, and that is where it is failing.

[lforst]

@pboling Would you mind opening a separate issue for this? Thank you!

[Lms24]

Hey @pboling for the documentation part, I just merged a PR to the SvelteKit SDK docs that shows the updated sentrySvelteKit options. This should have happened a while ago but it unfortunately slipped through. Apologies for that. Also I opened another one, to show more options (https://github.com/getsentry/sentry-docs/pull/10094).

If you still experience issues after migrating to the new options, then please open a new issue, thanks :)