search

getsentry / sentry

Developer-first error tracking and performance monitoring
https://sentry.io
Other
38.98k stars 4.18k forks source link

Can't upload Electron breakpad symbols to a Docker instance #7804

Closed DimitarNestorov closed 4 years ago

DimitarNestorov commented 6 years ago

Batch 1

Compressing 1 debug symbol files Uploading debug symbol files error: http error: [56] Failure when receiving data from the peer (Recv failure: Connection reset by peer) Found 3 debug information files Prepared debug information files for upload

Batch 1

Compressing 1 debug symbol files Uploading debug symbol files

Batch 2

Compressing 2 debug symbol files Uploading debug symbol files Found 84 debug information files Prepared debug information files for upload

Batch 1

Compressing 64 debug symbol files Uploading debug symbol files

Batch 2

Compressing 9 debug symbol files Uploading debug symbol files

Batch 3

Compressing 1 debug symbol files Uploading debug symbol files error: http error: [55] Failed sending data to the peer (Send failure: Broken pipe) Finished downloading and uploading to Sentry Feel free to delete the .electron-symbols

To test out if my computer was the issue I tried uploading to sentry.io and I succeeded. Also to test if my config was wrong I ran the sentry cli info command and got the email address that I used to register my account on my Docker instance.

The script somehow was able to upload 2 files, which I believe proves that my client config is fine: image

DimitarNestorov commented 6 years ago

By changing limit-post to 4294967296 (4 GB, electron symbols for mac are 1.44 GB) in SENTRY_WEB_OPTIONS we stopped getting http error 55. Now we only get error: http error: [56] Failure when receiving data from the peer (Recv failure: Connection reset by peer) and no new files came up in the Debug Information Files section of project settings.

DimitarNestorov commented 6 years ago

No more error 56 and error 55, the configuration for limit-post didn't apply. However no breakpad files are being uploaded. Only macOS symbols went through successfully. And I'm not getting error messages in the output of the script:

We are starting to download all possible electron symbols
We need it in order to symbolicate native crashes
This step is only needed once whenever you update your electron version
Just call this script again it should do everything for you.
> Found 7 debug information files
> Prepared debug information files for upload

Batch 1
> Compressing 1 debug symbol files
> Uploading debug symbol files

Batch 2
> Compressing 2 debug symbol files
> Uploading debug symbol files

Batch 3
> Compressing 1 debug symbol files
> Uploading debug symbol files

Batch 4
> Compressing 1 debug symbol files
> Uploading debug symbol files
> File upload complete:

  954f1c6c-2aea-30d2-be41-84b5cfb95867 (Electron Framework; x86_64)
  c11b8e11-edd2-37b9-9ab5-5fe9e97ffffb (Electron; x86_64)
  26bf611b-7acc-305a-9fc5-c535a513256f (Electron Helper; x86_64)
  d887239e-2626-3152-8fd6-5d25cf550a50 (libnode.dylib; x86_64)
  88e7d358-e6f8-3104-94dd-6586d9be6af6 (openssl-cli; x86_64)
> Found 3 debug information files
> Prepared debug information files for upload

Batch 1
> Compressing 1 debug symbol files
> Uploading debug symbol files

Batch 2
> Compressing 2 debug symbol files
> Uploading debug symbol files
> Found 3 debug information files
> Prepared debug information files for upload

Batch 1
> Compressing 1 debug symbol files
> Uploading debug symbol files

Batch 2
> Compressing 2 debug symbol files
> Uploading debug symbol files
> Found 84 debug information files
> Prepared debug information files for upload

Batch 1
> Compressing 64 debug symbol files
> Uploading debug symbol files

Batch 2
> Compressing 14 debug symbol files
> Uploading debug symbol files

Batch 3
> Compressing 1 debug symbol files
> Uploading debug symbol files

Batch 4
> Compressing 5 debug symbol files
> Uploading debug symbol files
Finished downloading and uploading to Sentry
Feel free to delete the .electron-symbols

Running the script a second time gives this output:

We are starting to download all possible electron symbols
We need it in order to symbolicate native crashes
This step is only needed once whenever you update your electron version
Just call this script again it should do everything for you.
> Found 7 debug information files
> Prepared debug information files for upload
> Nothing to upload, all files are on the server
> Nothing to upload
> Found 3 debug information files
> Prepared debug information files for upload

Batch 1
> Compressing 1 debug symbol files
> Uploading debug symbol files

Batch 2
> Compressing 2 debug symbol files
> Uploading debug symbol files
> Found 3 debug information files
> Prepared debug information files for upload

Batch 1
> Compressing 1 debug symbol files
> Uploading debug symbol files

Batch 2
> Compressing 2 debug symbol files
> Uploading debug symbol files
> Found 84 debug information files
> Prepared debug information files for upload

Batch 1
> Compressing 64 debug symbol files
> Uploading debug symbol files

Batch 2
> Compressing 14 debug symbol files
> Uploading debug symbol files

Batch 3
> Compressing 1 debug symbol files
> Uploading debug symbol files

Batch 4
> Compressing 5 debug symbol files
> Uploading debug symbol files
Finished downloading and uploading to Sentry
Feel free to delete the .electron-symbols
DimitarNestorov commented 6 years ago

I've come to the conclusion that 8.22 (latest release) can't accept breakpad symbols using this script. Adding loglevel info while running the script reveals that sentry cli makes a GET request to http://oursentryinstance.com:9000/api/0/organizations/codemotion/chunk-upload/ which returns 404. However it returns 200 on sentry.io. Looking at the code ChunkUpload didn't exist at the time 8.22 was released. Do you guys plan to make a release soon? I guess that as soon as you make the next release and we update I'll be closing the issue.

DimitarNestorov commented 6 years ago

@jan-auer Thanks for the information regarding this issue. Yes, I do not need the symbols for JavaScript errors, but would love to be able to report errors to the electron team in case a crash happens. Since most of our users are on macOS, this issue is not a big deal for us at the moment.

As for https://github.com/getsentry/sentry-electron/issues/54 I will try out your suggestion and report results back there.

jan-auer commented 6 years ago

Unfortunately, 8.22.0 does not yet support Electron native crashes. I will add this fact (and a version requirement for the future) to the documentation in case more people have the same issue.

As you stated correctly, required support for Breakpad symbols was only introduced after the last release. Sentry CLI will still try to upload the symbols but they are not processed and discarded by the server. This is why you receive an empty response after uploading those debug symbols.

Apart from that, we have introduced a new upload endpoint, called "chunk-upload". This allows us to upload larger files than what was possible before. The "broken pipe" error you are receiving is the result of trying to upload a 1.5GB debug symbol over the prior endpoint, which results in a connection reset. Also, the symbolication library in 8.22.0 is a bit outdated, so native stack traces even from macOS might not have the same quality.

BYK commented 4 years ago

Closing this issue due to staleness. Feel free to comment here if you think we should still work on this.