search

getsentry / sentry-cli

A command line utility to work with Sentry.
https://docs.sentry.io/cli/
BSD 3-Clause "New" or "Revised" License
903 stars 221 forks source link

sourcemaps upload - [56] Failure when receiving data from the peer (OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 0) #2155

Closed renepupil closed 1 month ago

renepupil commented 1 month ago

CLI Version

2.36.1

Operating System and Architecture

Operating System Version

Ubuntu 24.04.1 LTS

$ curl --version
curl 8.5.0 (x86_64-pc-linux-gnu) libcurl/8.5.0 OpenSSL/3.0.13 zlib/1.3 brotli/1.1.0 zstd/1.5.5 libidn2/2.3.7 libpsl/0.21.2 (+libidn2/2.3.7) libssh/0.10.6/openssl/zlib nghttp2/1.59.0 librtmp/2.3 OpenLDAP/2.6.7
Release-Date: 2023-12-06, security patched: 8.5.0-2ubuntu10.3
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

$ openssl version
OpenSSL 3.0.13 30 Jan 2024 (Library: OpenSSL 3.0.13 30 Jan 2024)

Link to reproduction repository

No response

CLI Command

sentry-cli --url https://errors.<self-hosted>.de --auth-token "<token>" sourcemaps upload --org <org> --project <project> --release 2.13.2 --url-prefix '~/app/2.13/<module>/assets/' 'vue_app_2_13/<module>/assets'

Exact Reproduction Steps

Run the command above on a Linux I guess, I think I can NOT expose company details, therefore I replaced any with placeholders.

MacOS of a colleague of mine seems to work, so I guess it is a Linux specific issue.

Expected Results

Succesful upload of source maps.

Actual Results

Error: 

error: API request failed
  caused by: [56] Failure when receiving data from the peer (OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 0)

Logs

Again I replaced all company related information with placeholders <...>:

  DEBUG   2024-09-16 15:11:49.036338280 +02:00 sentry-cli version: 2.36.1, platform: "linux", architecture: "x86_64"
  INFO    2024-09-16 15:11:49.037076126 +02:00 sentry-cli was invoked with the following command line: "/home/<user>/.nvm/versions/node/v20.13.1/lib/node_modules/@sentry/cli/node_modules/@sentry/cli-linux-x64/bin/sentry-cli" "--log-level" "debug" "--url" "https://errors.<self-hosted>.de" "--auth-token" "[REDACTED]" "sourcemaps" "upload" "--org" "<org>" "--project" "<project>" "--release" "2.13.2" "--url-prefix" "~/app/2.13/<module>/assets/" "vue_app_2_13/<module>/assets"
  DEBUG   2024-09-16 15:11:49.037514814 +02:00 request GET https://errors.<self-hosted>.de/api/0/organizations/<org>/chunk-upload/
  DEBUG   2024-09-16 15:11:49.037530326 +02:00 using token authentication
  DEBUG   2024-09-16 15:11:49.037545738 +02:00 retry number 0, max retries: 0
  DEBUG   2024-09-16 15:11:49.119976369 +02:00 > GET /api/0/organizations/<org>/chunk-upload/ HTTP/1.1
  DEBUG   2024-09-16 15:11:49.120001597 +02:00 > Host: errors.<self-hosted>.de
  DEBUG   2024-09-16 15:11:49.120008455 +02:00 > Accept: */*
  DEBUG   2024-09-16 15:11:49.120014007 +02:00 > Connection: TE
  DEBUG   2024-09-16 15:11:49.120019397 +02:00 > TE: gzip
  DEBUG   2024-09-16 15:11:49.120024820 +02:00 > User-Agent: sentry-cli/2.36.1
  DEBUG   2024-09-16 15:11:49.120410622 +02:00 > Authorization: Bearer sntrys_e***
  DEBUG   2024-09-16 15:11:49.155253529 +02:00 < HTTP/1.1 200 OK
  DEBUG   2024-09-16 15:11:49.155303171 +02:00 < Server: nginx/1.18.0
  DEBUG   2024-09-16 15:11:49.155313615 +02:00 < Date: Mon, 16 Sep 2024 13:11:49 GMT
  DEBUG   2024-09-16 15:11:49.155322111 +02:00 < Content-Type: application/json
  DEBUG   2024-09-16 15:11:49.155331497 +02:00 < Content-Length: 313
  DEBUG   2024-09-16 15:11:49.155343600 +02:00 < Connection: close
  DEBUG   2024-09-16 15:11:49.155351832 +02:00 < Allow: GET, POST, HEAD, OPTIONS
  DEBUG   2024-09-16 15:11:49.155359885 +02:00 < Access-Control-Allow-Methods: GET, POST, HEAD, OPTIONS
  DEBUG   2024-09-16 15:11:49.155378540 +02:00 < Access-Control-Allow-Headers: X-Sentry-Auth, X-Requested-With, Origin, Accept, Content-Type, Authentication, Authorization, Content-Encoding, sentry-trace, baggage, X-CSRFToken
  DEBUG   2024-09-16 15:11:49.155389439 +02:00 < Access-Control-Expose-Headers: X-Sentry-Error, X-Sentry-Direct-Hit, X-Hits, X-Max-Hits, Endpoint, Retry-After, Link
  DEBUG   2024-09-16 15:11:49.155398256 +02:00 < Access-Control-Allow-Origin: *
  DEBUG   2024-09-16 15:11:49.155406612 +02:00 < Vary: Accept-Language, Cookie
  DEBUG   2024-09-16 15:11:49.155414241 +02:00 < Content-Language: en
  DEBUG   2024-09-16 15:11:49.155421705 +02:00 < X-Frame-Options: deny
  DEBUG   2024-09-16 15:11:49.155428894 +02:00 < X-Content-Type-Options: nosniff
  DEBUG   2024-09-16 15:11:49.155436263 +02:00 < X-XSS-Protection: 1; mode=block
  DEBUG   2024-09-16 15:11:49.155444381 +02:00 < Content-Security-Policy-Report-Only: frame-ancestors 'none'; media-src *; script-src 'self' 'unsafe-inline' 'report-sample'; img-src blob: data: *; style-src 'unsafe-inline' *; default-src 'none'; connect-src 'self' *.algolia.net *.algolianet.com *.algolia.io; object-src 'none'; font-src 'self' data:; base-uri 'none'
  DEBUG   2024-09-16 15:11:49.155456351 +02:00 < Strict-Transport-Security: max-age=31536000
  DEBUG   2024-09-16 15:11:49.156577525 +02:00 response status: 200
  DEBUG   2024-09-16 15:11:49.156606430 +02:00 body: {"url":"organizations/<org>/chunk-upload/","chunkSize":8388608,"chunksPerRequest":64,"maxFileSize":2147483648,"maxRequestSize":33554432,"concurrency":8,"hashAlgorithm":"sha1","compression":["gzip"],"accept":["debug_files","release_files","pdbs","sources","bcsymbolmaps","il2cpp","portablepdbs","artifact_bundles"]}
  DEBUG   2024-09-16 15:11:49.158393780 +02:00 built glob set; 0 literals, 0 basenames, 0 extensions, 0 prefixes, 0 suffixes, 6 required extensions, 0 regexes  (from globset)
  DEBUG   2024-09-16 15:11:49.160004375 +02:00 built glob set; 1 literals, 0 basenames, 0 extensions, 0 prefixes, 0 suffixes, 0 required extensions, 0 regexes  (from globset)
  DEBUG   2024-09-16 15:11:49.160372664 +02:00 whitelisting vue_app_2_13/<module>/assets/index-legacy-jtx53sui.js.map: Whitelist(IgnoreMatch(Types(Glob(Matched { def: FileTypeDef { name: "map", globs: ["*.map"] } }))))  (from ignore)
  INFO    2024-09-16 15:11:49.160450682 +02:00 found: vue_app_2_13/<module>/assets/index-legacy-jtx53sui.js.map (5179788 bytes)
  DEBUG   2024-09-16 15:11:49.166006424 +02:00 whitelisting vue_app_2_13/<module>/assets/polyfills-B0erBpCs.js: Whitelist(IgnoreMatch(Types(Glob(Matched { def: FileTypeDef { name: "js", globs: ["*.js"] } }))))  (from ignore)
  INFO    2024-09-16 15:11:49.166091482 +02:00 found: vue_app_2_13/<module>/assets/polyfills-B0erBpCs.js (92973 bytes)
  DEBUG   2024-09-16 15:11:49.166207894 +02:00 ignoring vue_app_2_13/<module>/assets/<suborg>_logo-lh0h0xtw.svg: Ignore(IgnoreMatch(Types(Glob(UnmatchedIgnore))))  (from ignore)
  DEBUG   2024-09-16 15:11:49.166233999 +02:00 ignoring vue_app_2_13/<module>/assets/materialdesignicons-webfont-iyw0arvg.eot: Ignore(IgnoreMatch(Types(Glob(UnmatchedIgnore))))  (from ignore)
  DEBUG   2024-09-16 15:11:49.166259075 +02:00 whitelisting vue_app_2_13/<module>/assets/index-u9onhc0c.js.map: Whitelist(IgnoreMatch(Types(Glob(Matched { def: FileTypeDef { name: "map", globs: ["*.map"] } }))))  (from ignore)
  INFO    2024-09-16 15:11:49.166296914 +02:00 found: vue_app_2_13/<module>/assets/index-u9onhc0c.js.map (5278465 bytes)
  DEBUG   2024-09-16 15:11:49.169072098 +02:00 whitelisting vue_app_2_13/<module>/assets/index-legacy-jtx53sui.js: Whitelist(IgnoreMatch(Types(Glob(Matched { def: FileTypeDef { name: "js", globs: ["*.js"] } }))))  (from ignore)
  INFO    2024-09-16 15:11:49.169139304 +02:00 found: vue_app_2_13/<module>/assets/index-legacy-jtx53sui.js (2697382 bytes)
  DEBUG   2024-09-16 15:11:49.170406729 +02:00 ignoring vue_app_2_13/<module>/assets/materialdesignicons-webfont-61hda57n.woff: Ignore(IgnoreMatch(Types(Glob(UnmatchedIgnore))))  (from ignore)
  DEBUG   2024-09-16 15:11:49.170446806 +02:00 whitelisting vue_app_2_13/<module>/assets/polyfills-legacy-DjlTE9fn.js: Whitelist(IgnoreMatch(Types(Glob(Matched { def: FileTypeDef { name: "js", globs: ["*.js"] } }))))  (from ignore)
  INFO    2024-09-16 15:11:49.170485554 +02:00 found: vue_app_2_13/<module>/assets/polyfills-legacy-DjlTE9fn.js (52881 bytes)
  DEBUG   2024-09-16 15:11:49.170568828 +02:00 ignoring vue_app_2_13/<module>/assets/index-korprj88.css: Ignore(IgnoreMatch(Types(Glob(UnmatchedIgnore))))  (from ignore)
  DEBUG   2024-09-16 15:11:49.170591419 +02:00 whitelisting vue_app_2_13/<module>/assets/index-u9onhc0c.js: Whitelist(IgnoreMatch(Types(Glob(Matched { def: FileTypeDef { name: "js", globs: ["*.js"] } }))))  (from ignore)
  INFO    2024-09-16 15:11:49.170623651 +02:00 found: vue_app_2_13/<module>/assets/index-u9onhc0c.js (1955342 bytes)
  DEBUG   2024-09-16 15:11:49.171614914 +02:00 ignoring vue_app_2_13/<module>/assets/error404-g932kmgg.svg: Ignore(IgnoreMatch(Types(Glob(UnmatchedIgnore))))  (from ignore)
  DEBUG   2024-09-16 15:11:49.171646484 +02:00 ignoring vue_app_2_13/<module>/assets/materialdesignicons-webfont-n4sq6p8n.woff2: Ignore(IgnoreMatch(Types(Glob(UnmatchedIgnore))))  (from ignore)
  DEBUG   2024-09-16 15:11:49.171666180 +02:00 ignoring vue_app_2_13/<module>/assets/materialdesignicons-webfont-hlpg5jvj.ttf: Ignore(IgnoreMatch(Types(Glob(UnmatchedIgnore))))  (from ignore)
> Found 6 files
> Analyzing 6 sources
> Analyzing completed in 0.023s
> Rewriting sources
> Rewriting completed in 0.183s
> Adding source map references
  DEBUG   2024-09-16 15:11:49.448225205 +02:00 request GET https://errors.<self-hosted>.de/api/0/projects/<org>/<project>/releases/2.13.2/files/?cursor=&checksum=0229acb0c2bbf7134af7c3ebc79a68dea8fdc6d3&checksum=4a4e0d2c4962f9da5b792acc16c88c286b8cfed2&checksum=6acdb46eb53acc8d99c87eed093d69b8f032cb32&checksum=74d58baa984f4baf0ff6ae6b3677b08e9fa2b2b1&checksum=98e86a98275a7134ab0ae8d3fd8bf63fc3f1d381&checksum=d46fd3542d08dbe63694da1e2a56ba7a4c4d6ac3
  DEBUG   2024-09-16 15:11:49.448248366 +02:00 using token authentication
  DEBUG   2024-09-16 15:11:49.448255343 +02:00 retry number 0, max retries: 0
  DEBUG   2024-09-16 15:11:49.484859203 +02:00 > GET /api/0/projects/<org>/<project>/releases/2.13.2/files/?cursor=&checksum=0229acb0c2bbf7134af7c3ebc79a68dea8fdc6d3&checksum=4a4e0d2c4962f9da5b792acc16c88c286b8cfed2&checksum=6acdb46eb53acc8d99c87eed093d69b8f032cb32&checksum=74d58baa984f4baf0ff6ae6b3677b08e9fa2b2b1&checksum=98e86a98275a7134ab0ae8d3fd8bf63fc3f1d381&checksum=d46fd3542d08dbe63694da1e2a56ba7a4c4d6ac3 HTTP/1.1
  DEBUG   2024-09-16 15:11:49.484888846 +02:00 > Host: errors.<self-hosted>.de
  DEBUG   2024-09-16 15:11:49.484895526 +02:00 > Accept: */*
  DEBUG   2024-09-16 15:11:49.484901230 +02:00 > Connection: TE
  DEBUG   2024-09-16 15:11:49.484906445 +02:00 > TE: gzip
  DEBUG   2024-09-16 15:11:49.484912242 +02:00 > User-Agent: sentry-cli/2.36.1
  DEBUG   2024-09-16 15:11:49.484937195 +02:00 > Authorization: Bearer sntrys_e***
  DEBUG   2024-09-16 15:11:49.539426619 +02:00 < HTTP/1.1 404 Not Found
  DEBUG   2024-09-16 15:11:49.539524554 +02:00 < Server: nginx/1.18.0
  DEBUG   2024-09-16 15:11:49.539560856 +02:00 < Date: Mon, 16 Sep 2024 13:11:49 GMT
  DEBUG   2024-09-16 15:11:49.539594509 +02:00 < Content-Type: application/json
  DEBUG   2024-09-16 15:11:49.539633238 +02:00 < Content-Length: 50
  DEBUG   2024-09-16 15:11:49.539669550 +02:00 < Connection: close
  DEBUG   2024-09-16 15:11:49.539697746 +02:00 < Allow: GET, POST, HEAD, OPTIONS
  DEBUG   2024-09-16 15:11:49.539726098 +02:00 < Access-Control-Allow-Methods: GET, POST, HEAD, OPTIONS
  DEBUG   2024-09-16 15:11:49.539768409 +02:00 < Access-Control-Allow-Headers: X-Sentry-Auth, X-Requested-With, Origin, Accept, Content-Type, Authentication, Authorization, Content-Encoding, sentry-trace, baggage, X-CSRFToken
  DEBUG   2024-09-16 15:11:49.539805724 +02:00 < Access-Control-Expose-Headers: X-Sentry-Error, X-Sentry-Direct-Hit, X-Hits, X-Max-Hits, Endpoint, Retry-After, Link
  DEBUG   2024-09-16 15:11:49.539835731 +02:00 < Access-Control-Allow-Origin: *
  DEBUG   2024-09-16 15:11:49.539862152 +02:00 < Vary: Accept-Language, Cookie
  DEBUG   2024-09-16 15:11:49.539892587 +02:00 < Content-Language: en
  DEBUG   2024-09-16 15:11:49.539922472 +02:00 < X-Frame-Options: deny
  DEBUG   2024-09-16 15:11:49.539952583 +02:00 < X-Content-Type-Options: nosniff
  DEBUG   2024-09-16 15:11:49.539981947 +02:00 < X-XSS-Protection: 1; mode=block
  DEBUG   2024-09-16 15:11:49.540016588 +02:00 < Content-Security-Policy-Report-Only: frame-ancestors 'none'; media-src *; script-src 'self' 'unsafe-inline' 'report-sample'; img-src blob: data: *; style-src 'unsafe-inline' *; default-src 'none'; connect-src 'self' *.algolia.net *.algolianet.com *.algolia.io; object-src 'none'; font-src 'self' data:; base-uri 'none'
  DEBUG   2024-09-16 15:11:49.542995909 +02:00 response status: 404
> Bundling completed in 0.641s
> Bundled 6 files for upload
> Bundle ID: 47209531-7669-5c80-95d0-fad07fba030f
> Optimizing completed in 0.029s
  INFO    2024-09-16 15:11:50.223874474 +02:00 using 'gzip' compression for chunk upload
  DEBUG   2024-09-16 15:11:50.377533130 +02:00 request POST https://errors.<self-hosted>.de/api/0/organizations/<org>/chunk-upload/
  DEBUG   2024-09-16 15:11:50.377692495 +02:00 using token authentication
  DEBUG   2024-09-16 15:11:50.377715806 +02:00 sending form data
  DEBUG   2024-09-16 15:11:50.377737705 +02:00 retry number 0, max retries: 5
  DEBUG   2024-09-16 15:11:50.417189325 +02:00 > POST /api/0/organizations/<org>/chunk-upload/ HTTP/1.1
  DEBUG   2024-09-16 15:11:50.417272381 +02:00 > Host: errors.<self-hosted>.de
  DEBUG   2024-09-16 15:11:50.417313253 +02:00 > Accept: */*
  DEBUG   2024-09-16 15:11:50.417349568 +02:00 > Connection: TE
  DEBUG   2024-09-16 15:11:50.417385638 +02:00 > TE: gzip
  DEBUG   2024-09-16 15:11:50.417421924 +02:00 > User-Agent: sentry-cli/2.36.1
  DEBUG   2024-09-16 15:11:50.417667338 +02:00 > Authorization: Bearer sntrys_e***
  DEBUG   2024-09-16 15:11:50.417689154 +02:00 > Content-Length: 4121790
  DEBUG   2024-09-16 15:11:50.417708991 +02:00 > Content-Type: multipart/form-data; boundary=------------------------vcM9HHAzaHa6xRBH6e9YCH
error: API request failed
  caused by: [56] Failure when receiving data from the peer (OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 0)
  INFO    2024-09-16 15:11:50.794919566 +02:00 Skipping update nagger update check
szokeasaurusrex commented 1 month ago

Hey @renepupil, when did you notice this issue starting? Did it appear after upgrading Sentry CLI? If so, is there an older Sentry CLI version that works?

We downgraded our curl dependency in 2.36.1 due to an issue, does 2.36.0 work for you?

renepupil commented 1 month ago

@szokeasaurusrex Thanks for the quick response, 2.36.0 works!

Thank you very much!

So I guess we can use fixed version 2.36.0 for now, but hope you check for a fix for future versions...

when did you notice this issue starting? Did it appear after upgrading Sentry CLI?

I guess it started with a new sentry version we use npm install -g @sentry/cli && npm update -g @sentry/cli in a script.

> Found 6 files
> Analyzing 6 sources
> Analyzing completed in 0.023s
> Rewriting sources
> Rewriting completed in 0.179s
> Adding source map references
> Bundling completed in 0.653s
> Bundled 6 files for upload
> Bundle ID: <ID>
> Optimizing completed in 0.029s
> Uploading completed in 0.497s
> Uploaded files to Sentry
> Processing completed in 0.074s
> File processing complete
> Organization: <org>
> Project: <project>
> Release: 2.13.2
> Dist: None
> Upload type: artifact bundle

Source Map Upload Report
  Scripts
    ~/app/2.13/<module>/assets/index-legacy-jtx53sui.js (sourcemap at index-legacy-jtx53sui.js.map)
    ~/app/2.13/<module>/assets/index-u9onhc0c.js (sourcemap at index-u9onhc0c.js.map)
    ~/app/2.13/<module>/assets/polyfills-B0erBpCs.js (no sourcemap ref)
      - warning: could not determine a source map reference (Could not auto-detect referenced sourcemap for ~/app/2.13/<module>/assets/polyfills-B0erBpCs.js)
    ~/app/2.13/<module>/assets/polyfills-legacy-DjlTE9fn.js (no sourcemap ref)
      - warning: could not determine a source map reference (Could not auto-detect referenced sourcemap for ~/app/2.13/<module>/assets/polyfills-legacy-DjlTE9fn.js)
  Source Maps
    ~/app/2.13/<module>/assets/index-legacy-jtx53sui.js.map
    ~/app/2.13/<module>/assets/index-u9onhc0c.js.map> Found 6 files
szokeasaurusrex commented 1 month ago

Ok @renepupil, glad to hear this workaround works. I will have to investigate further to find a good solution; I could revert the change in 2.36.1, but then this bug would come back, which is not ideal.

I am at a conference through tomorrow, so I will only be able to think about this further from Thursday this week.

webmozart commented 1 month ago

Our Linux CI runners are affected by this as well

szokeasaurusrex commented 1 month ago

Our Linux CI runners are affected by this as well

Good to know @webmozart.

Are you also able to work around the issue by downgrading to Sentry CLI 2.36.0?

Jamira40 commented 1 month ago

Hi, Our CI was also affected. Downgrading to 2.36.0 helped.

szokeasaurusrex commented 1 month ago

@Jamira40 thanks for letting us know

szokeasaurusrex commented 1 month ago

We are planning to bump curl-sys to the latest version, which should hopefully fix this issue, since the issue originally arose after a downgrade of curl-sys in an attempt to solve a different issue (#2148).

renepupil commented 2 weeks ago

Change seems to work, thanks