Error Broken pipe

[LeoColomb]

I'm facing error with some environments (but not all).

/usr/lib/ruby/2.5.0/openssl/buffering.rb:325:in `syswrite': Broken pipe (Errno::EPIPE)
    from /usr/lib/ruby/2.5.0/openssl/buffering.rb:325:in `do_write'
    from /usr/lib/ruby/2.5.0/openssl/buffering.rb:344:in `block in write'
    from /usr/lib/ruby/2.5.0/openssl/buffering.rb:343:in `each'
    from /usr/lib/ruby/2.5.0/openssl/buffering.rb:343:in `inject'
    from /usr/lib/ruby/2.5.0/openssl/buffering.rb:343:in `write'
    from /usr/lib/ruby/2.5.0/net/protocol.rb:233:in `write0'
    from /usr/lib/ruby/2.5.0/net/protocol.rb:207:in `block in write'
    from /usr/lib/ruby/2.5.0/net/protocol.rb:224:in `writing'
    from /usr/lib/ruby/2.5.0/net/protocol.rb:206:in `write'
    from /usr/lib/ruby/2.5.0/net/http/generic_request.rb:188:in `send_request_with_body'
    from /usr/lib/ruby/2.5.0/net/http/generic_request.rb:121:in `exec'
    from /usr/lib/ruby/2.5.0/net/http.rb:1492:in `block in transport_request'
    from /usr/lib/ruby/2.5.0/net/http.rb:1491:in `catch'
    from /usr/lib/ruby/2.5.0/net/http.rb:1491:in `transport_request'
    from /usr/lib/ruby/2.5.0/net/http.rb:1464:in `request'
    from /usr/lib/ruby/2.5.0/net/http.rb:1457:in `block in request'
    from /usr/lib/ruby/2.5.0/net/http.rb:910:in `start'
    from /usr/lib/ruby/2.5.0/net/http.rb:1455:in `request'
    from /.../vendor/ruby/2.5.0/gems/httparty-0.16.2/lib/httparty/request.rb:143:in `perform'
    from /.../vendor/ruby/2.5.0/gems/httparty-0.16.2/lib/httparty.rb:563:in `perform_request'
    from /.../vendor/ruby/2.5.0/gems/httparty-0.16.2/lib/httparty.rb:505:in `post'
    from /.../vendor/ruby/2.5.0/gems/gitlab-4.5.0/lib/gitlab/request.rb:44:in `block (2 levels) in <class:Request>'
    from /.../vendor/ruby/2.5.0/gems/gitlab-4.5.0/lib/gitlab/client/commits.rb:143:in `create_commit'
    from /.../vendor/ruby/2.5.0/bundler/gems/dependabot-core-cc95449efb78/lib/dependabot/pull_request_creator/gitlab.rb:101:in `create_commit'
    from /.../vendor/ruby/2.5.0/bundler/gems/dependabot-core-cc95449efb78/lib/dependabot/pull_request_creator/gitlab.rb:36:in `create'
    from /.../vendor/ruby/2.5.0/bundler/gems/dependabot-core-cc95449efb78/lib/dependabot/pull_request_creator.rb:54:in `create'
    from ./generic-update-script.rb:160:in `block in <main>'
    from ./generic-update-script.rb:110:in `each'
    from ./generic-update-script.rb:110:in `<main>'

Any idea where to start?

[greysteil]

Huh. That looks like it's a network problem related to GitLab's API or API client.

I'd look at the size of the files that you're trying to submit to GitLab (you should be able to call files on the PullRequestCreator object, I think, and inspect the length of the content of each file). If the files are massive (like, over 1mb) then it could be related to that. Otherwise it's more likely to be GitLab themselves having a reliability issue.

[LeoColomb]

Well, even when dumping the file into the console, the log is not that big (~200ko): https://gist.github.com/LeoColomb/ceb8f5e669279166d66cb388e6715b2c

For a basic Laravel app with Capistrano:

Will investigate GitLab.com API reliability...

[greysteil]

👍 - the next thing I'd check is whether this happens consistently or intermittently. In the later case it's almost certainly a GitLab bug (although perhaps one we can work around by automatically retrying).

[LeoColomb]

I'd check is whether this happens consistently or intermittently

Consistently, when updates are available (of course), on only 2 projects of 5.

[greysteil]

Can you run this on a specific repo, for a specific dependency, and get the same result each time (i.e., PR creation fails due to the broken pipe)?

(Apologies not to be able to take this off your hands - it's tricky when Dependabot doesn't currently put any volume through GitLab.)

[LeoColomb]

Strating to investigate further.

• I notice that branches are often created (the first one at every dependabot run before raising the error), but without any change. (this confirm the error is about commit creation)

• When I run update script on a private gitlab-runner (on my machine), only npm tasks fail. (this seems to exclude dependabot script bug)

• I've added Sentry error capturing: https://sentry.io/share/issue/170c7484fd1047938211008819c71cc8/

• Creating a repo for GitLab API test: it works well.

  • The repo: https://gitlab.com/gitlab-api-commit-error/tester
  • The script: https://gitlab.com/gitlab-api-commit-error/tester/blob/master/script.rb
  • The CI log: https://gitlab.com/gitlab-api-commit-error/tester/-/jobs/107919404
  • The branch and commit are created: https://gitlab.com/gitlab-api-commit-error/tester/compare/master...test-error

So is it related to API usage?

[LeoColomb]

Some additional notes:

• It always break at the first dep update for a single dependency file, after creating the branch but before the commit was successfully submitted.
• It took too long time to submit the commit to the GitLab API, I suspect that a timeout is raised somewhere during the commit submission, which cause the end of the script.
• This affects only updates for dependency (lock-)files that are > 20 ko.

[LeoColomb]

@greysteil I succeeded to isolate what is not failing:

I rewrote my tester repo script: https://gitlab.com/gitlab-api-commit-error/tester

• One build with dependabot (fail)
  • Script
  • CI log
  • Branch (empty)
• Another "manual" by passing the expected file (success)
  • Script
  • CI log
  • Branch

But, when comparing dependabot generated dependencies file with the expecting one, everything seems fine. (Script / CI log)

Soooo, the file content is OK, manually the request with that same file is OK, the branch creation is OK. But the dependabot request is still failing every time. 😢

I've suspected encoding mismatch, but if the manual request works, I can't see the dependabot may fail.

[LeoColomb]

Oh and what about the commit message?