Bundler updates pausing for 3-5m on each dependency update

pavera commented 7 months ago

Is there an existing issue for this?

[x] I have searched the existing issues

Package ecosystem

Bundler

Package manager version

No response

Language version

No response

Manifest location and content before the Dependabot update

No response

dependabot.yml content

No response

Updated dependency

No response

What you expected to see, versus what you actually saw

The update to complete successfully, but due to these long pauses the update job times out instead.

Native package manager behavior

No response

Images of the diff or a link to the PR, issue, or logs

No response

Smallest manifest that reproduces the issue

No response

pavera commented 7 months ago

@deivid-rodriguez I wonder if you could take a look at this with me?

abashir236 commented 7 months ago

@pavera any update on this?

deivid-rodriguez commented 7 months ago

@deivid-rodriguez I wonder if you could take a look at this with me?

@pavera Sorry I was travelling. Back now though! I have a lot to catch up with but I'm happy to help once I find some time. I guess first step is being able to reproduce the problem.

honeyankit commented 7 months ago

@deivid-rodriguez I am now looking at the issue and thank you for helping! I will try to reproduce the issue.

lucien-heart commented 7 months ago

@honeyankit any updates here team?

honeyankit commented 7 months ago

@lucien-heart I got all the manifest files from the customers, I am in the process to reproduce the issue today as @deivid-rodriguez requested.

honeyankit commented 7 months ago

@deivid-rodriguez :Based on analyzing the logs_21188243323.zip, I can see a pattern that, whenever there is call to 304 https://index.rubygems.org:443/versions there is a significant delay of more then ~22 seconds to up to 2 minutes almost through out the log. I am still working out reproduce it.

~2 minutes delay

2024-02-28T02:37:59.9713384Z   proxy | 2024/02/28 02:37:59 [503] GET https://index.rubygems.org:443/versions
2024-02-28T02:37:59.9714869Z   proxy | 2024/02/28 02:37:59 [503] 304 https://index.rubygems.org:443/versions
2024-02-28T02:40:09.8322964Z   proxy | 2024/02/28 02:40:09 [505] GET https://pkgs.shopify.io:443/basic/gems/ruby/versions
2024-02-28T02:40:09.8324673Z 2024/02/28 02:40:09 [505] 404 https://pkgs.shopify.io:443/basic/gems/ruby/versions

~1.5 minutes delay

2024-02-28T03:16:59.2061579Z   proxy | 2024/02/28 03:16:59 [379] GET https://index.rubygems.org:443/versions
2024-02-28T03:16:59.2062788Z 2024/02/28 03:16:59 [379] 304 https://index.rubygems.org:443/versions
2024-02-28T03:18:21.6670837Z   proxy | 2024/02/28 03:18:21 [381] GET https://pkgs.shopify.io:443/basic/gems/ruby/versions

~1.5 minutes delay

2024-02-28T03:33:29.5730780Z   proxy | 2024/02/28 03:33:29 [754] GET https://index.rubygems.org:443/versions
2024-02-28T03:33:29.5732404Z 2024/02/28 03:33:29 [754] 304 https://index.rubygems.org:443/versions
2024-02-28T03:34:54.7041503Z   proxy | 2024/02/28 03:34:54 [756] GET https://pkgs.shopify.io:443/basic/gems/ruby/versions
2024-02-28T03:34:54.7043148Z 2024/02/28 03:34:54 [756] 404 https://pkgs.shopify.io:443/basic/gems/ruby/versions

Edit: The initial assessment is not valid. The call to https://index.rubygems.org:443/versions is getting completed immediately but the call to GET https://pkgs.shopify.io:443/basic/gems/ruby/versions are taking time. Since the customers provided manifest files are using private repos. I am not able to reproduce the issue. @deivid-rodriguez do you have any suggestions where can I look?

deivid-rodriguez commented 6 months ago

@honeyankit Sounds like the request to https://pkgs.shopify.io:443/basic/gems/ruby/versions may be timing out and that's why it takes so long?

honeyankit commented 6 months ago

@honeyankit Sounds like the request to https://pkgs.shopify.io:443/basic/gems/ruby/versions may be timing out and that's why it takes so long?

Thank you @deivid-rodriguez . I have requested for the read access to debug with the debugger and will report my findings once I get the access.

honeyankit commented 6 months ago

@deivid-rodriguez One thing stood out is the affected repo is having 110+ gemspec file in their repo where dependabot job is timing out. The file parser and resolve_version functions are taking time in general. Also, at times the request to their private registry is slow about ~20 sec to 3 min making things worse.

Also, this repo is already onboarded on Dependabot on actions (private beta) which means caching is already enabled.

dependabot / dependabot-core