Error when updating pip packages

[bedzinsk]

About 5-10 days ago dependabot pipeline started failing for python packages, which have updated versions. There is no specific dependency which triggers this failure, basically any dependency update will result in similar stracktrace as below.

Stacktrace

Task         : Dependabot
Description  : Automatically update dependencies and vulnerabilities in your code
Version      : 1.14.420
Author       : Tingle Software
Help         : For help please visit https://github.com/tinglesoftware/dependabot-azure-devops
==============================================================================
/usr/bin/docker run --rm -i -e GITHUB_ACCESS_TOKEN=*** -e DEPENDABOT_PACKAGE_MANAGER=pip -e DEPENDABOT_OPEN_PULL_REQUESTS_LIMIT=10 -e DEPENDABOT_DIRECTORY=/ -e DEPENDABOT_VERSIONING_STRATEGY=auto -e DEPENDABOT_MILESTONE=115080 -e DEPENDABOT_EXTRA_CREDENTIALS=[{"type":"docker_registry","url":null,"registry":"<redacted>azurecr.io","username":"AzureDevopsDependabot","password":"***"}] -e DEPENDABOT_FAIL_ON_EXCEPTION=true -e AZURE_ORGANIZATION=<redacted> -e AZURE_PROJECT=<redacted> -e AZURE_REPOSITORY=<redacted> -e AZURE_ACCESS_TOKEN=*** -e AZURE_MERGE_STRATEGY=squash -e DEPENDABOT_FAIL_ON_EXCEPTION=false tingle/dependabot-azure-devops:0.14
warning: parser/current is loading parser/ruby31, which recognizes3.1.3-compliant syntax, but you are running 3.1.2.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
GitHub access token has been provided.
Fetching GitHub's GraphQL schema (should only happen once per run)
Using '[https://dev.azure.com:443/'](https://dev.azure.com/') as API endpoint
Pull Requests shall be linked to milestone (work item) 115080
Working in <redacted>/_git/<redacted>, 'default' branch under '/' directory
Looking for configuration file in the repository ...
.......
Using configuration file at '/.github/dependabot.yml' 😎
Using 'auto' requirements update strategy
Fetching pip dependency files ...
.......
.......
Found 4 dependency file(s) at commit 4d5b321973718f669df4f76af0f0feda3219f42d
 - /requirements-dev.txt
 - /requirements-tests.txt
 - /requirements.txt
 - /setup.py
Parsing dependencies information
Found 24 dependencies
.......
Checking if matplotlib 3.6.3 needs updating
🌍 --> GET https://pypi.org/simple/matplotlib/
🌍 <-- 200 ://pypi.org:443/simple/matplotlib/
🌍 --> GET https://pypi.org/simple/matplotlib/
🌍 <-- 200 ://pypi.org:443/simple/matplotlib/
Error working on updates for matplotlib 3.6.3 (continuing)
/home/dependabot/dependabot-script/vendor/ruby/3.1.0/gems/dependabot-common-0.215.0/lib/dependabot/update_checkers/base.rb:269:in `block in preferred_version_resolvable_with_unlock?': undefined method `[]' for nil:NilClass (NoMethodError)

        updated_requirements.none? { |r| r[:requirement] == :unfixable }
                                          ^^^^^^^^^^^^^^
    from /home/dependabot/dependabot-script/vendor/ruby/3.1.0/gems/dependabot-common-0.215.0/lib/dependabot/update_checkers/base.rb:269:in `none?'
    from /home/dependabot/dependabot-script/vendor/ruby/3.1.0/gems/dependabot-common-0.215.0/lib/dependabot/update_checkers/base.rb:269:in `preferred_version_resolvable_with_unlock?'
    from /home/dependabot/dependabot-script/vendor/ruby/3.1.0/gems/dependabot-python-0.215.0/lib/dependabot/python/update_checker.rb:126:in `preferred_version_resolvable_with_unlock?'
    from /home/dependabot/dependabot-script/vendor/ruby/3.1.0/gems/dependabot-common-0.215.0/lib/dependabot/update_checkers/base.rb:252:in `numeric_version_can_update?'
    from /home/dependabot/dependabot-script/vendor/ruby/3.1.0/gems/dependabot-common-0.215.0/lib/dependabot/update_checkers/base.rb:202:in `version_can_update?'
    from /home/dependabot/dependabot-script/vendor/ruby/3.1.0/gems/dependabot-common-0.215.0/lib/dependabot/update_checkers/base.rb:44:in `can_update?'
    from ./update-script.rb:551:in `block in <main>'
    from ./update-script.rb:503:in `each'
    from ./update-script.rb:503:in `<main>'
.......

.github/dependabot.yml

version: 2
updates:

- package-ecosystem: "pip"
  directory: "/"
  open-pull-requests-limit: 10
  milestone: 115080
  versioning-strategy: "auto"

- package-ecosystem: "docker"
  directory: "/"
  open-pull-requests-limit: 10
  milestone: 115080
  versioning-strategy: "auto"

registries:
  ob-acr:
    type: docker-registry
    url: redacted
    username: AzureDevopsDependabot
    password: ${{ ACR_PASSWORD }}

[mburumaxwell]

Without expertise in pip around, it may be difficult to tell whatis happening here. Could you offer a public repro in Azure DevOps?

[bedzinsk]

Thank you for getting back on this issue! Unfortunately, that is private repo, which cannot be made public.

Is there a way to try to run it with older version of the plugin or older version of dependabot container/dependency? As I said it was working perfectly fine before, when it suddenly stopped: so to me it looks it is either extension, either dependebot packages.

[mburumaxwell]

A repro is a representation of the said issue to allow others reproduce similar behavior without exposing anything sensitive. Surely anyone can do this with a dummy organization, even a personal one. Others have done it here. Here are some samples: https://dev.azure.com/tingle/dependabot/_git with the matching builds at https://dev.azure.com/tingle/dependabot/_build

That said, running older versions can be done using the dockerImageTag input. However, if you are going far behind, it may not work. It may be possibly to fix the task too but I haven't tried this, e.g - task: dependabot@1.10.0

[bedzinsk]

Update on the issue:

rerun the pipeline today and I do not see the old error anymore, but a new one:

Checking if coverage 7.1.0 needs updating
🌍 --> GET https://pypi.org/simple/coverage/
🌍 <-- 200 https://pypi.org/simple/coverage/
🌍 --> GET https://pypi.org/simple/coverage/
🌍 <-- 200 https://pypi.org/simple/coverage/

/home/dependabot/dependabot-updater/vendor/ruby/3.1.0/bundler/gems/dependabot-core-f8c48ef2f569/python/lib/dependabot/python/update_checker/requirements_updater.rb:197:in `updated_requirement': Unexpected update strategy: auto (RuntimeError)
    from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/bundler/gems/dependabot-core-f8c48ef2f569/python/lib/dependabot/python/update_checker/requirements_updater.rb:38:in `block in updated_requirements'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/bundler/gems/dependabot-core-f8c48ef2f569/python/lib/dependabot/python/update_checker/requirements_updater.rb:33:in `map'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/bundler/gems/dependabot-core-f8c48ef2f569/python/lib/dependabot/python/update_checker/requirements_updater.rb:33:in `updated_requirements'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/bundler/gems/dependabot-core-f8c48ef2f569/python/lib/dependabot/python/update_checker.rb:78:in `updated_requirements'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/bundler/gems/dependabot-core-f8c48ef2f569/common/lib/dependabot/update_checkers/base.rb:269:in `preferred_version_resolvable_with_unlock?'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/bundler/gems/dependabot-core-f8c48ef2f569/common/lib/dependabot/update_checkers/base.rb:252:in `numeric_version_can_update?'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/bundler/gems/dependabot-core-f8c48ef2f569/common/lib/dependabot/update_checkers/base.rb:202:in `version_can_update?'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/bundler/gems/dependabot-core-f8c48ef2f569/common/lib/dependabot/update_checkers/base.rb:44:in `can_update?'
    from bin/update-script.rb:583:in `block in <main>'
    from bin/update-script.rb:533:in `each'
    from bin/update-script.rb:533:in `<main>'

According to dependabot specs versioning-strategy auto should be supported by pip: https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#versioning-strategy

[mburumaxwell]

Maybe this should be an issue raised in the main repo at https://github.com/dependabot/dependabot-core ?

[bedzinsk]

Thank you for your support! Apparently the setup is very sensitive to different dependabot versions. From my side issue can be closed.