tinglesoftware / dependabot-azure-devops

Tools for updating dependencies in Azure DevOps repositories using https://dependabot.com
MIT License
198 stars 65 forks source link

Unable to pass correct url for private registry for composer-repository #1307

Closed BertKooij closed 2 months ago

BertKooij commented 2 months ago

Describe the bug I tried different options for providing a private composer-repository but the all seem to result in an error (See below). A while ago i had Dependabot running for the same repository but that stopped working one day and I only tried recently to get it working again. Previous it was working withthe following variable:

variables:
  DEPENDABOT_EXTRA_CREDENTIALS: '[{"type":"composer_repository","host":"repo.****.nl","registry":"repo.*****.nl","username":"****@*****.nl","password":"$(REPO_API_KEY)"}]'

This stopped working one day and resulted in a 401 error form the custom repo. Since than I moved the configuration to a registry in the dependabot.yml file:

version: 2

registries:
  composer:
    type: composer-repository
    url: 'https://repo.******.nl'
    username: '*****@****.nl'
    password: ${{REPO_API_KEY}}

updates:
  - package-ecosystem: "composer"
    directory: "/"
    target-branch: "main"
    registries:
      - composer
    schedule:
      interval: "daily"

That doesn't result in a 401 anymore but it does raise the following error. I tried removing the schema from the repo url and a few other options but they all result in the same error.

Checking if laravel/framework 10.43.0 needs updating
/usr/local/lib/ruby/3.3.0/uri/common.rb:848:in `URI': bad argument (expected URI object or URI string) (ArgumentError)
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker/latest_version_finder.rb:127:in `block in fetch_registry_versions_from_url'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker/latest_version_finder.rb:127:in `each'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker/latest_version_finder.rb:127:in `find'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker/latest_version_finder.rb:127:in `fetch_registry_versions_from_url'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker/latest_version_finder.rb:120:in `block in registry_version_details'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker/latest_version_finder.rb:119:in `each'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker/latest_version_finder.rb:119:in `registry_version_details'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker/latest_version_finder.rb:96:in `available_versions'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker/latest_version_finder.rb:44:in `fetch_latest_version'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker/latest_version_finder.rb:28:in `latest_version'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker.rb:98:in `latest_version_from_registry'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker.rb:24:in `latest_version'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-common-0.259.0/lib/dependabot/update_checkers/base.rb:314:in `numeric_version_up_to_date?'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11406/lib/types/private/methods/call_validation.rb:270:in `bind_call'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11406/lib/types/private/methods/call_validation.rb:270:in `validate_call'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11406/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-common-0.259.0/lib/dependabot/update_checkers/base.rb:267:in `version_up_to_date?'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11406/lib/types/private/methods/call_validation.rb:270:in `bind_call'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11406/lib/types/private/methods/call_validation.rb:270:in `validate_call'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11406/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-common-0.259.0/lib/dependabot/update_checkers/base.rb:82:in `up_to_date?'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11406/lib/types/private/methods/call_validation.rb:270:in `bind_call'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11406/lib/types/private/methods/call_validation.rb:270:in `validate_call'
    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11406/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
    from bin/update_script.rb:585:in `block in <main>'
    from bin/update_script.rb:548:in `each'
    from bin/update_script.rb:548:in `<main>'
##[error]The process '/usr/bin/docker' failed with exit code 1

Dependabot pipeline:

trigger: none # Disable CI trigger

schedules:
- cron: '0 2 * * 3' # weekly at wednesday at 2am UTC
  always: true # run even when there are no code changes
  branches:
    include:
      - main
  batch: true
  displayName: Weekly

pool:
  vmImage: 'ubuntu-latest' # requires macos or ubuntu (windows is not supported)

steps:
  - task: dependabot@1
    inputs:
      openPullRequestsLimit: 25
      useConfigFile: true
      targetBranch: 'main'
      gitHubConnection: 'GithubRepos'
      azureDevOpsAccessToken: '$(TOKEN)'
      gitHubAccessToken: '$(GITHUB_TOKEN)'
      extraEnvironmentVariables: 'REPO_API_KEY=$(REPO_API_KEY)'

Dependabot.yml

version: 2

registries:
  composer:
    type: composer-repository
    url: 'https://repo.*****.nl'
    username: '*****@*****.nl'
    password: ${{REPO_API_KEY}}

updates:
  - package-ecosystem: "composer"
    directory: "/"
    target-branch: "main"
    registries:
      - composer
    schedule:
      interval: "daily"

Full output:

2024-08-23T21:07:21.2699674Z ##[section]Starting: dependabot
2024-08-23T21:07:21.2706160Z ==============================================================================
2024-08-23T21:07:21.2706313Z Task         : Dependabot
2024-08-23T21:07:21.2706381Z Description  : Automatically update dependencies and vulnerabilities in your code
2024-08-23T21:07:21.2706492Z Version      : 1.31.826
2024-08-23T21:07:21.2706570Z Author       : Tingle Software
2024-08-23T21:07:21.2706641Z Help         : https://github.com/tinglesoftware/dependabot-azure-devops/issues
2024-08-23T21:07:21.2706750Z ==============================================================================
2024-08-23T21:07:24.5080552Z [command]/usr/bin/docker run --rm -i -e GITHUB_ACCESS_TOKEN=*** -e DEPENDABOT_PACKAGE_MANAGER=composer -e DEPENDABOT_OPEN_PULL_REQUESTS_LIMIT=5 -e DEPENDABOT_DIRECTORY=/ -e DEPENDABOT_TARGET_BRANCH=main -e DEPENDABOT_EXTRA_CREDENTIALS=[{"type":"composer_repository","username":"*****@*****.nl","password":"***","url":"https://repo.*****.nl"},{"type":"git","username":"x-access-token","password":"***","url":"https://github.com"}] -e DEPENDABOT_FAIL_ON_EXCEPTION=true -e AZURE_ORGANIZATION=***** -e AZURE_PROJECT=***** -e AZURE_REPOSITORY=***** -e AZURE_ACCESS_TOKEN=*** -e AZURE_MERGE_STRATEGY=squash -e REPO_API_KEY=*** -e GITHUB_TOKEN=*** ghcr.io/tinglesoftware/dependabot-updater-composer:1.29.0 update_script
2024-08-23T21:07:24.6416294Z Unable to find image 'ghcr.io/tinglesoftware/dependabot-updater-composer:1.29.0' locally
2024-08-23T21:07:25.5474640Z 1.29.0: Pulling from tinglesoftware/dependabot-updater-composer
2024-08-23T21:07:25.5480949Z 4a023cab5400: Pulling fs layer
[...]
2024-08-23T21:08:30.0987882Z 02d07cd683a2: Pull complete
2024-08-23T21:08:30.1030927Z Digest: sha256:0d3f1d642aa32fae474e4d6b1b0f1de3e1e1d244e03bb35dc1b641c462435477
2024-08-23T21:08:30.1046477Z Status: Downloaded newer image for ghcr.io/tinglesoftware/dependabot-updater-composer:1.29.0
2024-08-23T21:08:31.9610531Z warning: parser/current is loading parser/ruby33, which recognizes 3.3.2-compliant syntax, but you are running 3.3.1.
2024-08-23T21:08:31.9611013Z Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
2024-08-23T21:08:32.8901851Z GitHub access token has been provided.
2024-08-23T21:08:32.8903873Z Using 'https://dev.azure.com:443/' as API endpoint
2024-08-23T21:08:32.8904277Z Pull Requests shall be linked to milestone (work item) 0
2024-08-23T21:08:32.8904733Z Working in *****/*****/_git/*****, 'main' branch under '/' directory
2024-08-23T21:08:32.8905235Z Cloning repository into /home/dependabot/dependabot-updater/tmp/*****/*****/_git/*****
2024-08-23T21:08:35.9722196Z Found 2 dependency file(s) at commit *****
2024-08-23T21:08:35.9723071Z  - /composer.json
2024-08-23T21:08:35.9723828Z  - /composer.lock
2024-08-23T21:08:35.9724167Z Parsing dependencies information
2024-08-23T21:08:36.0606863Z Found 74 dependencies
2024-08-23T21:08:36.0616065Z  - laravel/framework (10.43.0)
[...]
2024-08-23T21:08:36.0641632Z  - phpunit/phpunit (10.5.9)
2024-08-23T21:08:36.0642191Z  - laravel/sail (1.27.3)
2024-08-23T21:08:36.0643021Z 🌍 --> GET https://dev.azure.com/*****/_apis/connectionData
2024-08-23T21:08:36.1449320Z 🌍 <-- 200 https://dev.azure.com/*****/_apis/connectionData
2024-08-23T21:08:36.1459118Z 🌍 --> GET https://dev.azure.com/*****/*****/_apis/git/repositories/*****/pullrequests?api-version=7.1&searchCriteria.status=active&searchCriteria.creatorId=*****&searchCriteria.targetRefName=refs/heads/main
2024-08-23T21:08:36.2405723Z 🌍 <-- 200 https://dev.azure.com/*****/*****/_apis/git/repositories/*****/pullrequests?api-version=7.1&searchCriteria.status=active&searchCriteria.creatorId=*****&searchCriteria.targetRefName=refs/heads/main
2024-08-23T21:08:36.2415194Z Checking if laravel/framework 10.43.0 needs updating
2024-08-23T21:08:36.5246236Z /usr/local/lib/ruby/3.3.0/uri/common.rb:848:in `URI': bad argument (expected URI object or URI string) (ArgumentError)
2024-08-23T21:08:36.5247053Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker/latest_version_finder.rb:127:in `block in fetch_registry_versions_from_url'
2024-08-23T21:08:36.5247794Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker/latest_version_finder.rb:127:in `each'
2024-08-23T21:08:36.5248677Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker/latest_version_finder.rb:127:in `find'
2024-08-23T21:08:36.5249454Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker/latest_version_finder.rb:127:in `fetch_registry_versions_from_url'
2024-08-23T21:08:36.5250234Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker/latest_version_finder.rb:120:in `block in registry_version_details'
2024-08-23T21:08:36.5251183Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker/latest_version_finder.rb:119:in `each'
2024-08-23T21:08:36.5251872Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker/latest_version_finder.rb:119:in `registry_version_details'
2024-08-23T21:08:36.5252622Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker/latest_version_finder.rb:96:in `available_versions'
2024-08-23T21:08:36.5253373Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker/latest_version_finder.rb:44:in `fetch_latest_version'
2024-08-23T21:08:36.5254034Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker/latest_version_finder.rb:28:in `latest_version'
2024-08-23T21:08:36.5254736Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker.rb:98:in `latest_version_from_registry'
2024-08-23T21:08:36.5255411Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-composer-0.259.0/lib/dependabot/composer/update_checker.rb:24:in `latest_version'
2024-08-23T21:08:36.5256090Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-common-0.259.0/lib/dependabot/update_checkers/base.rb:314:in `numeric_version_up_to_date?'
2024-08-23T21:08:36.5256795Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11406/lib/types/private/methods/call_validation.rb:270:in `bind_call'
2024-08-23T21:08:36.5257463Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11406/lib/types/private/methods/call_validation.rb:270:in `validate_call'
2024-08-23T21:08:36.5258139Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11406/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
2024-08-23T21:08:36.5258741Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-common-0.259.0/lib/dependabot/update_checkers/base.rb:267:in `version_up_to_date?'
2024-08-23T21:08:36.5259484Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11406/lib/types/private/methods/call_validation.rb:270:in `bind_call'
2024-08-23T21:08:36.5260148Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11406/lib/types/private/methods/call_validation.rb:270:in `validate_call'
2024-08-23T21:08:36.5261066Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11406/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
2024-08-23T21:08:36.5261782Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/dependabot-common-0.259.0/lib/dependabot/update_checkers/base.rb:82:in `up_to_date?'
2024-08-23T21:08:36.5262425Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11406/lib/types/private/methods/call_validation.rb:270:in `bind_call'
2024-08-23T21:08:36.5263076Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11406/lib/types/private/methods/call_validation.rb:270:in `validate_call'
2024-08-23T21:08:36.5263793Z    from /home/dependabot/dependabot-updater/vendor/ruby/3.3.0/gems/sorbet-runtime-0.5.11406/lib/types/private/methods/_methods.rb:277:in `block in _on_method_added'
2024-08-23T21:08:36.5264188Z    from bin/update_script.rb:585:in `block in <main>'
2024-08-23T21:08:36.5264484Z    from bin/update_script.rb:548:in `each'
2024-08-23T21:08:36.5264696Z    from bin/update_script.rb:548:in `<main>'
2024-08-23T21:08:37.1071133Z ##[error]The process '/usr/bin/docker' failed with exit code 1
2024-08-23T21:08:37.1115444Z ##[section]Finishing: dependabot

Please also note the difference between the generated DEPENDABOT_EXTRA_CREDENTIALS and the one I previously used:

+ {"type":"composer_repository","username":"*****@*****.nl","password":"***","url":"https://repo.*****.nl"}
- {"type":"composer_repository","host":"repo.****.nl","registry":"repo.*****.nl","username":"****@*****.nl","password":"$(REPO_API_KEY)"}

Extension:

Installed version 1.31.0.826 (Latest) (Also tried with dockerImageTag 1.29 just to rule out the latest changes).

mburumaxwell commented 2 months ago

Looking at this, it seems the url is present where you would have instead set the host. Maybe we need to add a host parameter when processing the registries of type composer-repository. In the meantime, could you try adding the host parameter in your config file?

version: 2

registries:
  composer:
    type: composer-repository
    url: 'https://repo.*****.nl'
    host: 'repo.*****.nl' # <-- this here
    username: '*****@*****.nl'
    password: ${{REPO_API_KEY}}

updates:
  - package-ecosystem: "composer"
    directory: "/"
    target-branch: "main"
    registries:
      - composer
    schedule:
      interval: "daily"

Report back if this works so that we can bake it in.