tinglesoftware / dependabot-azure-devops

Tools for updating dependencies in Azure DevOps repositories using https://dependabot.com
MIT License
195 stars 65 forks source link

Subdirectories Search Doesn't Work #1328

Closed caglanurcetinn closed 2 weeks ago

caglanurcetinn commented 1 month ago

Describe the bug I have installed Dependabot extension in AzureDevops. I have created this config;

.github/dependabot.yml

version: 2
updates:
  - package-ecosystem: "nuget"
    directories: 
      - '/templates/Producer/*'
    schedule:
      interval: "daily"
    target-branch: "main"
    commit-message:
      prefix: "test"
    open-pull-requests-limit: 2

I have used many alternatives like these ones as well.

`directories: ['/templates/Producer/*']

directories: ['/', '/templates/Producer/XXFacades.*']

directories: "/templates/Producer/XXFacades.*"

directories: "*/"

directory: "*/"

`

This is my project path:

. ├── .github/ │ └──dependabot.yml ├── templates/ │ ├── Producer/ │ │ ├── XXFacades.{name}.Producer/ │ │ │ └── XXFacades.{name}.Producer.Application/ │ │ │ │ └──XXFacades.{name}.Producer.Application.csproj │ │ │ └── XXFacades.{name}.Producer.UnitTests/ │ │ │ │ └──XXFacades.{name}.Producer.UnitTests.csproj ├── TemplatePack.csproj

Whatever I tried, it only sees TemplatePack.csproj.

However, I need to run it for .csproj files in subdirectories.

I run it on my Azure Pipeline yaml;

stages:
  - stage: CheckDependencies
    jobs:
      - job: Dependabot
        displayName: 'Run Dependabot'
        steps:
          - task: dependabot@1
            displayName: 'Run Dependabot'

Additional context This is log from my pipeline;

Working in aa/eip/_git/dot-net-project-template, 'main' branch under '/' directory Cloning repository into /home/dependabot/dependabot-updater/tmp/aa/eip/_git/dot-net-project-template Found 1 dependency file(s) at commit 111111

To sum up, it only scans for '/' directory. I would like to learn how to run it for sub-directories.

Thanks in advance for help !

rhyskoedijk commented 1 month ago

@caglanurcetinn support for the "directories" config option was added in https://github.com/tinglesoftware/dependabot-azure-devops/pull/1186, but only works if you opt-in to the "vNext" update script. You can enable multi-directory support by setting useUpdateScriptvNext: true in your task input.

- task: dependabot@1
  displayName: 'Run Dependabot'
  inputs:
    useUpdateScriptvNext: true

Glob patterns in directory paths are supported starting from v1.31.0.

I haven't extensively tested multi-directories and glob patterns together, so let me know if this still causes issues after you've tried the above.

caglanurcetinn commented 1 month ago

Thank you so much! It worked. But now, It has been working non-stop for 20 minutes;

2024/09/06 06:29:25 INFO Starting job processing 2024/09/06 06:29:25 INFO Cloning repository 'https://dev.azure.com/aa/eip/_git/dot-net-project-template' to '/home/dependabot/dependabot-updater/job/1725604162/repo/aa/eip/_git/dot-net-project-template' running NuGet discovery: /opt/nuget/NuGetUpdater/NuGetUpdater.Cli discover --repo-root /home/dependabot/dependabot-updater/job/1725604162/repo/aa/eip/_git/dot-net-project-template --workspace /templates/Producer/XXFacades.{facadeName}.Producer --output /tmp/.dependabot/discovery.1.json --verbose

And then it gets below error;

2024/09/06 07:02:20 ERROR The following source could not be reached as it requires authentication (and any provided details were invalid or lacked the required permissions): (https://api.nuget.org/v3/index.json|https://aa.pkgs.visualstudio.com/eip/_packaging/eip-feed/nuget/v3/index.json)

How can I authorize it in Azure Devops? Could you please help me?

rhyskoedijk commented 1 month ago

2024/09/06 07:02:20 ERROR The following source could not be reached as it requires authentication (and any provided details were invalid or lacked the required permissions): (https://api.nuget.org/v3/index.json|https://aa.pkgs.visualstudio.com/eip/_packaging/eip-feed/nuget/v3/index.json)

@caglanurcetinn can you share the full contents (secrets redacted) of your dependabot.yaml and any nuget.config files in your source repository?

If you haven't read it already, the documentation and notes here might help: https://github.com/tinglesoftware/dependabot-azure-devops?tab=readme-ov-file#credentials-for-private-registries-and-feeds

nicolasproton commented 3 weeks ago

The "directories" options is not working as expected in the ADO task v2. Whatever I enter in the list of directories, it always picks up the top level folder. I am using poetry as package manager, but I am facing the same issue with nuget.

updater | 2024/10/10 10:16:03 INFO <job_update_0_pip_all> Found 1 group(s).
updater | 2024/10/10 10:16:03 INFO <job_update_0_pip_all> Starting update group for 'all-dependencies-python-services'
updater | 2024/10/10 10:16:03 INFO <job_update_0_pip_all> Updating the / directory.
rhyskoedijk commented 3 weeks ago

@nicolasproton thanks for the report, this is indeed a bug in TaskV2; If there is only one entry in directories, it will incorrectly fallback to directory: '/'. This will be fixed in https://github.com/tinglesoftware/dependabot-azure-devops/pull/1393.

You might be able to work around this by specifying the directory pattern twice in the list. This will get the extension to parse the directories config correctly and pass it to Dependabot which should then do a distinct filter on the resolved paths and remove any duplicates.

  - package-ecosystem: "pip"
    directories:
      - "/services/*"
      - "/services/*"