v1.11.1 422 Error

[chaseconey]

Describe the bug

It seems that the v1.11.1 version introduced a validation error being returned from the GitHub API at the very end of the process.

The error message:

Checking org/**** for compatible package managers
Traceback (most recent call last):
  File "/action/workspace/evergreen.py", line 427, in <module>
    main()  # pragma: no cover
    ^^^^^^
  File "/action/workspace/evergreen.py", line 162, in main
    pull = commit_changes(
           ^^^^^^^^^^^^^^^
  File "/action/workspace/evergreen.py", line 321, in commit_changes
    pull = repo.create_pull(
           ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/github3/decorators.py", line 24, in auth_wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/github3/repos/repo.py", line 1185, in create_pull
    return self._create_pull(data)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/github3/repos/repo.py", line 117, in _create_pull
    json = self._json(self._post(url, data=data), 201)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/github3/models.py", line 161, in _json
    raise exceptions.error_for(response)
github3.exceptions.UnprocessableEntity: 422 Validation Failed

To Reproduce

1. Pull latest
2. Run as normal with the config below

Expected behavior

Expect a successful processing of repos created in the last week.

Screenshots

No response

Additional context

Configuration

---
name: Weekly dependabot checks
on:
  workflow_dispatch:
    inputs:
      batch_size:
        description: "Number of repositories to process in a single run"
        required: false
        default: 5
        type: number
  schedule:
    # Runs twice per week, Tuesdays and Saturdays at 2:00 AM UTC
    - cron: "0 2 * * 2,6"

jobs:
  evergreen:
    name: evergreen
    runs-on: ubuntu-latest

    steps:
      - shell: bash
        run: |
          # Get the current date
          current_date=$(date +'%Y-%m-%d')

          # Calculate the previous month
          previous_date=$(date -d "$current_date -7 day" +'%Y-%m-%d')

          echo "$previous_date..$current_date"
          echo "one_week_ago=$previous_date" >> "$GITHUB_ENV"

      - name: Run evergreen action
        uses: github/evergreen@v1
        env:
          GH_APP_ID: "<id>"
          GH_APP_INSTALLATION_ID: "<id>"
          GH_APP_PRIVATE_KEY: ${{ secrets.GH_APP_PRIVATE_KEY }}
          ORGANIZATION: <org>
          GROUP_DEPENDENCIES: true
          PROJECT_ID: 9
          BATCH_SIZE: ${{ inputs.batch_size || 50 }}
          CREATED_AFTER_DATE: ${{ env.one_week_ago }}

[chaseconey]

Still seem to be experiencing this. Any other details needed to take a look? 😄

[cmmata]

I had the same error. Seeing this issue, I tried with previous versions (up to v1.9.2), and they are all not working also.

Could it be a change on the API? I doubt that this error was happening since earlier than that version.

[gbrindisi]

Same issue here, running on older evergreen@v1.8.0

[gbrindisi]

@zkoppert :wave: do you have any pointers to solve this?

[zkoppert]

Hey all! Sorry I'm late to the party here. A few clarifying questions:

• Is this happening for folks repeatably on the first time they try to create a pull request through the evergreen action?
• Is it always the same repo it does this on or does it move around?
• Can we confirm that the user/token has rate limit enough to make this request?
• Maybe we could run the evergreen code locally with VSCode with a run and debug to try to get more information on the server response?

I looked up the docs to check out why we could be receiving a 422 on this endpoint. Looks like its either a malformed request or the API thinks we are spamming the endpoint. Either of those seem logical in your cases?

[martincostello]

I get this error for every org I iterate through in a scheduled job running in GHES where a GitHub app is used for authentication.

I don't think it's rate limiting, my hunch is it's trying to create a PR even when there's no changes (as all the orgs I'm targeting are all onboarded so should be a no-op).

[gbrindisi]

I found the issue here https://github.com/github/evergreen/blob/ccd4146449acd1ae707b829e2500efda56472027/evergreen.py#L84C1-L89C22

        dependabot_filename_to_use = None
        for filename in filename_list:
            existing_config = check_existing_config(repo, filename, update_existing)
            if existing_config:
                dependabot_filename_to_use = filename
                break

If there isn't a dependabot config then dependabot_filename_to_use will stay None.

Down the line we explicitly pass it to commit_change that will pass it to repo.create_file:

def commit_changes(
    title,
    body,
    repo,
    dependabot_file,
    message,
    dependabot_filename=".github/dependabot.yml",
    existing_config=None,
):
    """Commit the changes to the repo and open a pull reques and return the pull request object"""
    default_branch = repo.default_branch
    # Get latest commit sha from default branch
    default_branch_commit = repo.ref("heads/" + default_branch).object.sha
    front_matter = "refs/heads/"
    branch_name = "dependabot-" + str(uuid.uuid4())
    repo.create_ref(front_matter + branch_name, default_branch_commit)
    if existing_config:
        repo.file_contents(dependabot_filename).update(
            message=message,
            content=dependabot_file.encode(),  # Convert to bytes object
            branch=branch_name,
        )
    else:
        repo.create_file(
            path=dependabot_filename, # This is now None
            message=message,
            content=dependabot_file.encode(),  # Convert to bytes object
            branch=branch_name,
        )

But create_file requires path, message and content to not be None, otherwise it will silently return None itself without error:

json = None
if path and message and content:
    ...
return json

So our create_file returns None and we just proceed to create a pull request without a commit.

This is when the API barks and throws a 422 because in fact nothing was committed:

2024-07-17 11:33:37,881 DEBUG POST https://api.github.com/repos/XXX/YYY/pulls with {"title": "Add dependabot configuration", "body": "Please add this dependabot configuration ", "base": "main", "head": "dependabot-7e58fa2f-3fea-4ad6-b194-73d64bfbd828"}, {}
{'message': 'Validation Failed', 'errors': [{'resource': 'PullRequest', 'code': 'custom', 'message': 'No commits between main and dependabot-7e58fa2f-3fea-4ad6-b194-73d64bfbd828'}], 'documentation_url': 'https://docs.github.com/rest/pulls/pulls#create-a-pull-request', 'status': '422'}

As a quick fix I added this and everything works again:

        filename_list = [".github/dependabot.yml", ".github/dependabot.yaml"]
        dependabot_filename_to_use = None
        for filename in filename_list:
            existing_config = check_existing_config(repo, filename, update_existing)
            if existing_config:
                dependabot_filename_to_use = filename
                break
            else:
                dependabot_filename_to_use = ".github/dependabot.yaml"

@zkoppert I am not sure if this is the most elegant solution to fix it, would you mind taking a look?

[zkoppert]

That looks great! Want to put up a PR with that in it or would you like me to do that? Either way, I think we can get this through.