search

asyncapi / community

AsyncAPI community-related stuff.
https://www.asyncapi.com/community
96 stars 101 forks source link

[FEATURE] Periodically validate all CODEOWNERS files under the AsyncAPI organization #1322

Open mszostok opened 1 month ago

mszostok commented 1 month ago

Why do we need this improvement?

While working on https://github.com/asyncapi/community/issues/1269, I noticed such problems related to the CODEOWNERS files under the AsyncAPI organization:

  1. Invalid CODEOWNERS filename (fixed by this pull request).
  2. Invalid user in CODEOWNERS
    • user changed their profile name and while this was updated here, it is still missing from other places.
    • missing write access e.g. damaru-inc useer

Issues report generate on 24.07.2024

Found 11 invalid CODEOWNERS files.

Click to see a detailed report

```md # Found 11 invalid CODEOWNERS files ## python-paho-template https://github.com/asyncapi/python-paho-template Unknown owner on line 8: make sure @damaru-inc exists and has write access to the repository * @damaru-inc @CameronRushton @asyncapi-bot-eve ^ ## .github https://github.com/asyncapi/.github Unknown owner on line 9: make sure @alequetzalli exists and has write access to the repository * @derberg @alequetzalli @KhudaDad414 @asyncapi-bot-eve ^ ## brand https://github.com/asyncapi/brand Unknown owner on line 8: make sure @mcturco exists and has write access to the repository * @fmvilas @mcturco @asyncapi-bot-eve ^ ## training https://github.com/asyncapi/training Unknown owner on line 8: make sure @alequetzalli exists and has write access to the repository * @alequetzalli @asyncapi-bot-eve ^ ## java-template https://github.com/asyncapi/java-template Unknown owner on line 9: make sure @JEFFLUFC exists and has write access to the repository * @AGurlhosur @dalelane @dan-r @KieranM1999 @lewis-relph @JEFFLUFC @asyncapi-bot-eve ^ ## learning-paths https://github.com/asyncapi/learning-paths Unknown owner on line 8: make sure @Barbanio exists and has write access to the repository @alequetzalli @asyncapi-bot-eve @Barbanio ^ ## spec-json-schemas https://github.com/asyncapi/spec-json-schemas Unknown owner on line 13: make sure @SrfHead exists and has write access to the repository /bindings/jms/ @rcoppen @SrfHead ^ Unknown owner on line 16: make sure @damaru-inc exists and has write access to the repository /bindings/solace/ @damaru-inc @CameronRushton ^ ## bindings https://github.com/asyncapi/bindings Unknown owner on line 16: make sure @damaru-inc exists and has write access to the repository /solace/ @damaru-inc @CameronRushton ^ ## raml-dt-schema-parser https://github.com/asyncapi/raml-dt-schema-parser Unknown owner on line 9: make sure @jstoiko exists and has write access to the repository * @fmvilas @jstoiko @smoya @asyncapi-bot-eve ^ ## java-spring-cloud-stream-template https://github.com/asyncapi/java-spring-cloud-stream-template Unknown owner on line 8: make sure @damaru-inc exists and has write access to the repository * @damaru-inc @CameronRushton @asyncapi-bot-eve ^ ## community https://github.com/asyncapi/community Unknown owner on line 8: make sure @alequetzalli exists and has write access to the repository * @alequetzalli @derberg @asyncapi-bot-eve @thulieblack ^ ```

[!TIP] To get a fresh report run:

gh extension install github.com/gitangle/gh-codeowners
gh codeowners validate  --owner asyncapi --all --ignore-repos shape-up-process

How will this change help?

Without that change, the MAINTAINERS.yaml file may still reflect an incorrect state, as the related CODEOWNERS file could be outdated, invalid, or even missing.

How could it be implemented/designed?

Start Minimal

  1. Create a single GitHub Action that runs weekly:
    • Checks for issues across all CODEOWNERS files using a query like this: 
      query ($owner: String!) {
 organization(login: $owner) {
   repositories(first: 100, visibility: PUBLIC) {
     nodes {
       name
       url
       codeowners {
         errors {
           path
           source
           kind
           line
           message
           suggestion
         }
       }
     }
   }
 }
}

      Alternatively, you can use this tool: gh-codeowners.

    • Detects if a repository does not have a CODEOWNERS file, eliminating issues like this: github-action-for-cli/pull/397.
  2. If there is an issue, post a Slack message with a link to the GitHub job and a human-readable report of the issue.

Improvements for Later

  1. Consider using more advanced validators to check for:

    • Lines with file patterns that do not exist in the repository.
    • Duplicate lines with the same file pattern.
    • Invalid syntax definitions. This is important because:

      If any line in your CODEOWNERS file contains invalid syntax, the file will not be detected and will not be used to request reviews. Invalid syntax includes inline comments and user or team names that do not exist on GitHub."

  2. Consider auto-fixing CODEOWNERS files. For example, when a user is removed from the organization, update the related CODEOWNERS files to reflect that change.

šŸš§ Breaking changes

No

šŸ‘€ Have you checked for similar open issues?

šŸ¢ Have you read the Contributing Guidelines?

Are you willing to work on this issue?

Yes I am willing to submit a PR!

github-actions[bot] commented 1 month ago

Welcome to AsyncAPI. Thanks a lot for reporting your first issue. Please check out our contributors guide and the instructions about a basic recommended setup useful for opening a pull request.
Keep in mind there are also other channels you can use to interact with AsyncAPI community. For more details check out this issue.

derberg commented 1 month ago

thanks for the issue

the problem is that even if we get a report that there are errors, we need to find someone who has time to fix it.

wouldn't it be better to have a workflow in each repo that is dedicated to codeowners validation? and it is triggered only if codeowners file is edited. This way we block errors on PR level which I think is best

mszostok commented 1 month ago

wouldn't it be better to have a workflow in each repo that is dedicated to codeowners validation? and it is triggered only if codeowners file is edited. This way we block errors on PR level which I think is best

In PRs, this is less useful. You create a PR with a change that already makes sense, and a reviewer checks it too. An extra layer of protection is nice but not essential.

The real problem is with CODEOWNERS files that remain on the main branch without any changes. They become easily outdated. A periodic check can help in this situation.

For example:

If you want to do this only via PR checks, you will need to wait for a PR to be created to trigger a workflow that checks the CODEOWNERS file. First, such PRs are not that frequent. Second, if a contributor simply wants to add someone as a new codeowner, they also need to deal with issues unrelated to their changes.

(..) we need to find someone who has time to fix it.

Maybe the first version should already come with a simple solution to fix related issues and be executed only once per month. It could be via Slack or CLI in terminal, sth like:

https://github.com/user-attachments/assets/51cc6aab-e16b-4021-9084-483e9235dd5f

I just think that it's hard to automate it and a human is required in the flow to approve or adjust fixes and decide whether to:

On the other hand, outdated CODEOWNERS should be rare, so the question is whether it makes sense to simplify it so much šŸ¤”

derberg commented 1 month ago

ok, so my assumption was your rather suggesting validation of codeowners from technical point of view, if data there is accurate, etc

but you mean also issues when someone changed name or someone was not invite or did not accept invite to the repo

yeah, some monthly report could be useful