Support annotation messages for GitHub Actions

[G-Rath]

GitHub Actions supports "annotations", which is a poorly documented but really cool feature whereby CI output can tell GH to show a message on a specific line in a specific file at a particular level (error, warning, or notice).

It would be cool if chusaku supported this, to tell us exactly which controller methods are missing their annotations.

https://docs.github.com/en/actions/learn-github-actions/workflow-commands-for-github-actions#setting-an-error-message

[nshki]

Thanks for raising this! Definitely an interesting thing to explore. To try and understand context a bit more, can you lay out some example scenarios where you'd use something like this?

Reading through the documentation a bit, it seems like these GitHub Actions messages will only show up in the workflow console. There's definitely some value in that. What if there were a custom GitHub Action that runs Chusaku and gives developers the choice to output results in the console or inline as pull request review comments?

[G-Rath]

Annotations are shown in the PR UI, not just the console:

The general scenario that I have in mind is that it would be useful in PRs for both author and reviewers to see which controller methods are affected by a change - currently you can see that by the diff due to the change in annotations, but if the annotating hasn't been done this would let you see which controller methods should have been annotated but weren't, allowing reviewers to get the information early (i.e. because they wouldn't have to wait for someone to run chusaku & push up the results).

I also just realised I've linked to the wrong thing - while you can add annotations using workflow commands (e.g. via puts), you can also tell GHA how to interpret output from tools to annotate PRs, using problem matchers which would be the better thing to use in this situation.

The requirement from chausaku for this to work is that there is an actual output that has the required information - taking the example from the above link:

badFile.js: line 50, col 11, Error - 'myVar' is defined but never used. (no-unused-vars)

If chausaku would output something like that about the annotations, e.g.:

`app/controllers/sessions_controller.rb: line x, col y, Method should be annotated with ...

We could write a problem matcher to show that as an annotation in PRs, and that could be maintained in this repo for folks to copy-paste into their projects (since they need to be in the .github folder, so I don't think it needs to be included in the published gem).

[nshki]

Ah got it, this is really helpful, thank you. I wasn't aware of problem matchers until now.

So it seems like the change that would best enable this sort of usage would be to change the gem's output to include something like:

example_controller.rb:48 [+]  # @route GET /waterlilies/:id (waterlily)
another_controller.rb:21 [-]  # @route gibberish

Which essentially is a git diff. I'm wondering if there's a clever way to use Git here to achieve that kind of output. Manually storing and outputting line changes that the gem would make should be manageable as well though.

In any case, having this kind of detailed output rather than the current list of changed files after running the gem would definitely be a value add. And as you mentioned, an example set of config that can be copy-pasted in people's GitHub repos to annotate PRs could be maintained in this project's documentation.

Does this seem on track to you?