Combining multiple snippets into single comment

[Lissy93]

Hey @angelikatyborska - Really enjoying your action :) I hadn't come across any similar actions, and this is a super useful task - awesome work!

Just a quick question, when multiple files patterns are matched, I would expect the comment to include all matching snippets between the header + footer, but instead it just includes the first one only. Am I missing something in my config maybe?

For reference, here's my config file and my action file.

Also I've noticed that the on-update option (recreate / edit /nothing) seems to do nothing. Each time whenever a new commit is pushed to the open PR, a new comment is always added.

Thank you very much, Alicia :)

[angelikatyborska]

Hi!

I would expect the comment to include all matching snippets between the header + footer, but instead it just includes the first one only.

This is exactly how it's supposed to work 🤔 if it doesn't, something's wrong.

Am I missing something in my config maybe?

At a first glance, your config looks 100% correct to me.

Also I've noticed that the on-update option (recreate / edit /nothing) seems to do nothing. Each time whenever a new commit is pushed to the open PR, a new comment is always added.

This sounds very suspicious! When the new comment is added, is the old comment removed or not? If it's removed, then it sounds like it uses the option "recreate" instead of "edit". If it doesn't get removed, then there is something broken with detecting an existing comment.

I personally use on-update: recreate in two repositories so it is possible that on-update: edit is somehow broken and I didn't notice.

I should have more time tomorrow to have a proper look at the problem. I'll take your config, put it in my test repository and see what happens.

Meanwhile, you could use this debugging option for GitHub workflows to get debug logs from the action. It does provide logs about which files are being changed in the PR and the patterns it tries to match and on which files and what's the result. For example:

##[debug]found changed files:
##[debug]  .github/workflows/test-action.yml
##[debug]  banana/2.txt

(...)

##[debug]processing snippet test_snippet_2
##[debug] checking pattern "banana/1.txt"
##[debug]  checking "any" patterns
##[debug]    matching patterns against file .github/workflows/test-action.yml
##[debug]   - banana/1.txt
##[debug]   banana/1.txt did not match
##[debug]    matching patterns against file banana/2.txt
##[debug]   - banana/1.txt
##[debug]   banana/1.txt did not match
##[debug]  "any" patterns did not match any files

[Lissy93]

Thank you so much for your reply :)

The previous comments are never removed, and a new one is added each time, and only contains the text from the first matching block. I tried setting on-update: recreate, instead of edit, but the same result.

Here's an example pull request which shows what it's doing: https://github.com/Lissy93/dashy/pull/224

So using ACTIONS_STEP_DEBUG and ACTIONS_RUNNER_DEBUG gave me the following logs from the most recent run: logs_2449.zip

And the text logs from the past few runs: Logs_3576429694.txt, Logs_3576417452.txt, Logs_3576336333.txt, Logs_3576334778.txt

Thanks again for your help :)

[angelikatyborska]

Problem 1: patterns not matching when expected to match

Possible reason 1: hidden files?

To match hidden files with a simple * wildcard, you need to pass this glob option like in the example:

https://github.com/exercism/pr-commenter-action#commentglob-options

I am not sure if you're trying to match any hidden files with *?

Possible reason 2: wrong patterns

I just started my testing with your config and I did notice a problem with some of the file patterns. Those three:

    - id: ignored-dist
      files:
      - /dist

    - id: ignored-dependencies
      files:
      - /node_modules

    - id: code-owners
      files:
      - /.github/CODEOWNERS

They all start with / and that causes them not to match. The paths of changed files returned by the GitHub API are relative to the project's root directory (but without ./). It can be seen in the debug logs that you included in the zip file:

2021-09-11T20:09:54.3092761Z ##[debug]fetching changed files for pr #224
2021-09-11T20:09:54.3104697Z ##[debug]found changed files:
2021-09-11T20:09:54.3106274Z ##[debug]  .github/CHANGELOG.md
2021-09-11T20:09:54.3107555Z ##[debug]  docs/configuring.md
2021-09-11T20:09:54.3109523Z ##[debug]  docs/searching.md
2021-09-11T20:09:54.3110827Z ##[debug]  package.json
2021-09-11T20:09:54.3112107Z ##[debug]  src/components/Settings/SearchBar.vue
2021-09-11T20:09:54.3113771Z ##[debug]  src/utils/ConfigSchema.json
2021-09-11T20:09:54.3114987Z ##[debug]  src/utils/Search.js
2021-09-11T20:09:54.3116394Z ##[debug]  src/utils/defaults.js
2021-09-11T20:09:54.3117616Z ##[debug]  src/views/Home.vue
2021-09-11T20:09:54.3118900Z ##[debug]  src/views/Minimal.vue

On top of that, the patterns would dist and node_modules match files with those exact names but nothing else. To match files within directories named like this, the patterns need to be dist/**/* and node_modules/**/* (that will cover both files directly in those directories as well as nested within other directories).

Would that be enough to explain the problem that you're only seeing a single snippet matched? In the logs in logs_2449.zip, in the list of changed files, I can only see a single file that is supposed to match a snippet (src/utils/ConfigSchema.json) so I am not sure if the cases where you wanted it to match but it didn't were all related to those patterns or not.

I noticed that while the PR https://github.com/Lissy93/dashy/pull/224 was opened, you committed this https://github.com/Lissy93/dashy/commit/a702fae146ff03b2163d4b0a47a03459c0eeeb33#diff-a699cacf48dcc3f436415ea8807ce5d72029cb2eddf1b2eff4629bebe948d4e1 to main, which removes the generic pattern for md files and adds a new pattern for the config file. This would explain to me why in the PR suddenly the comment snippet about docs is no longer in the final full comment, but the config one is. The action always reads its YAML config file from main so those changes would affect its behavior immediately. But I still don't know for sure if that explains all of the problems that you're seeing?

Logs_3576429694.txt, Logs_3576417452.txt, Logs_3576336333.txt, Logs_3576334778.txt

Unfortunately I am unable to access those files. I am getting the response {"count":11,"value":"Uri expired"}. But if those are all debug logs from workflow runs on that one PR, than I can just access them directly via GitHub (as I have done now).

Problem 2: existing comment not detected

As for the second problem of the comment always being added anew and never deleted or edited, that's more tricky. I didn't do a great job of adding debug statements for this flow. I'll need to fix that.

There could be multiple reasons.

Possible reason 1: no permissions?

I noticed that you're using a custom token (github-token: ${{ secrets.BOT_GITHUB_TOKEN }}) instead of the default value github.token. Is it possible that this token doesn't have permissions to read, edit, or delete comments for the PR? That would be rather unusual because it definitely has permissions to create comments 🤔 I don't know if those are separate. I think you could test that by manually trying to do an API call to https://docs.github.com/en/rest/reference/issues#list-issue-comments-for-a-repository for your PR using that token and see what happens.

Possible reason 2: something else is breaking the comment's metadata

The way pr-commenter detects its previous comments is by a HTML comment with metadata. You can see it if you try to edit the bot's comment:

How does that look like in your case? Is it there?

[angelikatyborska]

Oh no! I just figured out the second problem.

I added this "safety" check here: https://github.com/exercism/pr-commenter-action/blob/1e8d6b52bc0575d34cacbf3400288411e0dc0869/lib/run.js#L100

When looking for existing comments made by this action, I made an assumption that the author of the comment is going to be of type bot. Because that's what happens when you use the default Github token. But you're using a "real" user:

This was a mistake on my part. I think I should just remove this check.

[angelikatyborska]

@Lissy93 I just published v1.3.0 which should fix the second problem with the old comment never getting edited or deleted.

[Lissy93]

You hero! Thank you so so much! 🙌

I also updated the file patterns, with your recommendation and that part now works perfectly- I was just being thick- thank you for explaining :)

Will update to your new version and hopefully that will fix the recreation of my non-bot bot comments. Thank you so much for implementing that change, and for your help with explaining the first issue 💖

[angelikatyborska]

I'm glad to be of service and I'm happy somebody is using my action 🥰