ctrf-io / github-actions-test-reporter-ctrf

Publish and view test reporting directly in your GitHub Actions CI/CD workflow and Pull Requests with detailed test summaries, failed test analyses, and flaky test detection.
https://ctrf.io
MIT License
54 stars 4 forks source link

historical results - No runs found for the workflow: *** #28

Closed AlbertoBartolomeET closed 5 days ago

AlbertoBartolomeET commented 6 days ago

hey!

Perhaps is that I am not understanding how this command works or I am missing something. I am a bit stuck here, so it would be great if someone could help me. I have few questions.

  1. In the Actions in this repo, I don't see why in some cases it is working, and there is historical results, and in other cases there isn't: 1.2. In this run 526 run there is historical for 7 runs šŸ‘Œ 1.3. but in the 525 and 527, no historical, just results from the actual run. But in the logs there is no error or no clue.

  2. In my own repo, I was able to make it work, but only for 3 runs I was able to see the historical of those 3 runs, not anymore, not sure why or how. Now historical is always displaying the actual run, no previous ones. I spent a bit of time investigating this, but no luck fixing it. In my case I see in the logs this error "No runs found for the workflow: " 2.1. I always thought that my problems were coming from artifact/names, because even executing again same workflow that showed me historical, I have only the result of the actual run. 2.3. But seeing this error "No runs found for the workflow: ", it seems that the problem is not artifact/names. (or could be that is finding runs, but not the right artifact?)

It would be great to know a bit more about how this command works, how when searching for other runs is identifying the workflow name, how the search is done, if it has to be in the same branch...

Thanks in advance!

Ma11hewThomas commented 6 days ago

Hello, thank you for raising this issue. The historical method filters for relevant results:

For branches: It matches the workflow name and ensures the run was triggered by a push event on that specific branch. For pull requests: It matches the workflow name and ensures the run was triggered by a pull_request event with the specified pull request number.

It was decided to do it this way so that when running tests or workflows for a specific branch (e.g., main, develop, or a feature branch), you only see the history of workflow runs for that branch or PR.

However, if you see the error message "No runs found for the workflow", it means it could not find any historical workflow runs that match the same workflow name as the current one.

With this in mind, do you still experience an issue while satisfying the filtering?

Thanks

AlbertoBartolomeET commented 5 days ago

Thanks for the quick response and the explanation!

Now it is working for me, but I didn't change much from last time. I have examples of runs triggered by a pull_request event with the same pull request number with no historical, "No runs found for the workflow", but suddenly same PR is finding runs and displaying the historical šŸ¤·ā€ā™‚ļø I guess I did something wrong with those runs.

For branches: It matches the workflow name and ensures the run was triggered by a push event on that specific branch. For pull requests: It matches the workflow name and ensures the run was triggered by a pull_request event with the specified pull request number.

My idea of using this I guess it is not fitting in how it is designed. I have the CI running every night and I thought it would be useful to have the Historical of previous CI runs. But if it is working only with push_events and pull_request, it is still useful, but not exactly what I was looking for šŸ˜„

PS: it would be great if this could be documented in the Readme

Ma11hewThomas commented 5 days ago

I'm going to look into scheduled actions, as it seems like it might be a common use case.

Thanks for raising!

Ma11hewThomas commented 4 days ago

Branch filtering now includes schedule. Feedback welcome šŸ˜„