epic-open-source / seismometer

AI model evaluation with a focus on healthcare
https://epic-open-source.github.io/seismometer/
BSD 3-Clause "New" or "Revised" License
156 stars 15 forks source link

Make label checks more dynamic #23

Closed diehlbw closed 4 weeks ago

diehlbw commented 1 month ago

Overview

Closes #22

Description of changes

Use gh api to request labels dynamically instead of using the cached list that requires an event (commit) to renew. Split changelog workflow into two jobs; could not find a method for short-circuiting success within a job.

Note: this method is intentionally requiring manual retry, and NOT triggering off of label changes

Suggested testing-
As a step 0, I'm intentionally NOT setting labels prior to PR creation.
Tests can add and remove labels on this PR, then manually rerun the specific workflow.
Verify that the second job runs and is skipped depending on the skip-changelog label.
Before merge, update labels and drop the second commit to verify the push event also behaves correctly

Author Checklist

diehlbw commented 1 month ago

Initial run 'successfully' set needs_entry.
Rerun after setting a different label, timed out on first attempt(?) then set it correctly.