department-of-veterans-affairs / vets-design-system-documentation

Repository for design.va.gov website
https://design.va.gov
36 stars 55 forks source link

Troubleshoot issues with Chromatic CI check #2785

Open it-harrison opened 3 weeks ago

it-harrison commented 3 weeks ago

Sometimes the Chromatic CI check will generate diffs based on a commit in a PR that is not the most recent; the effect of this is that diffs can be flagged incorrectly and re-running the build does not always fix this problem. When this happens a PR cannot move forward unless the diffs are accepted (when they should not be), meaning that the next PR will have to deal with accepting changes that return the baseline where it should be. This behavior also makes it hard to determine the visual effect of a PR on other components.

jamigibbs commented 2 weeks ago

Thank you for this issue @it-harrison!

Dan started a Slack thread here about nailing down a review process that we might want to do as a follow-up to this issue too.

jamigibbs commented 2 weeks ago

Something that I noticed is that branches merged into main seem to need to Chromatic changes accepted again even after they have already been accepted in the merging branch. That might be related.

Screenshot 2024-05-01 at 9 50 19 AM