search

dequelabs / axe-devhub-action

Mozilla Public License 2.0
0 stars 2 forks source link

EPIC: gha comparison logic #115

Open dequejenn opened 11 months ago

dequejenn commented 11 months ago

When the GHA is ran, it is not taking into account the comparisons like it should. The below tickets will correct this logic error.

~TODO - the above logic needs to be changed a bit to reflect comparing to the common ancestor commit of the base branch, NOT the default branch. In order to do this, we will need to be grabbing the base branch from the git info and saving that as well as default branch info per commit. These tickets are on hold until this is thought out a bit further and the corresponding tickets are modified**~ EDIT: we are only comparing to the last scan of the default branch. We are trying to answer the question "how does my commit affect the current accessibility of the default branch?" so comparing to the last commit is the correct way to answer thing. No need to find a common ancenstor

### Tasks
- [ ] https://github.com/dequelabs/jazzband/issues/780
- [ ] https://github.com/dequelabs/walnut/issues/7236
- [ ] https://github.com/dequelabs/axe-devhub-action/issues/134
- [ ] https://github.com/dequelabs/jazzband/issues/1218
- [ ] https://github.com/dequelabs/jazzband/issues/1219
dequejenn commented 9 months ago

This was overkill and may be done later. However, for now, we just fixed the link to always show to commit to commit comparison so moving this to inactive epic for the moment

ballendq commented 2 months ago

@dequejenn still a bit confused by this ticket. I think we need to do some editing. My understanding is that:

  1. The problem to be solved: make the GHA comparison meaningful. In other words, compare the accessibility of the latest commit in the pull request with that of the default branch in order to answer the question "does this PR make my UI more or less accessible?"
  2. This only applies to the GHA, not other aspects of the UI/UX.
  3. Solution concerns I have - how do we get the accessibility data associated with the "main branch" if we haven't run DevHub against a commit within that branch? I'm still a bit confused on this point, and I'm not sure what I'm missing.
dequejenn commented 2 months ago

@ballendq

  1. Yes
  2. Yes
  3. Look at the details of this ticket. This is all of the use cases with behavior for each verified by Dylan in this thread