Audit: Ben Vinegar Codecov integration walkthrough (+ Spencer)

[codecovdesign]

Audit review

Looking at steps to integrate Codecov to a project. Auditor notes:

• i got momentarily confused that the "Ask for a Demo" CTA meant i couldnt just sign up (02:30)
• i got stuck a bit when i couldn't find my repo in the list of projects (04:38)
• i had trouble figuring out where to put the GitHub Actions .yaml command (08:45)
• the instructions assumed i was generating coverage reports, which i was not (i had to figure this out by inspecting the build output) (16:31)

https://youtu.be/LX3GZFliq8o

@spalmurray-codecov write up

Next steps

• Review video and team feedback
  • @codecovdesign review, notes, and issue follow ups
• Create issues documenting cited problems
• Proposals for issues fixes

### Tasks
- [ ] https://github.com/codecov/engineering-team/issues/1050
- [ ] https://github.com/codecov/engineering-team/issues/1051
- [ ] https://github.com/codecov/engineering-team/issues/1053
- [ ] https://github.com/codecov/engineering-team/issues/1054
- [ ] https://github.com/codecov/engineering-team/issues/1056
- [ ] https://github.com/codecov/engineering-team/issues/1057
- [ ] https://github.com/codecov/engineering-team/issues/1104
- [ ] https://github.com/codecov/engineering-team/issues/1105
- [ ] https://github.com/codecov/engineering-team/issues/1107
- [ ] https://github.com/codecov/engineering-team/issues/1124
- [ ] https://github.com/codecov/engineering-team/issues/1126
- [ ] https://github.com/codecov/engineering-team/issues/235
- [ ] https://github.com/codecov/engineering-team/issues/1226
- [ ] https://github.com/codecov/engineering-team/issues/1227
- [ ] https://github.com/codecov/engineering-team/issues/1334

[codecovdesign]

@codecovdesign review:

1. 4:20 - 4:40 auditor notices distracting "syncing" (ISSUE: https://github.com/codecov/engineering-team/issues/1050)

   • 

2. 5:00 interprets "active" and "inactive" as sorting vs configuration status (https://github.com/codecov/engineering-team/issues/1051)

3. 5:20 is unable to find repository, guesses that it's because it's private but notices a private repo. attempts to re-sync and notes it takes while and doesn't do anything. (https://github.com/codecov/engineering-team/issues/1104)

   • after changing to public in GH it displayed

4. 7:30 notes that name and secret may be helpful to copy/paste separately

5. 8:20 was unsure exactly where to place codecov yaml input (https://github.com/codecov/engineering-team/issues/1105)

6. 9:47 unsure about next steps to see status checks related to branch, decides to do on main (💡ToDo issue copy update to clarify?)

7. 10:50 notified in action of error in yaml syntax (ToDo issue, add VS code yaml editor plugin? related to 5? )

8. 12:02 after step 2 notices note about PR and changes; unsure about creating PR (👀ToDo issue, make more explicit in step 2 PR creation? consider in this case committed directly to main)

9. at 14:00 struggling in PR as it's not running correctly

   • notes gap in GH actions config
   • uses sentry codecov as reference (💡todo add example repo in guide?)

10. 16:15 reads once merged you'll find repo reports here (💡ToDo create issue to refresh page and/or test existing check to auto reload if first commit?)

11. at 17:00 realized need to generate coverage files, frustration that it didn't relay that initially int he guide (💡ToDo create issue to show prerequisite, note previously was shown reuse and/or look at remove reason for any context)

12. 20:30 after attempting coverage upload, try another provider and has challenges. was able to generate report.

13. 23:00 looking at latest commit that reloads page (shows no coverage on the default branch main page)

14. 25:10 then updates to default branch to view report:

15. 25:31 final reflection, participant outlines it took 25 minutes get it going but hit a few snags/wall along the way, that felt can be addressed, notes it wasnt too hard to figure out but curious about others.

@codecovdesign follow up with noted issues^, design updates, and review with team/participant.

[codecovdesign]

closing epic as all issues have been addressed