GitHub Actions: workflows with optional jobs still marked as :x:

[klauer]

Seen in https://github.com/pcdshub/pcdsdevices/pull/1102, the ❌ in the actions status is the subject of a continuously-requested feature over multiple years that GitHub apparently does not care to really investigate - so it seems to me. There's a lot of frustrated commenters in the following links:

https://github.com/actions/runner/issues/2347 https://github.com/orgs/community/discussions/15452

Any "solution" requires working around the system. We could:

1. Live with seeing ❌ for optional checks (and having to click to see what passed on a per-commit basis)
2. Remove our experimental tests from the workflow entirely (not a great choice)
3. Investigate a solution like this that replaces status information for "continue-on-error" jobs with a bot comment: https://github.com/mainmatter/continue-on-error-comment
4. Keep Travis CI until GitHub Actions comes around on the issue (... no)

Though all undesirable, I'd vote for (1) until we come up with a better solution.

[tangkong]

Comforting to see I wasn't the only one annoyed by this. I think out of the options available, I'd vote for (1). As long as we enable the right branch protection settings, I think it's clear enough that our experimental tests can fail. (not as clear as it could be, but I can come to terms with it). My primary concern is that it feels a bit less intuitive for anyone not familiar with the CI stack, and might make it harder for others to contribute.

2 and 4 aside, I'm not entirely clear how the linked bot-helper would help us. It seems like others get their experimental failures hidden behind a ✅ , while all ours clearly show as ❌ ? (Though the point of finding a workaround is well taken)

Either way, I'm not super optimistic about github changing this, seeing as the closed issue was opened in 2020...