nondeterministic failures of GitHub actions that rely on `Xvfb`

[mbutterick]

This isn't a bug in setup-racket but maybe you have advice.

I find that GitHub actions are inferior to TravisCI because jobs will fail for arbitrary, nondeterministic reasons with errors like nonrecoverable invalid memory reference or xvfb-run: error: Xvfb failed to start. For instance, here's the result of two runs of the same set of actions on the same commit, with different failures in each case.

The problem is that when a workflow fails, it’s impossible to know whether it's a bona fide error in my code or a random GH actions failure.

I suppose my questions are:

1. Are others experiencing these random failures?

2. Are these failures related to the stability of the GH actions platform, or related to Racket? (It’s hard for me to believe that GH actions are this unreliable; OTOH Travis always worked fine with Racket.)

[Bogdanp]

I haven't had these sorts of issues with GHA. The only ones I've encountered so far have had to do with worker availability/downtime, but even those have been fairly rare. It seems unlikely that runtime failures would specifically be caused by GHA and more likely that something about the job, the code or Racket would be causing them.

Some questions:

• Are these failures in the pollen repo or somewhere else?
• Are you seeing the memory errors with versions other than current CS (I believe that error message is specific to CS)? It may make sense to update the jobs to allow failures on current.
• Is it possible your project is affected by this issue (present in all released versions of CS)?
• Can you point me to some of the logs for these failures? I might be able to get a better idea of what's going on if I see what the jobs look like and what the output was at the time.

[mbutterick]

Thanks. I have had the problem with Pollen and Beautiful Racket (most recent log here).

But there is another clue. In Pollen, I traced the problems to a file that needed to run racket/gui while testing. It looks like that is the case for the Beautiful Racket failures too. Back when I was using Travis, I discovered that tests with racket/gui required using Xvfb, which may be the faulty ingredient here.

(This doesn’t completely resolve the mystery, since I would expect the Xvfb-related failures to happen consistently, but they do not)

Again, I don’t consider this a bug in setup-racket — I was just looking for wisdom.