GitHub Actions for some SSR specs are timing out (experimental runs are fine)

thescientist13 commented 1 year ago

Type of Change

Other

Summary

Noticing that it seems our release related GitHub Actions have been failing inconsistently consistent (not any particular OS, not necessarily experimental or not). Screen Shot 2023-02-18 at 11 45 54 AM

Details

In particular the build just running on for hours, stalled out on the task, until it is eventually shut down. Screen Shot 2023-02-18 at 12 19 40 PM

So not even really a case of our tests / build failing, just seems more of an operation issue? (hopefully?)

As of the v0.28.0 release, these offending specs have been excluded for now, but maybe this and https://github.com/ProjectEvergreen/greenwood/issues/787 maybe we need to rethink our testing strategy? Maybe one without gallinago?

A number of failed attempts are linked to this issue so definitely want these to be ironed out for the 1.0 release so adding this to the 1.0 milestone so we don't forget.

Somewhat (un)related, this happens from time to time, so maybe just more of a case to rethink our testing? - https://github.com/ProjectEvergreen/greenwood/actions/runs/4648145282/jobs/8225644571

thescientist13 commented 1 year ago

So was noticing this in the logs, maybe we just need to update our actions/setup-node version?

build (18) Node.js 12 actions are deprecated. Please update the following actions to use Node.js 16: actions/setup-node@v2. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/.

Screen Shot 2023-02-18 at 11 53 34 AM

Here is the linked blog article from GitHub re: the Node v12 deprecation https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/

thescientist13 commented 1 year ago

Eventually when the job is stopped or cancelled, it does seem to be related to a hung SSR test for static export, and only on Windows.

  1) Build Greenwood With: 
       A Server Rendered Application (SSR) that is statically exported
         "before all" hook in "A Server Rendered Application (SSR) that is statically exported":
     Error: Timeout of 90000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (D:\a\greenwood\greenwood\packages\cli\test\cases\build.default.ssr-static-export\build.default.ssr-static-export.spec.js)
      at listOnTimeout (node:internal/timers:569:17)
      at process.processTimers (node:internal/timers:512:7)

So maybe it is us?

thescientist13 commented 1 year ago

Still not sure if environment or our code, seems like the release was working fine until things started breaking, even having to merge those last two just keep the flow of work moving 🤔 🤷‍♂️ https://github.com/ProjectEvergreen/greenwood/pull/1034/commits

thescientist13 commented 1 year ago

So testing this particular spec out on my Windows VM and passed fine there once, but then running it a couple more times seemed to reproduce the issue of the process hanging? Will need to keep testing and playing around with things, but seems to be stalling consistently on the copy lifecycle. 🤔 Screen Shot 2023-02-20 at 4 28 12 PM

thescientist13 commented 1 year ago

Ok, so it actually seems it just doesn't exit the process in index.js? Added some logging and seems like all the lifecycles complete successfully. Screen Shot 2023-02-23 at 5 18 13 PM (2)

thescientist13 commented 1 year ago

Seems like we're seeing an issue now for just normal non Windows SSR specs!? 😞 😠

thescientist13 commented 8 months ago

Not sure if fixed from something before or just after #1193 , but seems like we can close this one out once that one is merged! 🎉

ProjectEvergreen / greenwood