SuffolkLITLab / ALKiln

Integrated automated end-to-end testing with docassemble, puppeteer, and cucumber.
https://assemblyline.suffolklitlab.org/docs/alkiln/intro
MIT License
14 stars 4 forks source link

Add maximum time for full test run #876

Open plocket opened 2 months ago

plocket commented 2 months ago

What should the default max time be? When we get to this issue, we might ask around and find out how long folks would want as a default.

When tests start to fail they can take 2 or 3 times the amount of time. When an interview has an infinite loop, it'll run for many hours, which might or might not be a problem for private repositories and their limits (we'd need to check on what specifically is limited).

Now that we have a solid debug log, we can add an env var that cuts off tests early. We could even generate a final report for them, when all goes well, with the information we have so far, since we have control.

We might be able to put this in BeforeAll(), Before(), and BeforStep().