search

bullet-train-co / bullet_train-core

The Open Source Ruby on Rails SaaS Framework
36 stars 44 forks source link

Does this test fail at a certain time of day? #455

Open andrewculver opened 1 year ago

andrewculver commented 1 year ago 
Failure:
SuperScaffoldingSystemTest#test_super_scaffolded_partials_function_properly [/home/circleci/project/tmp/starter/test/system/super_scaffolding_partial_test.rb:162]:
Expected: Mon, 21 Aug 2023
  Actual: Tue, 22 Aug 2023

rails test test/system/super_scaffolding_partial_test.rb:97
jagthedrummer commented 1 year ago

Somebody call the timecop!

Srsly tho, I looked at the test and the relevant steps go like this:

  1. Select today's date
  2. Submit form
  3. Assert that the date submitted == Date.today

I think this test is flaky, for potentially a couple of seconds a day, if you run it at just the right millisecond, which you seemed to do.

CleanShot 2023-08-21 at 21 30 11@2x

Not to say that we shouldn't look at fixing it, but just to say what are the chances. (And to say that it probably won't bite us that often.)

https://app.circleci.com/pipelines/github/bullet-train-co/bullet_train-core/1358/workflows/1c060905-4065-46f1-a52f-cfcc2fbfb361/jobs/4232/parallel-runs/11?filterBy=FAILED