Closed Tolsto closed 4 years ago
I don’t totally follow the failure mode. Are calls to “rand” failing for your app on Heroku CI but only with this gem?
Do you get a similar randomly failing error by adding this gem to a Rails new app and running on Heroku CI or is there something specific that triggers this behavior?
On Mon, Aug 26, 2019 at 4:08 AM Nils Mueller notifications@github.com wrote:
The Readme currently suggest to just include the gem in the Gemfile. However, it's a bad idea to include it in the test environment. When running on the Heroku CI it will start a thread that makes calls to Kernel.rand(). Those calls will mess up the test seed and test failures on the CI will not be reliably reproducible locally anymore. It took us a long time to debug this issue and it would be good if the Readme would state this.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/heroku/barnes/issues/24?email_source=notifications&email_token=AAAOSYB5O67IX4TMZGU25ULQGOMSHA5CNFSM4IPNRQQKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HHKQWOA, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAOSYAUUOUCNAYTZDIFUU3QGOMSHANCNFSM4IPNRQQA .
-- Richard Schneeman https://www.schneems.com
The Gem does not produce any errors on the CI. However, it starts a thread and makes calls to rand() from within that thread. This means that a test that runs in the main thread after one of those rand() calls and also makes a rand() call will receive a different random number than it would have received if barnes wouldn't run. However, because of the thread scheduler it's not predictable when exactly the thread will get resources to make the call to rand(). The result is that the same test in the test suite might get different random numbers for each suite run, even when using the same RNG seed in Rspec.
The Readme currently suggest to just include the gem in the Gemfile. However, it's a bad idea to include it in the test environment. When running on the Heroku CI it will start a thread that makes calls to
Kernel.rand()
. Those calls will mess up the test seed and test failures on the CI will not be reliably reproducible locally anymore. It took us a long time to debug this issue and it would be good if the Readme would state this.