Closed Simon-Hayden-iteratec closed 3 months ago
Because I assume this to be a race condition issue, it is also hard to reproduce
it is, and my best guess is that is that the problem is the following happens (according to the logs, and my best guesses):
this is at least my best guess from the logs, currently the checks are:
i guess i need to add a check to see if the download lock still exists before trying to use the binary
Would be amazing if this is the issue and you could fix it :smile:
Though I wonder, why we are the only once reporting on this, though. As this race condition you're describing could have happened for anybody, there should have been at least some report already?
Anyway, thanks a lot for your fast reply :)
Though I wonder, why we are the only once reporting on this, though. As this race condition you're describing could have happened for anybody, there should have been at least some report already?
i guess the extraction part is quite quick and so is basically "atomic" on any other system (the archive is about ~70mb and the extracted binary around ~130mb to ~180mb).
and mostly the instances are run on the same process instead of separate processes (like globalSetup
instead of in each setupFile
/ beforeAll
; thanks to how jest works), because if they are in the same process they would likely wait (dont know the specifics for that at the moment)
put in a fix in c60060977e91cd2c235d86d5ac27d194a7a9e3e3, which will be available as 9.2.1-beta.1
, please test it for a while once available and report back (there will be a bot message)
:tada: This issue has been resolved in version 9.2.1-beta.1 :tada:
The release is available on:
Your semantic-release bot :package::rocket:
I'll give it a spin and see if we get any errors
I've ran the pipeline now five times with the beta version and it didn't fail. But I also can't find the log message "binary found, but also a download-lock, trying to resolve lock" anywhere, which most likely means that I didn't trigger the race condition this time.
I'm not sure if something else is currently preventing the issue (e.g. lower load on the runner?) but I will keep an eye on this. If there are more failed runs I will let you know. But maybe in a new ticket, so this one can be closed.
Thanks for your effort and quick fix implementation :1st_place_medal:
i will give it a few more days open as there has not been the message you were searching for, at the latest it will be closed once the stale message comes up (when i forgot to close beforehand)
So our test pipeline hasn't been failing the last few days. After looking at the logs I also found at least one, which hat the log line binary found, but also a download-lock, trying to resolve lock
in it. To me that sounds like the race condition is fixed :tada:
I'll close the bug report now. Thanks again for your super fast help :)
:tada: This issue has been resolved in version 9.3.0 :tada:
The release is available on:
Your semantic-release bot :package::rocket:
Versions
actions/setup-node@v3
withnode-version: 20
)package: mongo-memory-server
What is the Problem?
We have a project with two integration tests using the mongodb-memory-server with a replica set because we are using Change Streams. When we run the tests on the CI, we sometimes get the this error (full logs below):
After some googling, the
errno
ETXTBSY
is supposed to mean "Text file busy". So I assume there is some race condition, which disallowsmongod
to be called/spawned twice in quick succession. However, I did not find any information tomongod
+ETXTBSY
.Because I assume this to be a race condition issue, it is also hard to reproduce. Locally, this error has not yet happened on any dev machine. On the CI, however, I'd say this error happens around 1/3 to 2/3 of runs. My assumption is that the CI cores are just slower and thus give a larger time window for the race condition to happen? Or maybe there is some weird OS issue with the runner, but I couldn't find anything providing an OS fix for this error either.
As a workaround, we are running tests with the
--runInBand
flag to instruct Jest to run one test after another. This is okay-ish for us, because we are using Nx to manage the monorepo, so we still parallelize based on Nx projects. However, if we have a second project which also needs to run MongoDB integration tests, I'd assume that the error would return.Code Example
Debug Output
The logs are quite verbose:
https://gist.github.com/Simon-Hayden-iteratec/93a2a822a8a081d987b4b427061adac4
Something I noticed is that it looks like there are multiple downloads of the
mongod
binary. Maybe the issue is also caused by process A still downloading the binary while process B tries to start mongod?Do you know why it happenes?
Sadly no