Elm crashes while initializing elm-stuff on highly contended multithreaded CPU

[zwilias]

Quick Summary:

Running elm make when elm-stuff needs to be initialized, on a very busy system with multithreading, can occasionally lead to a crash. There are a few variations to the error message:

 -- ERROR -----------------------------------------------------------------------

I ran into something that bypassed the normal error reporting process! I
extracted whatever information I could from the internal error:

>   thread blocked indefinitely in an MVar operation

These errors are usually pretty confusing, so start by asking around on one of
forums listed at https://elm-lang.org/community to see if anyone can get you
unstuck quickly.

-- REQUEST ---------------------------------------------------------------------

If you are feeling up to it, please try to get your code down to the smallest
version that still triggers this message. Ideally in a single Main.elm and
elm.json file.

From there open a NEW issue at https://github.com/elm/compiler/issues with your
reduced example pasted in directly. (Not a link to a repo or gist!) Do not worry
about if someone else saw something similar. More examples is better!

This kind of error is usually tied up in larger architectural choices that are
hard to change, so even when we have a couple good examples, it can take some
time to resolve in a solid way.elm: thread blocked indefinitely in an MVar operation

and also:

-- ERROR -----------------------------------------------------------------------

I ran into something that bypassed the normal error reporting process! I
extracted whatever information I could from the internal error:

>   /root/elm-stuff/0.19.1/d.dat: openBinaryFile: resource busy (file is locked)

These errors are usually pretty confusing, so start by asking around on one of
forums listed at https://elm-lang.org/community to see if anyone can get you
unstuck quickly.

-- REQUEST ---------------------------------------------------------------------

If you are feeling up to it, please try to get your code down to the smallest
version that still triggers this message. Ideally in a single Main.elm and
elm.json file.

From there open a NEW issue at https://github.com/elm/compiler/issues with your
reduced example pasted in directly. (Not a link to a repo or gist!) Do not worry
about if someone else saw something similar. More examples is better!

This kind of error is usually tied up in larger architectural choices that are
hard to change, so even when we have a couple good examples, it can take some
time to resolve in a solid way.elm: /root/elm-stuff/0.19.1/d.dat: openBinaryFile: resource busy (file is locked

SSCCE

• Run elm init

• Create a dummy src/Main.elm

• Put loads of stress on the system using something like stress

• Run elm make in a loop, forcing it to re-initialize elm-stuff: while (rm -rf elm-stuff; elm make src/Main.elm --output /dev/null); do true; done

• Elm: 0.19.1

Additional Details

An easy workaround is passing RTS flags to only allow Elm to use a single capability: +RTS -N1.

Patching the compiler to remove the use of forkIO from BackgroundWriter (admittedly negating the reason it exists in the first place) also seems to completely resolve this. At first, I thought this may be an ordering issue, where elm make runs withScope around (rather than within) acquiring and releasing the lock on elm-stuff, but that turns out to be a tangential issue.

[github-actions[bot]]

Thanks for reporting this! To set expectations:

• Issues are reviewed in batches, so it can take some time to get a response.
• Ask questions in a community forum. You will get an answer quicker that way!
• If you experience something similar, open a new issue. We like duplicates.

Finally, please be patient with the core team. They are trying their best with limited resources.