Closed vorce closed 2 years ago
And a thank you for all your excellent work! 💚
Awesome job @vorce! A fix will be available in the next Elixir version!
Wow that was fast. Thanks José! It was a big team effort at Remote 😃
Thanks for the quick fix @josevalim !
Elixir and Erlang/OTP versions
Erlang/OTP 25 [erts-13.0.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit:ns]
Elixir 1.13.4 (compiled with Erlang/OTP 25)
Operating system
Both macOS 12.5.1 and Debian (seen on both local execution as well as our CI)
Current behavior
Hello folks!
This week we’ve noticed an increase in our quite large test suite getting killed on CI servers (16GB RAM, 8vCPUs). The test suite consists of 18221 test cases, most of them async.
After some investigation we found that the
ExUnit.TestModule
structs can become very large (~137MB for a single phoenix controller test with ~110 test cases). Digging further we see that the test context is included in eachExUnit.Test
struct is sent to the formatters.Unfortunately for us we added our OpenApiSpex spec to the context in the Web.ConnCase module 🤦
We did this out of convenience, since the spec is needed to make assertions on if a response matches a spec (which we use a lot). The result is that the (very big!) spec is in every
ExUnit.Test
struct for our controller tests! We've fixed this, but still have some questions:tags
(merge starting)? It can grow quite big even with more sensible data in there (like ecto schemas). It gets worse the more ExUnit formatters you’re using.When we adjusted our test suite to remove the specs from the test context our memory usage dropped from ~9GB-16GB+ to a measly ~1.3GB. So, the gains from this are very significant.
As an aside, before that running without a formatter also already helped reduce memory usage.
If you need more details we’re happy to share! Thanks a lot.
Expected behavior
We’re unsure about the behavior of contexts being merged and kept as part of the
ExUnit.Test
struct - it seems to us that there could be room for improvement here (removed or reduced).Alternatively, guidance/docs to restrict context usage would be great.