search

Lumieducation / H5P-Nodejs-library

h5p-nodejs-library is a collection of server and client-side packages that help you use H5P with a NodeJS backend
https://docs.lumi.education
GNU General Public License v3.0
157 stars 77 forks source link

Test cases run out of memory #221

Closed sr258 closed 4 years ago

sr258 commented 4 years ago

The test cases in CI run out of memory regularly. This might be because of the "integration" tests that use all the hub examples. If these tests are run in parallel, it will use a lot of memory. We should either increase the resources of the test runners or think of a way of limiting the parallelism of tests.

JPSchellenberg commented 4 years ago

As we don't pay for CircleCI we can not increase resources. We could enforce a queue via the workflows. So certain tests have to pass first before other tests are run. It would increase the running-time, though, but that is fine for me.

JPSchellenberg commented 4 years ago

On second thought. I think that every test runs in its own virtual machine. Limiting the parallelism of the tests would not increase resources for one specific test, as every virtual machine has the same amount of resources. Maybe we have to improve our tests.

sr258 commented 4 years ago

Ok, I‘ll watch which tests are affected and report back here when I‘ve identified the culprit.

sr258 commented 4 years ago

I've moved the integration tests into a different directory and they are now run at a different IC stage with limited workers.

sr258 commented 4 years ago

The memory problems have returned, so I'm reopening this issue...

I've played around with this and using npx jest test/JsonStorage.test.ts --detectLeaks (requires weak package to be installed) shows memory leaks in 13 tests. Running npx jest --runInBand --logHeapUsage shows that the heap usage increases with every test, which is an indication of a memory leak. :(

However, I've been unsuccessful so far in finding the cause of this. Even a very simple test like JsonStorage.test.ts seems to have a memory leak, which is really weird, as not much is done there. I've reduced the test to just reading the JSON file from the disk and this still produces a memory leak!

Because of this, I suspect that the leak is (not necessarily) in our code but caused by Jest. A very similar issue is discussed (here)[https://github.com/facebook/jest/issues/7874] but it seems unresolved. I would still not rule it out that there might also be a memory leak in our code, though.

Temporarily, this problem might be adressed by restricted the parallelism of the tests further (see PR)