Incredibly slow (near unusable) on Docker container

[mbrio]

I've begun using Docker containers for many of my projects and have noticed how incredibly slow Jest runs in a Docker container when executing on a shared folder on OS X. Within Docker you can mount a local path and map it to a path within the container (e.g. ~/Source/projecta:/usr/src/app). When using this configuration my tests which consists of a single file with a single unit test takes over 30 seconds; but when run locally outside of Docker it takes 1 second. I'm using the command jest specs/unit; which I got from another Jest issue suggesting that supplying the test folder could speed up tests, and it did, locally, from 3 seconds to 1 second; but made no change to how it runs in Docker. I would have left it at that and said that it's a Docker for Mac issue (especially because there is a speed issue with mounted volumes that is known) but I ran the same exact single test using mocha and it executed in milliseconds. The unit test is a simple expect(true).toBe(true) (or for mocha+chai expect(true).to.equal(true) and for some reason Jest, under Docker, executed in ~30 seconds vs mocha's less than 1 second.

Are there any configuration settings or optimizations that I could use to help speed up Jest? I would much rather use Jest over Mocha, but will obviously use the test platform that executes in a reasonable amount of time. This is my Jest configuration:

"jest": {
    "automock": false,
    "verbose": true,
    "testFileExtensions": [
      "js"
    ],
    "moduleFileExtensions": [
      "js",
      "json"
    ],
    "testDirectoryName": "spec/unit",
    "testPathIgnorePatterns": [
      "/node_modules/",
      "<rootDir>/lib/"
    ],
    "modulePathIgnorePatterns": [
      "/node_modules/",
      "<rootDir>/lib/"
    ]
  }

[cpojer]

Hmm, that is very weird. Is the file system access slow, that's really the only thing that should be different here between Jest and mocha? Jest crawls the file system from the root (package.json) and looks at all the files for static analysis. How many cores does the system report that it has? Can you try invoking Jest with -i?

[cpojer]

Another thing is that if you are using babel-jest, babel actually loads a ton of modules so if the fs is slow, this might contribute to that.

[aaronabramov]

i remember having similar kind of issues inside docker container. The problem was the slowness of fs access. I think it was related to mounting a directory through a virtual container

[aaronabramov]

found the issue https://github.com/docker/compose/issues/631 not sure if it's still relevant

[aaronjensen]

This is a known docker for mac issue: https://forums.docker.com/t/file-access-in-mounted-volumes-extremely-slow-cpu-bound/8076/158

[cpojer]

Yeah I'm gonna close this out because unfortunately it isn't actionable from our side :(

[mbrio]

Hi @cpojer, sorry I didn't mean to drop off like that, I did get a chance to run jest -i, I also installed native watchman and ran jest -i --watchman, neither of which sped up the processing of tests. As I mentioned in my original post I understood that there was a speed issue with Docker and mounted drives. I would like to point out that I have implemented mocha in conjunction with chokidar for watching files and still do not see the kind of slowdown I'm seeing with jest. Chokidar+mocha detects file changes in no more than 2 seconds and executed tests in under a second; jest takes over 30 seconds, sometimes up to a minute to detect a file change and executes the single test in a little over a second. Also, I'd like to note that I am not using babel-jest, I am using only the features of es6 that are available to node 6. I understand if this is something you are not looking to persue, but there does seem to be some sort of bottleneck in how jest watches files when compared to chokidar. Finally, as to your question about cores, my VM has 2 cores and 4gb of RAM.

[amkoehler]

@mbrio I decided to switch over to mocha, and the tests run immediately within my docker container without an issue. No changes were made to my Dockerfile or docker-compose configuration. This seems like a Jest issue and not a Docker issue. I had no issues with running my Jests tests locally, only within the container.

[mbrio]

@amkoehler yeah, I think there's a significant difference in the libraries that Jest and Mocha use for watching filesystem changes. I assume that Mocha is less thorough, (and/or) less active, and/or less compatible with all filesystems. The speed issue is definitely an issue with Docker for Mac as I have friends who have slightly different configurations for running Docker on a Mac that do not see as big of an issue. With that said, since Docker for Mac is the official way to run Docker instances, it would be great for the developers of Jest to consider looking into how to be a bit more compatible with it.

[wmartins]

I came to this issue lots of times when trying to figure it out why jest was slow. Somehow, in my scenario, I was able to improve the performance. I'll just leave it here in case anyone is trying to figure it out too.

Scenario:

Building my application inside a docker container.

In my case, the container was based on Linux Alpine.

My Solution:

I changed the container base image to use Ubuntu instead of Alpine. My tests started to run faster (from 3 ~ 4 min to 40s ~ 60s).

Hope this helps someone! :)

[alexgorbatchev]

This page pretty much says that osxfs is slow and that's being worked on...

As of Dec'17, running jest can be considered unusable in my opinion. I'm seeing 28s time for 3 basic test files inside a container. Exact same thing on the host takes 0.4s. This can work for CI purposes, but not for developer env.

There are a few clunky workarounds for this, all of which in my opinion mostly defeat the point of running development env inside containers. docker-sync and d4m-nfs are considered to be currently best solutions for this. Personally I'm not interested in trying either because if I can't use docker pull as the only command to "setup" my dev environment, then I may as well not bother until osxfs is fixed.

[thymikee]

@rogeliog is this something you could potentially find someone in Docker team to fix?

[YRM64]

https://medium.com/@kevinsimper/how-to-disable-jsdom-in-jest-make-jest-run-twice-as-fast-a01193f23405

[jpbochi]

enabling the delegated flag (eg. docker -v ${PWD}:${PWD}:delegated) can make jester considerably faster, but still not close to native. Docs about this flag at at https://docs.docker.com/docker-for-mac/osxfs-caching/

[amayun]

I've investigated a bit, and it seems the source of slowness is this._crawl It crawls whole file tree from the root of project, including node_modules - and due slow fs access from docker to osxfs it takes almost 2 minutes to walk whole file tree. And for me solution was simply add "roots": ["./src"] to jest.config.js

[SimenB]

That's great!

6732 landed recently and fixed the roots support, so that's probably a viable solution for folks wanting to run things in docker now

[neilpalima]

Is the fix already on the latest jest version? I am using the latest and running 3 tests on docker took 2+ mins.

[cdomigan]

Using the new PnP feature of Yarn enabled Jest to run fast in Docker for me. It removes the node_modules folder meaning Jest doesn't have to crawl it.

[virasana]

"roots": ["./src"] not supported for create-react-app (boo!)

[elliotlarson]

I feel like this is still an issue. Running a single test file was taking me 17-20 seconds in a Docker container. After some Googling, I landed on this issue. I already have a jest roots config setup, like @amayun recommended. But, using the delegated approach recommended by @jpbochi allowed me to cut the run time down to 8 seconds. However, that's still a long time to wait for a single test to run. I hear that this is basically a Docker/Mac/Jest issue, and I get that there are Docker pieces that can't be addressed by the Jest team. Still though, this seems like a pretty common set of technologies to develop with. And, given that Mocha seems to run quickly in a Docker container leads me to believe that there could be something tweakable on the Jest side of things.

[jpbochi]

I agree with @elliotlarson. Even if most of the blame is on Docker for Mac, I believe jest could optimize its scan for tests in ways that would help regardless of Docker.

[SimenB]

Ideas and PRs are most welcome. It's, as far as I know, not an issue for anyone on the core team - so if anything is gonna happen here it'll have to come from the community

[jpbochi]

@SimenB Fair enough. I only wished there was some incentive for the core team to try to reach more people. However cool jest might be, there are alternatives out there. And being super slow to run is a deal breaker.

[jasonwilliams]

Linking to https://github.com/docker/for-mac/issues/1358

[toryas]

I had a similar problem, in a kubernete's pod the unit-test takes until 200 segs to done, then I found thins issue on Jest's GitHub.

So, I add this --maxWorkers=1 to jest script and then test-units just takes 14 segs to done

[SamanShafigh]

I use create-react-app and I can use the --roots to fix the issue. This is they way I run my test using react-scripts.

./node_modules/.bin/react-scripts test --roots "${PWD}/src"

Note: I am using "react-scripts": "^2.1.5" and react-scripts test is documented that --roots accepts an array of strings but when I define it as an array like the following code it is not working.

./node_modules/.bin/react-scripts test --roots ["${PWD}/src"]

However if I pass it as string it will work. I don't know if I am doing a mistake or something

[ghost]

would be useful to have a debug method for whats going on. For example I tried to figure out why my glob patterns were not improving speed until I found this issue https://github.com/facebook/jest/issues/7031 which made me realize all the files are crawled before the filter. so I added a "roots" folder and it sped up the crawling significantly.

[jpbochi]

there's a new issue to work on a docker-for-mac improvement to solve this: https://github.com/docker/roadmap/issues/7

[cklll]

(I don't use watch mode, I run test manually after I made some changes)

My issue is mounting the node_modules/ directory so the setup time is very long (15s) With the below change, it is around 5s

Before

    volumes:
      - .:/opt/app
      - .:/opt/app:cached # Or with cache, but still slow

After

    volumes:
      - .:/opt/app
      - /opt/app/node_modules/

[sunderls]

@jpbochi 's mention solved for my case. using :delegated though still a lot slower than native ones, but acceptable now

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.