Open rderollepot opened 1 year ago
Remaining problem:
- The app works fine for me in production mode (using
launch.sh
), but somehow I can't get it working in dev mode (usinglaunch_dev.sh
), where I keep having this loop:2023-02-08 14:52:07,596 - distributed.nanny.memory - WARNING - Worker tcp://127.0.0.1:51827 (pid=5170) exceeded 95% memory budget. Restarting... 2023-02-08 14:52:07,694 - distributed.core - INFO - Connection to tcp://127.0.0.1:51829 has been closed. 2023-02-08 14:52:07,695 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://127.0.0.1:51827', status: running, memory: 15, processing: 3> 2023-02-08 14:52:07,695 - distributed.core - INFO - Removing comms to tcp://127.0.0.1:51827 2023-02-08 14:52:07,695 - distributed.nanny - INFO - Worker process 5170 was killed by signal 15 2023-02-08 14:52:07,695 - distributed.scheduler - INFO - Lost all workers 2023-02-08 14:52:07,697 - distributed.nanny - WARNING - Restarting worker
I suspected this was a performance issue with my machine and indeed (dev and production mode seem to have different ways of allocating resources ?), when increasing the number of workers/threads in launch_dev.sh
it works fine too.
dask worker 127.0.0.1:8786 --nworkers 1 --nthreads 6 --local-directory work-dir &
This dashboard really interested me as it shows an example on how to deal with making a map based on a huge dataset. Unfortunately it was not working out of the box, but with a few updates I got it working again.
Problems solved:
Remaining problem:
launch.sh
), but somehow I can't get it working in dev mode (usinglaunch_dev.sh
), where I keep having this loop: