dandi / dandi-hub

Infrastructure and code for the dandihub
https://hub.dandiarchive.org
Other
11 stars 24 forks source link

Incorrect default number of workers in MATLAB server option #83

Open vijayiyer05 opened 1 year ago

vijayiyer05 commented 1 year ago

The Brain Observatory Toolbox lead developer has confirmed successful initial operation on the DandiHub MATLAB server option.

One issue has arisen however: the default number of parallel workers for 'Process' workers is somehow set to 48. Given there are only 6 cores, this should be set to 5 or 6.

Updating the default will help prevent errors/crashes in some workflows that rely on the default value.

Suggest to patch this for now as needed, e.g. using a startup file. Eventually the root cause should be identified and addressed.

vijayiyer05 commented 1 year ago

Any initial thoughts/insights @aranega?

aranega commented 1 year ago

@vijayiyer05 We have a good control over the startup.m file from the Dockerfile, so I think your proposal to set the number of the parallel workers there is the best option right now. I'll check how to modify this parameter programmatically and inject it in the startup file :)

ehennestad commented 1 year ago

I think this is what you need:

myCluster = parcluster('Processes')
myCluster.NumWorkers = 5
saveProfile(myCluster)

The saveProfile might not be necessary if its in the startup file

aranega commented 1 year ago

@ehennestad Thanks a lot! I'll update the startup file accordingly