search

MIT-LCP / physionet-build

The new PhysioNet platform.
https://physionet.org/
BSD 3-Clause "New" or "Revised" License
56 stars 20 forks source link

Too many open files #2242

Open bemoody opened 4 months ago

bemoody commented 4 months ago

The project file upload handler will fail (EMFILE: Too many open files) if there are too many files included in the upload.

"DATA_UPLOAD_MAX_NUMBER_FILES" is set to 1000, but the process's FD limit is presumably set to the default of 1024. If we want to allow 1000 files at a time, we should raise the rlimit.

Also note that the inline script in edit_files_panel.html doesn't check if the number of files will be permitted.

bemoody commented 4 months ago

Having 18 to 20 FDs open when idle seems typical:

$ sudo lsof -a -c uwsgi -d 0-999999999 | awk '{print $2}' | uniq -c
      1 PID
      6 485021
     38 485022
     18 815631
     20 815844
     19 815877
     20 816536
     18 818355
     19 818500
     20 819132
     20 819671
     20 820690
     20 825215

The one process with 38 is presumably the "emperor" that is managing the others, and isn't handling requests itself.