Closed sonnyarora closed 4 years ago
Thanks for reporting this! This seems to be caused by resource leakage when pybbi raises exceptions internally. Will work on a fix here: https://github.com/nvictus/pybbi/issues/10
This is now fixed in pybbi 0.2.2
Thanks for taking care of that! What are the next steps to get higlass-server to use the new version of pybbi?
If you're using the docker image, it will be distributed in the next release, probably in the next day or so. If you're running higlass-server in a local environment, you just need to upgrade pybbi via pip
.
Should be updated in v0.6.39 of the Docker container 👍 Please let us know if you still encounter this issue.
And thanks for reporting it :-)
Hey, thanks for taking care of that! I'm not seeing any open bigwigs in lsof anymore. Also, extra file handles don't appear to be opened when zooming. Marking this as closed.
I am running higlass through docker on CentOS 7.7.1908. I ran into an issue when I had several files loaded in HiGlass. In particular, when I have 8 bigWigs and an mcool file in HiGlass, after zooming in and out, I see the following error in the higlass container in /data/log/hgs.log :
[Errno 24] Too many open files: '/data/media/mm10.tsv'
Running some tests: With no loaded files, I check how many open files all the uwsgi processes have open:
lsof -p 2640,2649,2651,2652,2653 | wc -l
I get 553. After loading a single big wig, the number rises to: 1020. As I zoom in, the number keeps increasing: 1020,1031,1259. Passing this input to grep to search only for the names of the bigWig I have loaded in higlass, I see more lines appear as I zoom. I see the same pattern of the number of open files increasing as I zoom in different locations with all 8 bigWigs and the mcool file loaded until I get the above error.
The limit on number of open files for a single process set by default for my CentOS is 1024. If I increase this limit, I don't run into the same problem. Setting this limit high enough is a good enough solution for me for this problem.