search

ablab / IsoQuant

Transcript discovery and quantification with long RNA reads (Nanopores and PacBio)
https://ablab.github.io/IsoQuant/
Other
153 stars 14 forks source link

I was able to run smoothly after setting the parameter 'ulimit -n 1000000', thanks! #232

Open lmulroney opened 2 months ago

lmulroney commented 2 months ago 
          I was able to run smoothly after setting the parameter 'ulimit -n 1000000', thanks!

Originally posted by @shizhuoxing in https://github.com/ablab/IsoQuant/issues/133#issuecomment-1914198573

lmulroney commented 2 months ago

I'm getting the following error

2024-09-03 12:15:57,476 - INFO - Resolving multimappers 2024-09-03 12:16:05,263 - CRITICAL - IsoQuant failed with the following error, please, submit this issue to https://github.com/ablab/IsoQuant/issuesTraceback

File "/path/to/isoquant.py", line 819, in File "/path/to/isoquant.py", line 813, in main File "/path/to/isoquant.py", line 763, in run_pipeline File "/path/to/isoquant/share/isoquant-3.5.1-0/src/dataset_processor.py", line 458, in process_all_samples File "/path/to/isoquant/share/isoquant-3.5.1-0/src/dataset_processor.py", line 482, in process_sample File "/path/to/isoquant/share/isoquant-3.5.1-0/src/dataset_processor.py", line 573, in collect_reads File "/path/to/isoquant/share/isoquant-3.5.1-0/src/dataset_processor.py", line 621, in resolve_multimappers OSError: [Errno 24] Too many open files: '/outpath/isoquant_counts/Nat_H_r1/aux/Nat_H_r1.save_multimappers_HLA-B*35:02:01'

I read in the linked issue that using ulimit -n 1000000 could solve the problem, but I am not able to adjust the ulimit on my cluster environment. Any advice would be really helpdul!

Thanks, Logan

andrewprzh commented 2 months ago

Dear @lmulroney

I am not sure how your cluster is constructed and configured, but if you are using a cluster, I think you need to make sure that ulimit is set on the node the task is actually launched. You may contact your system administrator about this.

Also, what reference genome are you using? IsoQuant creates a separate file for each "chromosome" in the reference. If your reference contains a lot of scaffolds and patches (which you might be not interested at all), it may be useful to remove some sequences from you reference genome.

Best Andrey

lmulroney commented 2 months ago

Dear @andrewprzh,

Thanks for your rapid response!

I do not have permissions to adjust the ulimit on the cluster. I tried to adjust it in each job right before executing isoquant, but got an error, You do not have permission to adjust ulimit.

I am using GRCH38 full with all decoys, then use samtools to filter for only chrom 1-22, X,Y, and M. But I do not think that samtools removes the other contigs from the bam header. I will try to remove the unused headers, or simply align to the primary assembly in the first place and give it another go.

Thanks for your feedback, Logan