sys.meta_path is None

[jaanckae]

When running the FUCHS command with a command that looks like this:

FUCHS -N 10_HLF_1min_treated-305643917 -D CircRNACount -B 10_HLF_1min_treated-305643917_merged.bam -A transcriptome_filtered.stringent.nochr.exon_bed6.bed -O 07_reconstrout -F 04_srout/mate1/Chimeric.out.junction -R 04_srout/mate2/Chimeric.out.junction -J 04_srout/Chimeric.out.junction -T tmp/ -p ensembl -r 1 -e 2 -q 1 -P 1 -s step6,step7,step8

I get the following error:

Exception ignored in: <function Pool.__del__ at 0x7efbffe0cf70> Traceback (most recent call last): File "/usr/local/lib/python3.9/dist-packages/multiprocess/pool.py", line 268, in __del__ File "/usr/local/lib/python3.9/dist-packages/multiprocess/queues.py", line 374, in put File "/usr/local/lib/python3.9/dist-packages/multiprocess/reduction.py", line 54, in dumps File "/usr/local/lib/python3.9/dist-packages/multiprocess/reduction.py", line 42, in __init__ File "/usr/local/lib/python3.9/dist-packages/dill/_dill.py", line 573, in __init__ ImportError: sys.meta_path is None, Python is likely shutting down

What would be the best way to fix this issue? What am I missing?

I think I could locate the issue to the from pathos.multiprocessing import ProcessingPool as Pool part

Thanks Jasper

[jaanckae]

Was in contact with the developer of the pathos module wo replied the following:

Do you expect it to close upon exiting the context? Currently, it does nothing. If I were to add a close(), then the pool would close... however, any new pool instance that is created with the same configuration would also be closed until restart() is called. Another option is to clear() the pool, which closes the pool then deletes the singleton... which would enable the creation of a new open pool -- however closing it could adversely affect any of the other handles to the pool singleton. Given the above, I eventually decided that the current behavior (not closing the pool) is the best on exit. What do you think?

I 'm trying to run this code in an apptainer instance with Python 3.9 on Debian

[tjakobi]

Hi @jaanckae,

Thank you for your report and thank you for already reaching out to the pathos developers.

What pathos version are you using? I remember I has issues with pathos/multiprocessing on MacOS recently, however, I don't have the time right now to follow up on that as FUCHS is entering legacy code status.

It might be a problem with a not correctly closed file connection.

At which step do yo see this error? This would help to find the corresponding file handle.

Thank you,

Tobias

[jaanckae]

Hi @tjakobi

I'm using pathos version 0.2.9. The problem is occurring when running the FUCHS command. I think it already rises up in step 2 (extract_reads) of the workflow. According to the pathos developers it has something to do with the combination with apptainer (singularity)

I'm thinking that the current issue stems from the apptainer (and other containers people are using). Generally the error is seen when working with selenium and similar.

Apparently it's an innocuous error, so you can likely ignore it. However, I believe you can use one of the two choices I mentioned above. Add a close or clear to the pool within the context. Try it and let me know if that removes the behavior you are seeing.