search

AntonelliLab / seqcap_processor

Bioinformatic pipeline for processing Sequence Capture data for Phylogenetics
MIT License
21 stars 12 forks source link

FileExistsError: [Errno 17] File exists: '/home/gmeng/work/4.assembly/secapr/April-5-2022/stats' #27

Open linzhi2013 opened 2 years ago

linzhi2013 commented 2 years ago

Dear Tobias,

I have 97 samples to analyze with SEQCAP, but it always failed when it just finished 81 samples. The error is weird:

multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/home/gmeng/.conda/envs/mybase/envs/secapr_env/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/gmeng/.conda/envs/mybase/envs/secapr_env/lib/python3.8/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
  File "/home/gmeng/.conda/envs/mybase/envs/secapr_env/lib/python3.8/site-packages/secapr/assemble_reads.py", line 288, in process_subfolder
    os.makedirs(out_dir)
  File "/home/gmeng/.conda/envs/mybase/envs/secapr_env/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
FileExistsError: [Errno 17] File exists: '/home/gmeng/work/4.assembly/secapr/April-5-2022/stats'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/gmeng/.conda/envs/mybase/envs/secapr_env/bin/secapr", line 10, in <module>
    sys.exit(main())
  File "/home/gmeng/.conda/envs/mybase/envs/secapr_env/lib/python3.8/site-packages/secapr/__main__.py", line 55, in main
    args.func(args)
  File "/home/gmeng/.conda/envs/mybase/envs/secapr_env/lib/python3.8/site-packages/secapr/assemble_reads.py", line 381, in main
    contig_count_df_list = list(pool.map(partial(process_subfolder), pool_args))
  File "/home/gmeng/.conda/envs/mybase/envs/secapr_env/lib/python3.8/multiprocessing/pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/home/gmeng/.conda/envs/mybase/envs/secapr_env/lib/python3.8/multiprocessing/pool.py", line 771, in get
    raise self._value
FileExistsError: [Errno 17] File exists: '/home/gmeng/work/4.assembly/secapr/April-5-2022/stats'

because the /home/gmeng/work/4.assembly/secapr/April-5-2022/stats path should have been created at the very beginning.

It seems that it is similar to https://github.com/Theano/Theano/issues/6610.

Currently, I can't figure out the exact reasons for this error, so I just simply rerun the unfinished samples as I talked about in https://github.com/AntonelliLab/seqcap_processor/issues/26.

Cheers Guanliang