Error AttributeError: Can't pickle local object 'parallel.<locals>.init_worker'

Hi,

Thanks for building and maintaining maast. While trying to run it on a small dataset of ~80 genomes from the tutorial, it produced an error.

Command I ran:

maast end_to_end --in-dir ./101346 --out-dir ./101346_out

output:

skip ./101346/AM23-11_1.fastq.gz: not fasta format
skip ./101346/TM07-1_1.fastq.gz: not fasta format
skip ./101346/TF09-22_1.fastq.gz: not fasta format
skip ./101346/OM08-11_1.fastq.gz: not fasta format
skip ./101346/AF25-38AC_1.fastq.gz: not fasta format
skip ./101346/AF21-53_1.fastq.gz: not fasta format
skip ./101346/AF36-11BH_1.fastq.gz: not fasta format
skip ./101346/AM31-23_1.fastq.gz: not fasta format
skip ./101346/AM23-11_1.fastq.gz: not fasta format
skip ./101346/TM07-1_1.fastq.gz: not fasta format
skip ./101346/TF09-22_1.fastq.gz: not fasta format
skip ./101346/OM08-11_1.fastq.gz: not fasta format
skip ./101346/AF25-38AC_1.fastq.gz: not fasta format
skip ./101346/AF21-53_1.fastq.gz: not fasta format
skip ./101346/AF36-11BH_1.fastq.gz: not fasta format
skip ./101346/AM31-23_1.fastq.gz: not fasta format
[Warning] Total number of genomes (79) < min. number of genomes required for effective SNP calling with MAF 0.01 (100)
[Warning] Skip tag genome selection, all genomes will be used
reference genome path: ./101346/GUT_GENOME223927.fna
[building mash sketch]: start
[calculating mash distance]: start
[clustering] start
[clustering] done
./101346/GUT_GENOME265356.fna
Running mummer4; start
reference genome path: ./101346/GUT_GENOME265356.fna
[paired alignment]: start
[paired alignment]: done
Traceback (most recent call last):
  File "/share/maast-1.0.8-0/bin/maast.py", line 1395, in <module>
    main()
  File "/share/maast-1.0.8-0/bin/maast.py", line 1390, in main
    end2end_main(args)
  File "/share/maast-1.0.8-0/bin/maast.py", line 1330, in end2end_main
    call_snps_main(args)
  File "/share/maast-1.0.8-0/bin/maast.py", line 1187, in call_snps_main
    run_mummer4(args)
  File "/share/maast-1.0.8-0/bin/maast.py", line 514, in run_mummer4
    parallel(run_mummer4_single, arg_list, args['threads'])
  File "/share/maast-1.0.8-0/bin/maast.py", line 274, in parallel
    pool = mp.Pool(int(threads), init_worker)
  File "/lib/python3.10/multiprocessing/context.py", line 119, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
  File "/lib/python3.10/multiprocessing/pool.py", line 215, in __init__
    self._repopulate_pool()
  File "/lib/python3.10/multiprocessing/pool.py", line 306, in _repopulate_pool
    return self._repopulate_pool_static(self._ctx, self.Process,
  File "/lib/python3.10/multiprocessing/pool.py", line 329, in _repopulate_pool_static
    w.start()
  File "/lib/python3.10/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/lib/python3.10/multiprocessing/context.py", line 288, in _Popen
    return Popen(process_obj)
  File "/lib/python3.10/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/lib/python3.10/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/lib/python3.10/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/lib/python3.10/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'parallel.<locals>.init_worker'

Installed via conda on macos m1 chip.

zjshi / Maast

Error AttributeError: Can't pickle local object 'parallel.<locals>.init_worker' #24