search

tanghaibao / jcvi

Python library to facilitate genome assembly, annotation, and comparative genomics
BSD 2-Clause "Simplified" License
761 stars 186 forks source link

The error of running ALLMAP #246

Closed shehongbing closed 4 years ago

shehongbing commented 4 years ago

Hi, when I used test data to test the ALLMAP, and I am sure install all the python modules, but it does not work. The error log is following:

=================================== Traceback (most recent call last): File "/data/pub/liuzy/miniconda2/lib/python2.7/runpy.py", line 174, in _run_module_as_main "main", fname, loader, pkg_name) File "/data/pub/liuzy/miniconda2/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/data/pub/liuzy/miniconda2/lib/python2.7/site-packages/jcvi/assembly/allmaps.py", line 2031, in main() File "/data/pub/liuzy/miniconda2/lib/python2.7/site-packages/jcvi/assembly/allmaps.py", line 857, in main p.dispatch(globals()) File "/data/pub/liuzy/miniconda2/lib/python2.7/site-packages/jcvi/apps/base.py", line 111, in dispatch globalsaction File "/data/pub/liuzy/miniconda2/lib/python2.7/site-packages/jcvi/assembly/allmaps.py", line 1495, in path weights = Weights(weightsfile, mapnames) File "/data/pub/liuzy/miniconda2/lib/python2.7/site-packages/jcvi/assembly/allmaps.py", line 653, in init super(Weights, self).init(filename, cast=cast) File "/data/pub/liuzy/miniconda2/lib/python2.7/site-packages/jcvi/formats/base.py", line 101, in init self[key] = value File "/data/pub/liuzy/miniconda2/lib/python2.7/collections.py", line 76, in setitem root = self.__root AttributeError: 'Weights' object has no attribute '_OrderedDict__root'

shehongbing commented 4 years ago

my command is 'python -m jcvi.assembly.allmaps path JM-2.test.bed scaffolds.fasta'

tanghaibao commented 4 years ago

Duplicate of #227

tanghaibao commented 4 years ago

@shehongbing

This issue should have been fixed in the recent version (v1.0.6). Would you please try upgrading jcvi?

pip install -U jcvi

Also, please note that we are migrating to Python 3 now and Python 2 support (which is what you used) will be dropped soon.

Haibao

shehongbing commented 4 years ago

I just re-run the command by using Python3.7. And the jcvi version is v1.0.6. But the error still exists; And the error log is following:

ESC[0;33m12:56:04 [base]ESC[0mESC[0;35m Load file /data/pub/shehb/pacbio/Pacbio_HIC/ALLMAPS-testdata/scaffolds.fastaESC[0m ESC[0;33m12:56:05 [base]ESC[0mESC[0;35m Load file scaffolds.fasta.sizesESC[0m ESC[0;33m12:56:05 [allmaps]ESC[0mESC[0;35m Working on JMMale-23|JMFemale-23 ...ESC[0m ESC[0;33m12:56:05 [allmaps]ESC[0mESC[0;35m Graph size: |V|=24, |E|=474.ESC[0m ESC[0;33m12:56:06 [base]ESC[0mESC[0;35m tsp_work not found. Creating new.ESC[0m ESC[0;33m12:56:06 [allmaps]ESC[0mESC[0;35m concorde-TSP failed. Use default scaffold ordering.ESC[0m ESC[0;33m12:56:06 [allmaps]ESC[0mESC[0;35m Start EC round 0ESC[0m ESC[0;33m12:56:06 [ec]ESC[0mESC[0;35m GA setup: ngen=500 npop=100 cpus=16 seed=666ESC[0m Traceback (most recent call last): File "/data/pub/shehb/miniconda2/envs/trinity/lib/python3.7/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/data/pub/shehb/miniconda2/envs/trinity/lib/python3.7/runpy.py", line 85, in _run_code exec(code, run_globals) File "/data/pub/shehb/.local/lib/python3.7/site-packages/jcvi/assembly/allmaps.py", line 2037, in main() File "/data/pub/shehb/.local/lib/python3.7/site-packages/jcvi/assembly/allmaps.py", line 863, in main p.dispatch(globals()) File "/data/pub/shehb/.local/lib/python3.7/site-packages/jcvi/apps/base.py", line 111, in dispatch globalsaction File "/data/pub/shehb/.local/lib/python3.7/site-packages/jcvi/assembly/allmaps.py", line 1600, in path seed=seed, File "/data/pub/shehb/.local/lib/python3.7/site-packages/jcvi/assembly/allmaps.py", line 205, in init toolbox, ngen=ngen, npop=npop, cpus=cpus, seed=seed, callback=callbacki File "/data/pub/shehb/.local/lib/python3.7/site-packages/jcvi/algorithms/ec.py", line 179, in GA_run pool = multiprocessing.Pool(cpus) File "/data/pub/shehb/miniconda2/envs/trinity/lib/python3.7/multiprocessing/context.py", line 119, in Pool context=self.get_context()) File "/data/pub/shehb/miniconda2/envs/trinity/lib/python3.7/multiprocessing/pool.py", line 158, in init self._setup_queues() File "/data/pub/shehb/miniconda2/envs/trinity/lib/python3.7/multiprocessing/pool.py", line 251, in _setup_queues self._inqueue = self._ctx.SimpleQueue() File "/data/pub/shehb/miniconda2/envs/trinity/lib/python3.7/multiprocessing/context.py", line 112, in SimpleQueue return SimpleQueue(ctx=self.get_context()) File "/data/pub/shehb/miniconda2/envs/trinity/lib/python3.7/multiprocessing/queues.py", line 332, in init self._rlock = ctx.Lock() File "/data/pub/shehb/miniconda2/envs/trinity/lib/python3.7/multiprocessing/context.py", line 67, in Lock return Lock(ctx=self.get_context())

tanghaibao commented 4 years ago

@shehongbing

It is working on my end though - the error message that you posted is truncated and miss the critical error traceback at the end. Please post more when you have it.

shehongbing commented 4 years ago

Traceback (most recent call last): File "/data/pub/shehb/miniconda2/envs/trinity/lib/python3.7/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/data/pub/shehb/miniconda2/envs/trinity/lib/python3.7/runpy.py", line 85, in _run_code exec(code, run_globals) File "/data/pub/shehb/.local/lib/python3.7/site-packages/jcvi/assembly/allmaps.py", line 2037, in main() File "/data/pub/shehb/.local/lib/python3.7/site-packages/jcvi/assembly/allmaps.py", line 863, in main p.dispatch(globals()) File "/data/pub/shehb/.local/lib/python3.7/site-packages/jcvi/apps/base.py", line 111, in dispatch globalsaction File "/data/pub/shehb/.local/lib/python3.7/site-packages/jcvi/assembly/allmaps.py", line 1600, in path seed=seed, File "/data/pub/shehb/.local/lib/python3.7/site-packages/jcvi/assembly/allmaps.py", line 205, in init toolbox, ngen=ngen, npop=npop, cpus=cpus, seed=seed, callback=callbacki File "/data/pub/shehb/.local/lib/python3.7/site-packages/jcvi/algorithms/ec.py", line 179, in GA_run pool = multiprocessing.Pool(cpus) File "/data/pub/shehb/miniconda2/envs/trinity/lib/python3.7/multiprocessing/context.py", line 119, in Pool context=self.get_context()) File "/data/pub/shehb/miniconda2/envs/trinity/lib/python3.7/multiprocessing/pool.py", line 158, in init self._setup_queues() File "/data/pub/shehb/miniconda2/envs/trinity/lib/python3.7/multiprocessing/pool.py", line 251, in _setup_queues self._inqueue = self._ctx.SimpleQueue() File "/data/pub/shehb/miniconda2/envs/trinity/lib/python3.7/multiprocessing/context.py", line 112, in SimpleQueue return SimpleQueue(ctx=self.get_context()) File "/data/pub/shehb/miniconda2/envs/trinity/lib/python3.7/multiprocessing/queues.py", line 332, in init self._rlock = ctx.Lock() File "/data/pub/shehb/miniconda2/envs/trinity/lib/python3.7/multiprocessing/context.py", line 67, in Lock return Lock(ctx=self.get_context()) File "/data/pub/shehb/miniconda2/envs/trinity/lib/python3.7/multiprocessing/synchronize.py", line 162, in init SemLock.init(self, SEMAPHORE, 1, 1, ctx=ctx) File "/data/pub/shehb/miniconda2/envs/trinity/lib/python3.7/multiprocessing/synchronize.py", line 59, in init unlink_now) PermissionError: [Errno 13] Permission denied

tanghaibao commented 4 years ago

@shehongbing

Weird error. It seems that the multiprocessing module isn't working properly for the python that you are using. This has little to do with ALLMAPS.

See also: https://www.reddit.com/r/learnpython/comments/bspue5/os_error_permission_denied_when_using_mppool/

Are you running on a Linux server or some sort of virtual environment?

shehongbing commented 4 years ago

I run it on a Linux. Thank you very much

shehongbing commented 4 years ago

I' sorry. I can not open the Link you provided. BTW, when I run the ALLMAP, I just got the *.tour file

shehongbing commented 4 years ago

Hi, I run it successfully. Now I have another question. the following is the result of ALLMAP. I used the genetica and his result from ALLHIC. But I do not think it's a good result because the consensus map only with 76.4% bases.

Summary for each individual map 

                     o           SLAF_genetic                     hic
        Linkage Groups                      6                     992
      Markers (unique)                  3,119                  18,510
        Markers per Mb                    7.1                    20.1
         N50 Scaffolds                    500                     739
             Scaffolds                  1,541                   9,255

Scaffolds with 1 marker 916 0 Scaffolds with 2 markers 303 9,255 Scaffolds with 3 markers 126 0 Scaffolds with >=4 markers 196 0 Total bases 440,389,050 (47.8%) 921,342,157 (100.0%)

Summary for consensus map 

                     o               Anchored               Oriented               Unplaced
      Markers (unique)                 11,412                 11,317                 10,217
        Markers per Mb                   16.2                   16.2                   47.0
         N50 Scaffolds                    682                    677                     57
             Scaffolds                  4,312                  4,287                  4,943

Scaffolds with 1 marker 0 0 0 Scaffolds with 2 markers 3,035 3,035 4,679 Scaffolds with 3 markers 701 682 215 Scaffolds with >=4 markers 576 570 49 Total bases 704,131,065 (76.4%) 699,809,467 (76.0%) 217,211,092 (23.6%)