search

papaemmelab / toil_cnacs

toil wrapper for CNACS
MIT License
9 stars 3 forks source link

Error while generating normal pools #4

Open tuhina3184 opened 2 years ago

tuhina3184 commented 2 years ago

Description

I am setting up a CNACS run using 5 samples as a panel of normals which is sequences using targeted panel. BAM files are generated using trimmomatic (trimming) and bwa (aligner) and are mapped to the hg19 reference genome.

What I Did

I am running CNACS with 5 samples to generate normal pool with the following code -
toil_cnacs_ generate_pool CNV_test/toil_cnacs/normal_pools/jobstore_generate_pool \
        --stats \
        --writeLogs CNV_test/toil_cnacs/normal_pools/toil_logs \
        --logFile CNV_test/toil_cnacs/normal_pools/toil_logs.txt \
        --outdir CNV_test/toil_cnacs/normal_pools/ \
        --probe_bed bedfile/06112021_Leukemia_Panel_sorted.bed \
        --fasta reference_genomes/hg19_broad/hg19_all.fasta \
        --pool_samp F1/F1.final.bam F \
        --pool_samp F2/F2.final.bam F \
        --pool_samp M1/M1.final.bam M \
        --pool_samp M2/M2.final.bam M \
        --pool_samp NA12878/NA12878.final.bam F \
        --db_dir /scratch/hematopath/programs/toil_cnacs/toil_cnacs/data/

The log file showed the following error -

_DEBUG:toil.resource:Module dir is /scratch/hematopath/miniconda3/lib/python3.7/site-packages DEBUG:toil.resource:Module dir is /scratch/hematopath/miniconda3/lib/python3.7/site-packages DEBUG:toil.resource:Module dir is /scratch/hematopath/miniconda3/lib/python3.7/site-packages Traceback (most recent call last): File "/scratch/hematopath/miniconda3/bin/toil_cnacs", line 8, in sys.exit(main()) File "/scratch/hematopath/miniconda3/lib/python3.7/site-packages/toil_cnacs/cli.py", line 53, in main commands.main(step="generate_pool") File "/scratch/hematopath/miniconda3/lib/python3.7/site-packages/toil_cnacs/commands.py", line 337, in main run_toil(toil_options=args, step=step) File "/scratch/hematopath/miniconda3/lib/python3.7/site-packages/toil_cnacs/commands.py", line 314, in run_toil ContainerJob.Runner.startToil(head, toiloptions) File "/scratch/hematopath/miniconda3/lib/python3.7/site-packages/toil/job.py", line 786, in startToil return toil.start(job) File "/scratch/hematopath/miniconda3/lib/python3.7/site-packages/toil/common.py", line 758, in start self.writePIDFile() File "/scratch/hematopath/miniconda3/lib/python3.7/site-packages/toil/common.py", line 1086, in writePIDFile f.write(str(os.getpid())) TypeError: a bytes-like object is required, not 'str'

Can you please help me out with this issue.

Cheers, Tuhina Srivastava

juanesarango commented 2 years ago

Hi Tuhina, the error is caused by using python 3.x. This version uses a toil version that works only in python 2.x.

I suggest you try running it with python 2.7, or any 2.x. Until we update the package to use python 3.

tuhina3184 commented 2 years ago

Thankyou for your response.

Python3 is made default in our shared server that our institute is using and I tried creating python2.7 environment using conda to switch between them. $python --version shows Python 2.7.15 But when I am running the same toil_cnacs command I am getting the same error as I stated in the issue message and it seems it is still using python 3.7.

_Traceback (most recent call last): File "/scratch/hematopath/miniconda3/bin/toil_cnacs", line 8, in sys.exit(main()) File "/scratch/hematopath/miniconda3/lib/python3.7/site-packages/toil_cnacs/cli.py", line 53, in main commands.main(step="generate_pool") File "/scratch/hematopath/miniconda3/lib/python3.7/site-packages/toil_cnacs/commands.py", line 337, in main run_toil(toil_options=args, step=step) File "/scratch/hematopath/miniconda3/lib/python3.7/site-packages/toil_cnacs/commands.py", line 314, in run_toil ContainerJob.Runner.startToil(head, toil_options) File "/scratch/hematopath/miniconda3/lib/python3.7/site-packages/toil/job.py", line 786, in startToil return toil.start(job) File "/scratch/hematopath/miniconda3/lib/python3.7/site-packages/toil/common.py", line 758, in start self.writePIDFile() File "/scratch/hematopath/miniconda3/lib/python3.7/site-packages/toil/common.py", line 1086, in writePIDFile f.write(str(os.getpid())) TypeError: a bytes-like object is required, not 'str' (python2) [hematopath@hpc toil_cnacs]$ (python2) [hematopath@hpc toilcnacs]$ python --version Python 2.7.15

As you can see, it is still using python3.7 even though conda python2 is activated. Do you think I need to install toil_cnacs again with python2.7 ?

Cheers, Tuhina

juanesarango commented 2 years ago

yes, reinstall it with python 2.

I would suggest creating a conda virtual environment with python2, and install toil_cnacs in it.

tuhina3184 commented 2 years ago

Sure, will do that. I'll let you know if I encounter any other error. Thankyou

MatteoZam commented 2 years ago

I have the same problem. Unfortunately, when you try to reinstall toil with python 2.7 several errors occur. In particular there is the requirement of packaging 21.0 that is not supported by python 2.7.

How can i fix this problem?

Thanks for any kind of help

Mat

$pip install . DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support Processing /Users/matteozampini/Documents/toil_cnacs Collecting docker==2.5.1 Using cached https://files.pythonhosted.org/packages/b8/4c/fb6c7d26ecfb6ea9e24d940edec4c67c2b9fc7c95c72435dfa4a5a0c6085/docker-2.5.1-py2.py3-none-any.whl Collecting toil_container>=1.1.4 Using cached https://files.pythonhosted.org/packages/07/30/1317446f5486d05585f9448668750d20d67c2972582f55eeeca47b9c4a49/toil_container-2.0.3-py2.py3-none-any.whl Collecting backports.ssl-match-hostname>=3.5; python_version < "3.5" Downloading https://files.pythonhosted.org/packages/ff/2b/8265224812912bc5b7a607c44bf7b027554e1b9775e9ee0de8032e3de4b2/backports.ssl_match_hostname-3.7.0.1.tar.gz Collecting ipaddress>=1.0.16; python_version < "3.3" Downloading https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl Collecting docker-pycreds>=0.2.1 Using cached https://files.pythonhosted.org/packages/f5/e8/f6bd1eee09314e7e6dee49cbe2c5e22314ccdb38db16c9fc72d2fa80d054/docker_pycreds-0.4.0-py2.py3-none-any.whl Collecting websocket-client>=0.32.0 Downloading https://files.pythonhosted.org/packages/f7/0c/d52a2a63512a613817846d430d16a8fbe5ea56dd889e89c68facf6b91cb6/websocket_client-0.59.0-py2.py3-none-any.whl (67kB) |████████████████████████████████| 71kB 18.2MB/s Collecting six>=1.4.0 Using cached https://files.pythonhosted.org/packages/d9/5a/e7c31adbe875f2abbb91bd84cf2dc52d792b5a01506781dbcf25c91daf11/six-1.16.0-py2.py3-none-any.whl Collecting requests!=2.11.0,!=2.12.2,!=2.18.0,>=2.5.2 Using cached https://files.pythonhosted.org/packages/2d/61/08076519c80041bc0ffa1a8af0cbd3bf3e2b62af10435d269a9d0f40564d/requests-2.27.1-py2.py3-none-any.whl ERROR: Could not find a version that satisfies the requirement packaging>=21.0 (from toil_container>=1.1.4->toil-cnacs==0.2.0) (from versions: 14.0, 14.1, 14.2, 14.3, 14.4, 14.5, 15.0, 15.1, 15.2, 15.3, 16.0, 16.1, 16.2, 16.3, 16.4, 16.5, 16.6, 16.7, 16.8, 17.0, 17.1, 18.0, 19.0, 19.1, 19.2, 20.0, 20.1, 20.2, 20.3, 20.4, 20.5, 20.6, 20.7, 20.8, 20.9) ERROR: No matching distribution found for packaging>=21.0 (from toil_container>=1.1.4->toil-cnacs==0.2.0)

mflevine commented 2 years ago

Hi, I believe you can freeze the version of toil_container here to toil_container==1.1.4. It looks like the new versions of toil require a higher packaging than provided by Python2.

Hope this helps.