search

zhangrengang / TEsorter

TEsorter: an accurate and fast method to classify LTR-retrotransposons in plant genomes
https://doi.org/10.1093/hr/uhac017
GNU General Public License v3.0
86 stars 19 forks source link

AssertionError #3

Closed oushujun closed 4 years ago

oushujun commented 4 years ago

Dear Ren-gang,

I was using TEsorter to identify TE-related genes and encounter this error:

2019-10-08 19:46:02,971 -INFO- total 31808 sequences classified. 2019-10-08 19:46:02,971 -INFO- see classified sequences in mRNA.fa.rexdb.cls.tsv 2019-10-08 19:46:02,971 -INFO- writing library for RepeatMasker in mRNA.fa.rexdb.cls.lib 2019-10-08 19:46:07,722 -INFO- writing classified protein domains in mRNA.fa.rexdb.cls.pep Traceback (most recent call last): File "/home/oushujun/las/git_bin/TEsorter/TEsorter.py", line 906, in pipeline(Args()) File "/home/oushujun/las/git_bin/TEsorter/TEsorter.py", line 203, in pipeline assert raw_id in d_class AssertionError

I was only using the -p 16 parameter: nohup python2 ~/las/git_bin/TEsorter/TEsorter.py mRNA.fa -p 16 &

However, I am still getting the expected results. This error was not persistent, which only occurred on some runs. Please take a look. Thanks!

Best, Shujun

zhangrengang commented 4 years ago

@oushujun, please send me the mRNA.fa to make me reproduce the error. Thanks.

zhangrengang commented 4 years ago

@oushujun, it might because there are some special characters in sequence id, such as |. I have revised it and you can re-run it. Note that you should run sh build_database.sh firstly to build HMM database using the new version.

oushujun commented 4 years ago

@zhangrengang, thanks for the update. I saw it has a new dependency psutil. I was trying to install it with pip but not successful:

python2 -m pip install --user psutil Collecting psutil Using cached https://files.pythonhosted.org/packages/1c/ca/5b8c1fe032a458c2c4bcbe509d1401dca9dda35c7fc46b36bb81c2834740/psutil-5.6.3.tar.gz Installing collected packages: psutil Running setup.py install for psutil ... error Complete output from command /usr/bin/python2 -u -c "import setuptools, tokenize;file='/tmp/pip-build-8MIs9q/psutil/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-ilPM2r-record/install-record.txt --single-version-externally-managed --compile --user --prefix=: /usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires' warnings.warn(msg) running install running build running build_py creating build creating build/lib.linux-x86_64-2.7 creating build/lib.linux-x86_64-2.7/psutil copying psutil/init.py -> build/lib.linux-x86_64-2.7/psutil copying psutil/_common.py -> build/lib.linux-x86_64-2.7/psutil copying psutil/_compat.py -> build/lib.linux-x86_64-2.7/psutil copying psutil/_psaix.py -> build/lib.linux-x86_64-2.7/psutil copying psutil/_psbsd.py -> build/lib.linux-x86_64-2.7/psutil copying psutil/_pslinux.py -> build/lib.linux-x86_64-2.7/psutil copying psutil/_psosx.py -> build/lib.linux-x86_64-2.7/psutil copying psutil/_psposix.py -> build/lib.linux-x86_64-2.7/psutil copying psutil/_pssunos.py -> build/lib.linux-x86_64-2.7/psutil copying psutil/_pswindows.py -> build/lib.linux-x86_64-2.7/psutil creating build/lib.linux-x86_64-2.7/psutil/tests copying psutil/tests/init.py -> build/lib.linux-x86_64-2.7/psutil/tests copying psutil/tests/main.py -> build/lib.linux-x86_64-2.7/psutil/tests copying psutil/tests/runner.py -> build/lib.linux-x86_64-2.7/psutil/tests copying psutil/tests/test_aix.py -> build/lib.linux-x86_64-2.7/psutil/tests copying psutil/tests/test_bsd.py -> build/lib.linux-x86_64-2.7/psutil/tests copying psutil/tests/test_connections.py -> build/lib.linux-x86_64-2.7/psutil/tests copying psutil/tests/test_contracts.py -> build/lib.linux-x86_64-2.7/psutil/tests copying psutil/tests/test_linux.py -> build/lib.linux-x86_64-2.7/psutil/tests copying psutil/tests/test_memory_leaks.py -> build/lib.linux-x86_64-2.7/psutil/tests copying psutil/tests/test_misc.py -> build/lib.linux-x86_64-2.7/psutil/tests copying psutil/tests/test_osx.py -> build/lib.linux-x86_64-2.7/psutil/tests copying psutil/tests/test_posix.py -> build/lib.linux-x86_64-2.7/psutil/tests copying psutil/tests/test_process.py -> build/lib.linux-x86_64-2.7/psutil/tests copying psutil/tests/test_sunos.py -> build/lib.linux-x86_64-2.7/psutil/tests copying psutil/tests/test_system.py -> build/lib.linux-x86_64-2.7/psutil/tests copying psutil/tests/test_unicode.py -> build/lib.linux-x86_64-2.7/psutil/tests copying psutil/tests/test_windows.py -> build/lib.linux-x86_64-2.7/psutil/tests running build_ext building 'psutil._psutil_linux' extension creating build/temp.linux-x86_64-2.7 creating build/temp.linux-x86_64-2.7/psutil gcc -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -fPIC -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -I/include -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=563 -DPSUTIL_LINUX=1 -I/usr/include/python2.7 -c psutil/_psutil_common.c -o build/temp.linux-x86_64-2.7/psutil/_psutil_common.o gcc: error: unrecognized command line option ‘-fno-plt’ error: command 'gcc' failed with exit status 1

----------------------------------------

Command "/usr/bin/python2 -u -c "import setuptools, tokenize;file='/tmp/pip-build-8MIs9q/psutil/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-ilPM2r-record/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-build-8MIs9q/psutil/ You are using pip version 18.1, however version 19.2.3 is available. You should consider upgrading via the 'pip install --upgrade pip' command.`

The gcc of my system is 4.8.5, which seems to be too old for psutil based on this post. Is there a way to install psutil without updating the gcc? Or is it possible to drop this dependency?

Thanks! Shujun

zhangrengang commented 4 years ago

@oushujun, I have dropped this dependency and you can re-try it. My gcc is 5.4.0 and I installed psutil with pip install psutil. I have no experience with older gcc. You may install older psutil by pip install psutil=="an older version". The older psutil may support an old gcc.

zhangrengang commented 4 years ago

@oushujun, here is the version list of psutil. You may try from 5.5.0 by pip install psutil==5.5.0.

oushujun commented 4 years ago

@zhangrengang, thanks for the list. I tried from 5.5.0 to all the way down the versions. It turns out 0.2.0 is the highest version that supports gcc 4.8.5. I am curious, what benefit does this module provide?

zhangrengang commented 4 years ago

@oushujun, it provide CPU and Memory count, which make flexible to run shell commands. However, TEsorter have not been benifited from it. The newest version of TEsorter do not depend on this module.

oushujun commented 4 years ago

Sounds good! Do I need to run sh build_database.sh every time I used the program? I was switching from a previous version using git checkout, and then the program asked me to hmmpress again.

zhangrengang commented 4 years ago

@oushujun, you need to run sh build_database.sh only once for a recent version which HMM databases have been deleted to make it smaller. hmmpress is very fast. If there are no HMM databases (database/.hmm.h), you have to run sh build_database.sh again.

zhangrengang commented 4 years ago

@oushujun, the HMM databases is database/*.hmm.h*.

oushujun commented 4 years ago

Thanks for clarifying. I want to make a note here psutil==0.2.0, gcc==4.8.5 run without error message and blazingly fast, but the number is much less than the version without psutil. Not sure what was wrong. Removing the dependency on psutil certainly made things easier.