aitgon / vtam

MIT License
3 stars 3 forks source link

Error taxonomy #8

Open meglecz opened 3 years ago

meglecz commented 3 years ago

(vtam) emese@pcf-meglecz:~/vtam_demo/vtam_db$ vtam taxonomy --output taxonomy.tsv

new_taxdump.tar.gz: : 119824384it [00:42, 2834732.87it/s]
Traceback (most recent call last): File "/home/emese/miniconda3/envs/vtam/bin/vtam", line 8, in sys.exit(main()) File "/home/emese/miniconda3/envs/vtam/lib/python3.7/site-packages/vtam/init.py", line 273, in main VTAM(sys.argv[1:]) File "/home/emese/miniconda3/envs/vtam/lib/python3.7/site-packages/vtam/init.py", line 244, in init taxonomy.main(precomputed=precomputed) File "/home/emese/miniconda3/envs/vtam/lib/python3.7/site-packages/vtam/CommandTaxonomy.py", line 188, in main self.create_denovo_from_ncbi() File "/home/emese/miniconda3/envs/vtam/lib/python3.7/site-packages/vtam/CommandTaxonomy.py", line 81, in create_denovo_from_ncbi tar.extractall(path=self.tempdir) File "/home/emese/miniconda3/envs/vtam/lib/python3.7/tarfile.py", line 2000, in extractall numeric_owner=numeric_owner) File "/home/emese/miniconda3/envs/vtam/lib/python3.7/tarfile.py", line 2042, in extract numeric_owner=numeric_owner) File "/home/emese/miniconda3/envs/vtam/lib/python3.7/tarfile.py", line 2112, in _extract_member self.makefile(tarinfo, targetpath) File "/home/emese/miniconda3/envs/vtam/lib/python3.7/tarfile.py", line 2161, in makefile copyfileobj(source, target, tarinfo.size, ReadError, bufsize) File "/home/emese/miniconda3/envs/vtam/lib/python3.7/tarfile.py", line 247, in copyfileobj buf = src.read(bufsize) File "/home/emese/miniconda3/envs/vtam/lib/python3.7/gzip.py", line 287, in read return self._buffer.read(size) File "/home/emese/miniconda3/envs/vtam/lib/python3.7/_compression.py", line 68, in readinto data = self.read(len(byte_view)) File "/home/emese/miniconda3/envs/vtam/lib/python3.7/gzip.py", line 482, in read uncompress = self._decompressor.decompress(buf, size) zlib.error: Error -3 while decompressing data: invalid block type

aitgon commented 3 years ago

Is this error reproducible? Did you try later or in a different computer with different internet connection? Maybe the file was not completely downloaded at that time because of some internet problem.

meglecz commented 3 years ago

It happened several times (within a 30 minutes interval) on the same computer with a quite good connection.

I also tried on a different desktop with a much worse connection. The error message is more explicit in this case.

(vtam) 19:04 meglecz@bombyx ~/vtam_demo/vtam_db % vtam taxonomy --output taxonomy_20210426.tsv new_taxdump.tar.gz: 20%|███████████████ | 23560192/117703462 [01:17<05:07, 305673.49it/s] Traceback (most recent call last): File "/home/meglecz/miniconda3/envs/vtam/bin/vtam", line 8, in sys.exit(main()) File "/home/meglecz/miniconda3/envs/vtam/lib/python3.7/site-packages/vtam/init.py", line 273, in main VTAM(sys.argv[1:]) File "/home/meglecz/miniconda3/envs/vtam/lib/python3.7/site-packages/vtam/init.py", line 244, in init taxonomy.main(precomputed=precomputed) File "/home/meglecz/miniconda3/envs/vtam/lib/python3.7/site-packages/vtam/CommandTaxonomy.py", line 188, in main self.create_denovo_from_ncbi() File "/home/meglecz/miniconda3/envs/vtam/lib/python3.7/site-packages/vtam/CommandTaxonomy.py", line 64, in create_denovo_from_ncbi new_taxdump_path = self.__download_ncbi_taxonomy_dump() File "/home/meglecz/miniconda3/envs/vtam/lib/python3.7/site-packages/vtam/CommandTaxonomy.py", line 60, in __download_ncbi_taxonomy_dump urllib.request.urlretrieve(remotefile, new_taxdump_path, reporthook=tqdm_hook(t)) File "/home/meglecz/miniconda3/envs/vtam/lib/python3.7/urllib/request.py", line 288, in urlretrieve % (read, size), result) urllib.error.ContentTooShortError: <urlopen error retrieval incomplete: got only 23554080 out of 117703462 bytes> zsh: exit 1 vtam taxonomy --output taxonomy_20210426.tsv

meglecz commented 3 years ago

The command runs without error un a cluster:

vtam taxonomy --output taxonomy.tsv new_taxdump.tar.gz: : 117710848it [00:07, 15455653.32it/s]

ls taxonomy.tsv

aitgon commented 3 years ago

Me too I did not see any error. I think this is an internet problem. Can we close this issue?

meglecz commented 3 years ago

Is it possible to add an explicit error message ? It is frustrating to have an error without knowing what to do.