gemini load_chunks function is trying to create chunks in the below path which is not exists. temp directory path is /home/fazulur/tmp/ and my current working directory is /home/fazulur/create-gemini-db/.
/home/fazulur/tmp//home/fazulur/create-gemini-db/test.vcf.chunk3.db
Here is the Error:
2019-07-17 08:53:41.998 [IPClusterStop] Stopping cluster [pid=5089] with [signal=2]
Traceback (most recent call last):
File "$TOOl_DIR/gemini/v0.30.1/tools/bin/gemini", line 7, in
gemini_main.main()
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1249, in main
args.func(parser, args)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 204, in load_fn
gemini_load.load(parser, args)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 47, in load
load_ipython(args)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 99, in load_ipython
chunks = load_chunks_ipython(grabix_file, args, view)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 339, in load_chunks_ipython
chunk_dbs = view.map(load_chunk, chunk_steps, [load_args] total_chunks)
File "<$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/decorator.pyc:decorator-gen-137>", line 2, in map
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/view.py", line 52, in sync_results
ret = f(self, args, kwargs)
File "<$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/decorator.pyc:decorator-gen-136>", line 2, in map
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/view.py", line 37, in save_ids
ret = f(self, *args, *kwargs)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/view.py", line 1114, in map
return pf.map(sequences)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/remotefunction.py", line 299, in map
return self(sequences, ipp_mapping=True)
File "<$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/decorator.pyc:decorator-gen-119>", line 2, in call__
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/remotefunction.py", line 80, in sync_view_results
return f(self, args, kwargs)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/remotefunction.py", line 285, in call
return r.get()
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/asyncresult.py", line 169, in get
raise self.exception()
ipyparallel.error.CompositeError: one or more exceptions from call to method: load_chunk
[18:apply]: CalledProcessError: Command 'grabix grab test.vcf.gz 1 123256 | gemini load_chunk -v - -t snpEff --skip-gerp-bp --skip-cadd --skip-info-string --tempdir /home/fazulur/tmp -o 1 /home/fazulur/tmp//home/fazulur/create-gemini-db/test.vcf.chunk0.db' returned non-zero exit status 1
[24:apply]: CalledProcessError: Command 'grabix grab test.vcf.gz 123257 246512 | gemini load_chunk -v - -t snpEff --skip-gerp-bp --skip-cadd --skip-info-string --tempdir /home/fazulur/tmp -o 123257 /home/fazulur/tmp///home/fazulur/create-gemini-db/test.vcf.chunk1.db' returned non-zero exit status 1
[22:apply]: CalledProcessError: Command 'grabix grab test.vcf.gz 246513 369768 | gemini load_chunk -v - -t snpEff --skip-gerp-bp --skip-cadd --skip-info-string --tempdir /home/fazulur/tmp -o 246513 /home/fazulur/tmp///home/fazulur/create-gemini-db/test.vcf.chunk2.db' returned non-zero exit status 1
[38:apply]: CalledProcessError: Command 'grabix grab test.vcf.gz 369769 493024 | gemini load_chunk -v - -t snpEff --skip-gerp-bp --skip-cadd --skip-info-string --tempdir /home/fazulur/tmp -o 369769 /home/fazulur/tmp//home/fazulur/create-gemini-db/test.vcf.chunk3.db' returned non-zero exit status 1
.... 46 more exceptions ...
Dear Team,
I tried to run gemini database using LSF as scheduler and specifying tmp directory to some other location where I have more space. Command:
gemini load --cores 64 --scheduler lsf --queue normal --tempdir /home/fazulur/tmp/ -t snpEff -v test.vcf.gz test.db
gemini load_chunks function is trying to create chunks in the below path which is not exists. temp directory path is /home/fazulur/tmp/ and my current working directory is /home/fazulur/create-gemini-db/. /home/fazulur/tmp//home/fazulur/create-gemini-db/test.vcf.chunk3.db
Here is the Error:
2019-07-17 08:53:41.998 [IPClusterStop] Stopping cluster [pid=5089] with [signal=2] Traceback (most recent call last): File "$TOOl_DIR/gemini/v0.30.1/tools/bin/gemini", line 7, in
gemini_main.main()
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1249, in main
args.func(parser, args)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 204, in load_fn
gemini_load.load(parser, args)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 47, in load
load_ipython(args)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 99, in load_ipython
chunks = load_chunks_ipython(grabix_file, args, view)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 339, in load_chunks_ipython
chunk_dbs = view.map(load_chunk, chunk_steps, [load_args] total_chunks)
File "<$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/decorator.pyc:decorator-gen-137>", line 2, in map
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/view.py", line 52, in sync_results
ret = f(self, args, kwargs)
File "<$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/decorator.pyc:decorator-gen-136>", line 2, in map
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/view.py", line 37, in save_ids
ret = f(self, *args, *kwargs)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/view.py", line 1114, in map
return pf.map(sequences)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/remotefunction.py", line 299, in map
return self(sequences, ipp_mapping=True)
File "<$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/decorator.pyc:decorator-gen-119>", line 2, in call__
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/remotefunction.py", line 80, in sync_view_results
return f(self, args, kwargs)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/remotefunction.py", line 285, in call
return r.get()
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/asyncresult.py", line 169, in get
raise self.exception()
ipyparallel.error.CompositeError: one or more exceptions from call to method: load_chunk
[18:apply]: CalledProcessError: Command 'grabix grab test.vcf.gz 1 123256 | gemini load_chunk -v - -t snpEff --skip-gerp-bp --skip-cadd --skip-info-string --tempdir /home/fazulur/tmp -o 1 /home/fazulur/tmp//home/fazulur/create-gemini-db/test.vcf.chunk0.db' returned non-zero exit status 1
[24:apply]: CalledProcessError: Command 'grabix grab test.vcf.gz 123257 246512 | gemini load_chunk -v - -t snpEff --skip-gerp-bp --skip-cadd --skip-info-string --tempdir /home/fazulur/tmp -o 123257 /home/fazulur/tmp///home/fazulur/create-gemini-db/test.vcf.chunk1.db' returned non-zero exit status 1
[22:apply]: CalledProcessError: Command 'grabix grab test.vcf.gz 246513 369768 | gemini load_chunk -v - -t snpEff --skip-gerp-bp --skip-cadd --skip-info-string --tempdir /home/fazulur/tmp -o 246513 /home/fazulur/tmp///home/fazulur/create-gemini-db/test.vcf.chunk2.db' returned non-zero exit status 1
[38:apply]: CalledProcessError: Command 'grabix grab test.vcf.gz 369769 493024 | gemini load_chunk -v - -t snpEff --skip-gerp-bp --skip-cadd --skip-info-string --tempdir /home/fazulur/tmp -o 369769 /home/fazulur/tmp//home/fazulur/create-gemini-db/test.vcf.chunk3.db' returned non-zero exit status 1
.... 46 more exceptions ...
Could you please help me to resolve this issue.
Thanks In Advance Fazulur Rehaman