could not find drmaa library

[lamevan]

Hi,I run the ProcessCollapsedAssembly.snake.sh and it occured an error: Could you explain why this error occured?THANKS

Building DAG of jobs... Exception in thread Thread-1: Traceback (most recent call last): File "/home/linpeifan/Software/sd/SDA/envs/sda-python-3/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/home/linpeifan/Software/sd/SDA/envs/sda-python-3/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/linpeifan/Software/sd/SDA/envs/sda-python-3/lib/python3.6/site-packages/snakemake/executors.py", line 973, in _wait_for_jobs import drmaa File "/home/linpeifan/Software/sd/SDA/envs/sda-python-3/lib/python3.6/site-packages/drmaa/init.py", line 65, in from .session import JobInfo, JobTemplate, Session File "/home/linpeifan/Software/sd/SDA/envs/sda-python-3/lib/python3.6/site-packages/drmaa/session.py", line 39, in from drmaa.helpers import (adapt_rusage, Attribute, attribute_names_iterator, File "/home/linpeifan/Software/sd/SDA/envs/sda-python-3/lib/python3.6/site-packages/drmaa/helpers.py", line 36, in from drmaa.wrappers import (drmaa_attr_names_t, drmaa_attr_values_t, File "/home/linpeifan/Software/sd/SDA/envs/sda-python-3/lib/python3.6/site-packages/drmaa/wrappers.py", line 54, in '{0}').format(_drmaa_lib_env_name)) RuntimeError: Could not find drmaa library. Please specify its full path using the environment variable DRMAA_LIBRARY_PATH

WorkflowError: Error loading drmaa support: Could not find drmaa library. Please specify its full path using the environment variable DRMAA_LIBRARY_PATH

[mrvollger]

DRMMA is a way to submit jobs on a cluster system like sungrid, and it looks like it is not installed on your system.

If you do NOT want to do a cluster submission then edit ProcessCollapsedAssembly.snake.sh so that part containing "sungrid" == "sungrid" evaluates to false, e.g. "sungrid" == "x".

I you have gird then you can install drmaa with ease (I hope):

source /path/to/your/sda/repo/env_python3.cfg conda install drmaa source deactivate

And try again.

[mrvollger]

Also out of personal scientific curiosity, what genome are you working on? (feel free not to answer)

[lamevan]

Rice Ok,I will try!thanks

[lamevan]

I tried it. It seems to have been installed, but it is still prompted that the file cannot be found

[mrvollger]

I have never tried a plant genome so I am interested in what your results will be.

If you are just running my test case then you can run it without a cluster (should run easily with 8 threads and 32GB of total ram). In which case I recommend the "sungrid" == "x" option.

However, when you move onto the rice genome you may want to get cluster submissions working. DRMMA is not actually necessary so give me a little time and I will update the run script to have a mode to do cluster submissions without DRMMA.

[lamevan]

I tried setting the parameter to "sungrid" == "x"and run. And an error occured: /home/linpeifan/Software/sd/SDA/envs/sda-python-3/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(*args, *kwds) /home/linpeifan/Software/sd/SDA/envs/sda-python-3/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(args, **kwds) Building DAG of jobs... Using shell: /bin/bash Provided cores: 28 Rules claiming more threads will be scaled down. Job counts: count jobs 18 BamIndex 18 BamToCoverage 1 BedForCollapses 1 CountOverlappingRepeatElements 1 FilterCollapses 1 GenerateBatchRunScript 1 GenerateMinAndMax 1 GetCoverageStats 1 LocalAssembliesBam 1 LocalAssembliesBed 1 LocalAssembliesConfig 1 LocalAssembliesRef 1 LocalAssembliesRegions 1 MakeASMsa 18 MapReads 1 MergeBed 1 MergeBedForCollapses 1 RepeatMaskerBed 1 all 1 mergeRepeatMasker 1 mergeTRFandRM 72

[Tue Apr 23 21:46:13 2019] rule mergeRepeatMasker: input: reference/mask0/ref.0.fasta.out, reference/mask1/ref.1.fasta.out output: reference/ref.RM.out jobid: 76

            cat reference/mask0/ref.0.fasta.out reference/mask1/ref.1.fasta.out > reference/ref.RM.out

[Tue Apr 23 21:46:13 2019] rule MakeASMsa: input: ref.fasta output: reference/denovo.sa jobid: 52

sawriter ref.fasta Removing temporary output file reference/mask0/ref.0.fasta.out. Removing temporary output file reference/mask1/ref.1.fasta.out. [Tue Apr 23 21:46:14 2019] Finished job 76. 1 of 72 steps (1%) done

[Tue Apr 23 21:46:14 2019] rule RepeatMaskerBed: input: reference/ref.RM.out output: reference/ref.RM.out.bed jobid: 74

            /home/linpeifan/Software/sd/SDA//scripts/RepeatMaskingToBed.py reference/ref.RM.out

/home/linpeifan/Software/sd/SDA/envs/sda-python-3/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(*args, *kwds) /home/linpeifan/Software/sd/SDA/envs/sda-python-3/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(args, **kwds) Removing temporary output file reference/ref.RM.out. [Tue Apr 23 21:46:20 2019] Finished job 74. 2 of 72 steps (3%) done

[Tue Apr 23 21:46:20 2019] rule mergeTRFandRM: input: reference/ref.RM.out.bed output: reference/all.repeats.bed jobid: 51

cat reference/ref.RM.out.bed | sort -k1,1 -k2,2n | bedtools merge -i - > reference/all.repeats.bed

Removing temporary output file reference/ref.RM.out.bed. [Tue Apr 23 21:46:24 2019] Finished job 51. 3 of 72 steps (4%) done Waiting at most 1000 seconds for missing files. MissingOutputException in line 233 of /home/linpeifan/Software/sd/SDA/ProcessCollapsedAssembly.py: Missing files after 1000 seconds: reference/denovo.sa This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait. Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Complete log: /home/linpeifan/Software/sd/testiDentity/GenomeTest/.snakemake/log/2019-04-23T214610.495656.snakemake.log Can't find file /home/linpeifan/.vnc/mu01:38.pid You'll have to kill the Xvnc process manually Could you explain why this error occured?

I really appreciate your efforts on update the run script.

[mrvollger]

Hmm. It is odd that the rule executes without error but reference/denovo.sa is not made. Does it appear in the directory at a later time?

I am swamped this week but I will try to find a fix for you by the end of the week. Sorry about all these issues.

[mchaisso]

Possibly the rule should write reference/ref.fa.sa, not reference/denovo.sa ? I'll be running some SDA here as well, so I can try and submit PRs -Mark

On Wed, Apr 24, 2019 at 3:41 PM Mitchell Robert Vollger < notifications@github.com> wrote:

Hmm. It is odd that the rule executes without error but reference/ denovo.sa is not made. Does it appear in the directory at a later time?

I am swamped this week but I will try to find a fix for you by the end of the week. Sorry about all these issues.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/mvollger/SDA/issues/8#issuecomment-486454491, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKPGPQVMKFK52MTQF6JZK3PSDOYPANCNFSM4HHP57SQ .

[mrvollger]

Mark is right I made a mistake in moving the output file of sawritter.

I will push a change today, but only after I actually run the test case this time....

[mrvollger]

Test case runs so this commit 217b838e386711d6e2d4d18f7aede44da513c934 should fix it. Pull and let me know if it fixes it for you. Thanks for your patience!

[lamevan]

The test case now can be sucessfully running.Thank you!