Difficulties deploying GOMAP using Singularity container

[jgreen51515]

My colleague has been trying to deploy GOMAP using the Singularity container and has run into a host of issues. His comments are below:

"I've made progress but I'm constantly stymied by the confounded documentation. They seem to have forked the containers and scripts they ask you to use to run the container into multiple versions. But none of that is documented. At this point I'm getting an error from the container trying to run its own Rscript. I think they are doing a lot of work on this with special implementation for individual HPC centers (the tags are condor, bridges which are clusters). However, this and much of the approach breaks the masks the beauty of containers. 1) They did not create a fully containerized repeatable scientific workflow. THey expect you to pull down extra code from git to actually run the workflow. THen they will push part of the workflow out to an external service that is called argus. 2) The forking into different (and more sloppily undocumented) versions breaks with that portability too."

We really want to try to use GOMAP, but are really limited by the documentation. Do you plan to improve your documentation? Also, I'd really hoped this would be deployed as a fully self-contained Singularity container. Do you have plans to improve the container so everything can be run locally instead of pushing code out to an external service?

[wkpalan]

We apologize for the issues encountered. We are currently working to fix a number of bugs before we update the documentation

I will respond with details to all you questions in the next few days

[wkpalan]

We have worked on releasing a properly documented container for GOMAP. Please do check it out at https://github.com/Dill-PICL/GOMAP-singularity

[jgreen51515]

Thank you! I will indeed check it out! :)

On Wed, Feb 20, 2019 at 3:03 PM Kokulapalan (Gokul) Wimalanathan < notifications@github.com> wrote:

We have worked on releasing a properly documented container for GOMAP. Please do check it out at https://github.com/Dill-PICL/GOMAP-singularity

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Dill-PICL/GOMAP/issues/1#issuecomment-465791130, or mute the thread https://github.com/notifications/unsubscribe-auth/ApMA3Sjt0CMOrEajqEX6Iw8oXkfr13_7ks5vPdQ4gaJpZM4WlzEv .

[jgreen51515]

Hello. We've again run into issues trying to set up and run GOMAP. Below is the feedback from a colleague who tried to install it via the Singularity container:

Best, Julie

I’ve spent some time trying to get GOMAP-singularity to run. There are a few issues I’ve run into and I anticipate some more based on the documentation. First, the set up script wants to download a rather large data set down with iCommands. We aren’t set up to go through our firewall to the source, Cyverse, but I downloaded with another approach. It would be nice if there was a way for the set up script to facilitate an already downloaded copy of the date file. Even better, already uncompressed. I’ve staged the data set where I believe it is expected. Another matter is that in order to run FANNGO Matlab is needed but this is documented as optional. If you want to use that the user needs to have a license. I’ve removed this step from the test script. I also, noted that one of the mixed methods is Argot 2.5 which is run on an external web server. I don’t think that is going to work for us so it would be nice if we could turn that off too. I’ve attempted to do that by removing the step from the script. I’ve tried to run both with and without the mixed method step removed and I get the error below.

In addition, to run this on more than one server they expect you to have a grid scheduler that we don’t have, SLURM. It would be nice if there was a Grid Engine option too (we use Univa Grid Engine).

./test-GOMAP.sh

There are no instances named GOMAP running.

Running Sequence-similarity based Annotation Step

/usr/local/lib/python2.7/dist-packages/Bio/SearchIO/init.py:211: BiopythonExperimentalWarning: Bio.SearchIO is an experimental submodule which may undergo significant changes prior to its future official release.

BiopythonExperimentalWarning)

Traceback (most recent call last):

File "./gomap.py", line 73, in

run_seqsim(config)

File "/opt/GOMAP/code/gomap_seqsim.py", line 19, in run_seqsim

make_input_blastdb(config)

File "/opt/GOMAP/code/pipeline/run_rbh_blast.py", line 14, in make_input_blastdb

make_blastdb(input_fa,config)

File "/opt/GOMAP/code/utils/blast_utils.py", line 13, in make_blastdb

check_output_and_run(fasta_db,makedb_command)

File "/opt/GOMAP/code/utils/basic_utils.py", line 23, in check_output_and_run

subprocess.check_call(command,stdin=stdin_file,stdout=stdout_file)

File "/usr/lib/python2.7/subprocess.py", line 185, in check_call

retcode = call(*popenargs, **kwargs)

File "/usr/lib/python2.7/subprocess.py", line 172, in call

return Popen(*popenargs, **kwargs).wait()

File "/usr/lib/python2.7/subprocess.py", line 394, in init

errread, errwrite)

File "/usr/lib/python2.7/subprocess.py", line 1047, in _execute_child

raise child_exception

OSError: [Errno 2] No such file or directory

Stopping GOMAP instance of /home/moughto1/gomap-singularity/gomap4/GOMAP.simg (PID=2654)

On Thu, Feb 21, 2019 at 1:08 PM Julie Green jgreen51515@gmail.com wrote:

Thank you! I will indeed check it out! :)

On Wed, Feb 20, 2019 at 3:03 PM Kokulapalan (Gokul) Wimalanathan < notifications@github.com> wrote:

We have worked on releasing a properly documented container for GOMAP. Please do check it out at https://github.com/Dill-PICL/GOMAP-singularity

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Dill-PICL/GOMAP/issues/1#issuecomment-465791130, or mute the thread https://github.com/notifications/unsubscribe-auth/ApMA3Sjt0CMOrEajqEX6Iw8oXkfr13_7ks5vPdQ4gaJpZM4WlzEv .