Cannot install bds for RNAsik-pipe

Hi. Just got the Ansible code running. Everything was fine until I hit the following:

TASK [bds : build BigDataScript 0.99999g with java_local] ************************************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": true, "cmd": "cd /Users/ayates/bioansible/software/source/BigDataScript-0.99999g/; ./scripts/install.sh", "delta": "0:00:25.324549", "end": "2018-02-16 14:25:09.317847", "msg": "non-zero return code", "rc": 127, "start": "2018-02-16 14:24:43.993298", "stderr": "+ BDS_HOME=/Users/ayates/.bds\n++ dirname ./scripts/install.sh\n+ echo 'Changing dir' ./scripts\n++ dirname ./scripts/install.sh\n+ cd ./scripts\n+ cd ..\n+ mkdir -p /Users/ayates/.bds\n+ mkdir bin\n+ echo Building JAR file\n+ ant\n+ echo\n+ echo Building GO program\n+ cd go/bds/\n++ pwd\n+ export GOPATH=/Users/ayates/bioansible/software/source/BigDataScript-0.99999g/go/bds\n+ GOPATH=/Users/ayates/bioansible/software/source/BigDataScript-0.99999g/go/bds\n+ go clean\n./scripts/install.sh: line 27: go: command not found", "stderr_lines": ["+ BDS_HOME=/Users/ayates/.bds", "++ dirname ./scripts/install.sh", "+ echo 'Changing dir' ./scripts", "++ dirname ./scripts/install.sh", "+ cd ./scripts", "+ cd ..", "+ mkdir -p /Users/ayates/.bds", "+ mkdir bin", "+ echo Building JAR file", "+ ant", "+ echo", "+ echo Building GO program", "+ cd go/bds/", "++ pwd", "+ export GOPATH=/Users/ayates/bioansible/software/source/BigDataScript-0.99999g/go/bds", "+ GOPATH=/Users/ayates/bioansible/software/source/BigDataScript-0.99999g/go/bds", "+ go clean", "./scripts/install.sh: line 27: go: command not found"], "stdout": "Changing dir ./scripts\nBuilding JAR file\nBuildfile: /Users/ayates/bioansible/software/source/BigDataScript-0.99999g/build.xml\n\nclean:\n   [delete] Deleting directory /Users/ayates/bioansible/software/source/BigDataScript-0.99999g/bin\n    [mkdir] Created dir: /Users/ayates/bioansible/software/source/BigDataScript-0.99999g/bin\n\ncompile:\n    [javac] Compiling 403 source files to /Users/ayates/bioansible/software/source/BigDataScript-0.99999g/bin\n    [javac] /Users/ayates/bioansible/software/source/BigDataScript-0.99999g/src/org/bds/data/DataFtp.java:9: warning: FtpURLConnection is internal proprietary API and may be removed in a future release\n    [javac] import sun.net.www.protocol.ftp.FtpURLConnection;\n    [javac]                                ^\n    [javac] warning: Supported source version 'RELEASE_6' from annotation processor 'org.antlr.v4.runtime.misc.NullUsageProcessor' less than -source '1.8'\n    [javac] /Users/ayates/bioansible/software/source/BigDataScript-0.99999g/src/org/bds/data/DataFtp.java:9: warning: FtpURLConnection is internal proprietary API and may be removed in a future release\n    [javac] import sun.net.www.protocol.ftp.FtpURLConnection;\n    [javac]                                ^\n    [javac] /Users/ayates/bioansible/software/source/BigDataScript-0.99999g/src/org/bds/data/DataFtp.java:9: warning: FtpURLConnection is internal proprietary API and may be removed in a future release\n    [javac] import sun.net.www.protocol.ftp.FtpURLConnection;\n    [javac]                                ^\n    [javac] /Users/ayates/bioansible/software/source/BigDataScript-0.99999g/src/org/bds/data/DataFtp.java:27: warning: FtpURLConnection is internal proprietary API and may be removed in a future release\n    [javac] \t\t((FtpURLConnection) connection).close();\n    [javac] \t\t  ^\n    [javac] 5 warnings\n\ncreate_run_jar:\n     [copy] Copying 1 file to /Users/ayates/bioansible/software/source/BigDataScript-0.99999g/bin/org/bds\n     [copy] Copying 1 file to /Users/ayates/bioansible/software/source/BigDataScript-0.99999g/bin/org/bds\n     [copy] Copying 1 file to /Users/ayates/bioansible/software/source/BigDataScript-0.99999g/bin/org/bds\n      [jar] Building jar: /Users/ayates/.bds/bds.jar\n\nBUILD SUCCESSFUL\nTotal time: 24 seconds\n\nBuilding GO program", "stdout_lines": ["Changing dir ./scripts", "Building JAR file", "Buildfile: /Users/ayates/bioansible/software/source/BigDataScript-0.99999g/build.xml", "", "clean:", "   [delete] Deleting directory /Users/ayates/bioansible/software/source/BigDataScript-0.99999g/bin", "    [mkdir] Created dir: /Users/ayates/bioansible/software/source/BigDataScript-0.99999g/bin", "", "compile:", "    [javac] Compiling 403 source files to /Users/ayates/bioansible/software/source/BigDataScript-0.99999g/bin", "    [javac] /Users/ayates/bioansible/software/source/BigDataScript-0.99999g/src/org/bds/data/DataFtp.java:9: warning: FtpURLConnection is internal proprietary API and may be removed in a future release", "    [javac] import sun.net.www.protocol.ftp.FtpURLConnection;", "    [javac]                                ^", "    [javac] warning: Supported source version 'RELEASE_6' from annotation processor 'org.antlr.v4.runtime.misc.NullUsageProcessor' less than -source '1.8'", "    [javac] /Users/ayates/bioansible/software/source/BigDataScript-0.99999g/src/org/bds/data/DataFtp.java:9: warning: FtpURLConnection is internal proprietary API and may be removed in a future release", "    [javac] import sun.net.www.protocol.ftp.FtpURLConnection;", "    [javac]                                ^", "    [javac] /Users/ayates/bioansible/software/source/BigDataScript-0.99999g/src/org/bds/data/DataFtp.java:9: warning: FtpURLConnection is internal proprietary API and may be removed in a future release", "    [javac] import sun.net.www.protocol.ftp.FtpURLConnection;", "    [javac]                                ^", "    [javac] /Users/ayates/bioansible/software/source/BigDataScript-0.99999g/src/org/bds/data/DataFtp.java:27: warning: FtpURLConnection is internal proprietary API and may be removed in a future release", "    [javac] \t\t((FtpURLConnection) connection).close();", "    [javac] \t\t  ^", "    [javac] 5 warnings", "", "create_run_jar:", "     [copy] Copying 1 file to /Users/ayates/bioansible/software/source/BigDataScript-0.99999g/bin/org/bds", "     [copy] Copying 1 file to /Users/ayates/bioansible/software/source/BigDataScript-0.99999g/bin/org/bds", "     [copy] Copying 1 file to /Users/ayates/bioansible/software/source/BigDataScript-0.99999g/bin/org/bds", "      [jar] Building jar: /Users/ayates/.bds/bds.jar", "", "BUILD SUCCESSFUL", "Total time: 24 seconds", "", "Building GO program"]}

Do you know what's gone wrong here?

@andrewyatz sorry about late ish replay. So building BigDataScript (BDS) requires three dependencies

java1.8
go (go lang)
ant

From the error it appears that golang isn't installed. can you do this? This is from office github wiki

Try:

sudo apt-get install golang-go

If that's too old for you, try:

sudo add-apt-repository ppa:gophers/archive
sudo apt-get update
sudo apt-get install golang-1.9-go

This is for debian based distros of course

Thanks for trying

Also @andrewyatz once everything has been installed, can you double check that this bds.config file has 644 permission on it. I think there is a minor glitch in bio-ansible that I need to fix.

That file (bds.config) should be located about there ...apps/BigDataScript-version/bds.config this will depend on your installation path. Default installation path is ~/bioansible/software/apps/BigDataScript...

To change permissions use sudo chmod 644 path/to/your/bds.config

p.s bio-ansible is a separate project that I'm involved in, with multiple developers. I'm just using bio-ansible to install RNAsik with all its dependencies. All those errors are not directly related to RNAsik. Feel free to suggest an alternative build/installation way. I've been considering these two:

conda which isn't just python specific, it is actually quite generic. The problem with conda package, we need an existing BigDataScript and GoLang conda packages, which we currently don't have.
docker container very common approach is to build and provide docker image. We have that in beta, but I'm not a big fan of docker mainly because it isn't as easy to run as people think. You can't run docker images on most HPC clusters due to docker's security model.

Just one more comment to clarify bio-ansible and RNAsik installation, specifically this error with BDS installation.

bio-ansible would install golang, ant and all other required dependencies if you run with all.yml instead of bio.yml, however all.yml requires sudo privileged and will apt-get install stuff for you. I would only recommend using all.yml if you are using ephemeral instance, that you gonna scrape after you've tested RNAsik.

If you have spare minute have a glance at bio-ansible docs

cheers

Thank you for the details. I'd suggest you should change your docs to say that java1.8, go and ant are requirements. I couldn't find anything in https://monashbioinformaticsplatform.github.io/RNAsik-pipe/docs/ mentioning this requirement. In addition you should add your preferred installation operation system to your docs. Both of these would have been of great help before

Another query:

TASK [bds : moving executable bds 0.99999g to apps directory] ******************
fatal: [localhost]: FAILED! => {"changed": true, "cmd": "mv \"/Users/ayates/.bds\" \"/Users/ayates/bioansible/software/apps/BigDataScript-0.99999g\"", "delta": "0:00:00.023850", "end": "2018-02-19 13:03:55.847116", "msg": "non-zero return code", "rc": 1, "start": "2018-02-19 13:03:55.823266", "stderr": "mv: rename /Users/ayates/.bds to /Users/ayates/bioansible/software/apps/BigDataScript-0.99999g: No such file or directory", "stderr_lines": ["mv: rename /Users/ayates/.bds to /Users/ayates/bioansible/software/apps/BigDataScript-0.99999g: No such file or directory"], "stdout": "", "stdout_lines": []}

I can confirm I have a ~/bioansible/software directory but no apps directory underneath this.

@andrewyatz I worked a bit more on the docs, particular installation part. Also made a little youtube video of following the docs installation live :)

Can you please have another go at it?

Just to reflect on you previous comment about golang and and being undocumented. Those are BigDataScript dependencies and so I assumed that those could be found on BigDataScript install page. It is a bit tricky to try to cover every tools dependency. I've included most, common/crucial dependencies into RNAsik install docs.

Also it appears that you are trying to run/install this on your Mac? As I've mentioned in the docs you'll need fair bit system resources. For the test data set 16 Gb of RAM and at least 4 cpus.

Cheers

MonashBioinformaticsPlatform / RNAsik-pipe

Cannot install bds for RNAsik-pipe #16