search

GMOD / Apollo

Genome annotation editor with a Java Server backend and a Javascript client that runs in a web browser as a JBrowse plugin.
http://genomearchitect.readthedocs.io/
Other
128 stars 85 forks source link

No bin/prepare_refseqs.pl found #1020

Closed jbhangoo closed 8 years ago

jbhangoo commented 8 years ago

The instructions say:

The data generation pipeline is based on the typical jbrowse commands such as prepare-refseqs.pl and flatfile-to-json.pl, and these scripts are automatically copied to a local bin/ directory when you run the setup scripts (e.g. apollo run-local or apollo deploy or install_jbrowse.sh).

If you don't see a bin/ subdirectory containing these scripts after running the setup, check setup.log and check the troubleshooting guide for additional tips or feel free to post the error and setup.log on GitHub or the mailing list._

But after running apollo run-local there is no bin directory created. No setup.log file was created. The troubleshooting guide does not cover this topic. So I am posting this problem to GitHub as suggested in the guide. Thank you!

nathandunn commented 8 years ago

Thanks for reporting.

Are you running a particular version or cloning from the repository directly?

Nathan

On May 5, 2016, at 8:04 AM, Jasbir Bhangoo notifications@github.com wrote:

The instructions say:

_> The data generation pipeline is based on the typical jbrowse commands such as prepare-refseqs.pl and flatfile-to-json.pl, and these scripts are automatically copied to a local bin/ directory when you run the setup scripts (e.g. apollo run-local or apollo deploy or install_jbrowse.sh).

If you don't see a bin/ subdirectory containing these scripts after running the setup, check setup.log and check the troubleshooting guide for additional tips or feel free to post the error and setup.log on GitHub or the mailing list._

But after running apollo run-local there is no bin directory created. No setup.log file was created. The troubleshooting guide does not cover this topic. So I am posting this problem to GitHub as suggested in the guide. Thank you!

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub

jbhangoo commented 8 years ago

Changelog says version 2.0.2

nathandunn commented 8 years ago

@jbhangoo Looks like only "./install_jbrowse.sh" script installs it to the local jbrowse directory.

nathandunn commented 8 years ago

JBrowse is installed just not into the local "./bin" directory, but we can fix that.

cmdcolin commented 8 years ago

I think this is only on master. Doesn't affect 2.0.2. I would suggest getting 2.0.2 with "git checkout 2.0.2", or actually probably just re-clone the git repo and checkout 2.0.2 manually it as it might get confused by the partially installed "state".

Also, kind of random but I personally like installing the jbrowse scripts this way: http://gmod.org/wiki/JBrowse_FAQ#Can_I_install_the_perl_packages_using_cpanm.3F

That way you can just have them in your $PATH

jbhangoo commented 8 years ago

I tried this:

./apollo run-local ./install_jbrowse.sh

But it failed to install Perl prerequisities, saying

bin/cpanm: No such file or directory

No bin directory was created. I also tried that link to install jbrowse using cpnam, but got a similar error of

bash:cpanm: command not found

I searched for an install package that provides cpanm, but found none.

nathandunn commented 8 years ago

Well, that explains stuff as well. CPANM (like CPAN) is a perl installer.

If you are using a mac I would suggest using hombrew. If you have apt-get or yum I would use those. Failing that they provide instructions here:

http://search.cpan.org/~miyagawa/App-cpanminus-1.7040/lib/App/cpanminus.pm http://search.cpan.org/~miyagawa/App-cpanminus-1.7040/lib/App/cpanminus.pm

Nathan

On May 5, 2016, at 9:40 AM, Jasbir Bhangoo notifications@github.com wrote:

I tried this:

./apollo run-local ./install_jbrowse.sh

But it failed to install Perl prerequisities, saying

bin/cpanm: No such file or directory

No bin directory was created. I also tried that link to install jbrowse using cpnam, but got a similar error of

bash:cpanm: command not found

I searched for an install package that provides cpanm, but found none.

— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/GMOD/Apollo/issues/1020#issuecomment-217205545

jbhangoo commented 8 years ago

yum did not have a repository for cpanm. The instructions on the link didn't work until I changed https to http

After installing cpanm, install_jbrowse now gives a different error:

chmod: cannot accessweb-app/jbrowse/bin/cpanm': No such file or directory `

nathandunn commented 8 years ago

It probably didn't install correctly prior. Try this:

Try:

./apollo clean-all (removed old installs) ./apollo run-local

./install_jbrowse.sh

I'm fixing it so that the extra install_jbrowse.sh is not in there, but in the interim this should work.

jbhangoo commented 8 years ago

This gives the same error as before.

I also deleted the whole install, and wiped the databse. Then got the V2.0.2 archive and unzipped it again/ Then I followed this process again, but got to the same error.

nathandunn commented 8 years ago

2 questions:

  1. what does your setup.log file say?
  2. what happens when you type cpanm?

Thanks.

jbhangoo commented 8 years ago

1.

+ chmod +x web-app/jbrowse/bin/cpanm
chmod: cannot access `web-app/jbrowse/bin/cpanm': No such file or directory
+ cd web-app/jbrowse
+ bin/cpanm -v --notest -l ../../extlib/ --installdeps .
./install_jbrowse.sh: line 27: bin/cpanm: No such file or directory
+ bin/cpanm -v --notest -l ../../extlib/ --installdeps .
./install_jbrowse.sh: line 28: bin/cpanm: No such file or directory
+ set -e
+ bin/cpanm -v --notest -l ../../extlib/ --installdeps .
./install_jbrowse.sh: line 30: bin/cpanm: No such file or directory

2..

Usage: cpanm [options] Module [...]

Try `cpanm --help` or `man cpanm` for more options.
nathandunn commented 8 years ago

Sorry this is such a pain. Some more to try:

  1. does web-app/jbrowse exist?
  2. does web-app/jbrowse/bin exist?
  3. does jbrowse-download exist?
  4. does jbrowse-download/bin exist?
  5. does jbrowse-download/bin/cpanm exist?
  6. does jbrowse-download/setup.log exist?

If the answer to 6 is yes, what is in that file?

If the answer to 5 is yes, what happens when you type ant copy-resources?

If we get nowhere from above, if you could provide output for when you ant run-local after typing ant clean-all that would be helpful. Thanks.

jbhangoo commented 8 years ago 
ls web-app/:
annotator  css  images  jbrowse  js  META-INF  translation_tables  WEB-INF

ls web-app/jbrowse/:
plugins

ls jbrowse-download/:
plugins

ant clean-all then ant run-local:

Buildfile: build.xml

BUILD FAILED
Target "run-local" does not exist in the project "Apollo". 

Total time: 0 seconds
nathandunn commented 8 years ago

. . . sorry I meant:

apollo clean-all

apollo run-local

JBrowse doesn't seem to be installing itself correctly. We've fixed some of this in 2.0.3 (which we are currently working on).

jbhangoo commented 8 years ago 
Buildfile: build.xml

copy.apollo.plugin.webapp:

setup-jbrowse:

git.clone:
     [exec] Result: 129

copy.apollo.plugin.jbrowse:
     [echo] copying plugin to jbrowse git ...
     [copy] Copying 190 files to /usr/local/src/Apollo-2.0.2/jbrowse-download/plugins/WebApollo
     [echo] Will be setting up jbrowse
     [copy] Copying 190 files to /usr/local/src/Apollo-2.0.2/web-app/jbrowse

install.jbrowse.perl:

copy-resources:

javac:
    [mkdir] Created dir: /usr/local/src/Apollo-2.0.2/target/classes/gwt
    [javac] Compiling 63 source files to /usr/local/src/Apollo-2.0.2/target/classes/gwt
    [javac] Note: /usr/local/src/Apollo-2.0.2/src/gwt/org/bbop/apollo/gwt/client/ReferenceSequenceOracle.java uses or overrides a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Note: Some input files use unchecked or unsafe operations.
    [javac] Note: Recompile with -Xlint:unchecked for details.
     [copy] Copying 65 files to /usr/local/src/Apollo-2.0.2/target/classes
     [copy] Copied 20 empty directories to 2 empty directories under /usr/local/src/Apollo-2.0.2/target/classes

gwtc:
     [java] Compiling module org.bbop.apollo.gwt.Annotator
     [java]    Compiling 5 permutations
     [java]       Compiling permutation 0...
     [java]       Compiling permutation 1...
     [java]       Compiling permutation 2...
     [java]       Compiling permutation 3...
     [java]       Compiling permutation 4...
     [java]    Compile of permutations succeeded
     [java]    Compilation succeeded -- 47.939s
     [java] Linking into /usr/local/src/Apollo-2.0.2/web-app/annotator
     [java]    Link succeeded
     [java]    Linking succeeded -- 1.051s

BUILD SUCCESSFUL
| Compiling 10 source files

| Compiling 208 source files
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
| Compiling 229 source files

| Compiling 229 source files.....
***
Metrics servlet injected into web.xml
Metrics Admin servlet-mapping (for /metrics/*) injected into web.xml
***
| Running Grails application
Configuring Shiro ...
Shiro Configured
| Server running. Browse to http://localhost:8085/apollo
nathandunn commented 8 years ago

I think you might have a version of git that is too old.

This is the command that is failing (apologize for it doing it silently).

git clone --recursive -b 1.11.6-release --single-branch https://github.com/GMOD/jbrowse jbrowse-download

Apollo-2.0.2% git --version git version 2.6.4 (Apple Git-63)

I tried this with git 1.7.9 and it does not work as the “--single-branch” option is not supported until 1.7.10.

So, I think that the your two options are (assuming that this is the problem) :

1 - upgrade to git 1.7.10+ (or git 2 if possible). Our Ubuntu system is 1.7.9, so it may not work. 2 - remove the line <arg value="--single-branch”/> in the build.xml file at the root level.

Nathan

On May 5, 2016, at 12:54 PM, Jasbir Bhangoo notifications@github.com wrote:

` Buildfile: build.xml

copy.apollo.plugin.webapp:

setup-jbrowse:

git.clone: [exec] Result: 129

copy.apollo.plugin.jbrowse: [echo] copying plugin to jbrowse git ... [copy] Copying 190 files to /usr/local/src/Apollo-2.0.2/jbrowse-download/plugins/WebApollo [echo] Will be setting up jbrowse [copy] Copying 190 files to /usr/local/src/Apollo-2.0.2/web-app/jbrowse

install.jbrowse.perl:

copy-resources:

javac: [mkdir] Created dir: /usr/local/src/Apollo-2.0.2/target/classes/gwt [javac] Compiling 63 source files to /usr/local/src/Apollo-2.0.2/target/classes/gwt [javac] Note: /usr/local/src/Apollo-2.0.2/src/gwt/org/bbop/apollo/gwt/client/ReferenceSequenceOracle.java uses or overrides a deprecated API. [javac] Note: Recompile with -Xlint:deprecation for details. [javac] Note: Some input files use unchecked or unsafe operations. [javac] Note: Recompile with -Xlint:unchecked for details. [copy] Copying 65 files to /usr/local/src/Apollo-2.0.2/target/classes [copy] Copied 20 empty directories to 2 empty directories under /usr/local/src/Apollo-2.0.2/target/classes

gwtc: [java] Compiling module org.bbop.apollo.gwt.Annotator [java] Compiling 5 permutations [java] Compiling permutation 0... [java] Compiling permutation 1... [java] Compiling permutation 2... [java] Compiling permutation 3... [java] Compiling permutation 4... [java] Compile of permutations succeeded [java] Compilation succeeded -- 47.939s [java] Linking into /usr/local/src/Apollo-2.0.2/web-app/annotator [java] Link succeeded [java] Linking succeeded -- 1.051s

BUILD SUCCESSFUL | Compiling 10 source files

| Compiling 208 source files Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. | Compiling 229 source files

| Compiling 229 source files.....

Metrics servlet injected into web.xml Metrics Admin servlet-mapping (for /metrics/*) injected into web.xml

| Running Grails application Configuring Shiro ... Shiro Configured | Server running. Browse to http://localhost:8085/apollo

`

— You are receiving this because you commented. Reply to this email directly or view it on GitHub

jbhangoo commented 8 years ago

Oh, I think I see it now:

[exec] Result: 129 That is probably an error report.

I copied the latest version of Git from github.com and replaced the current one. Now, it gives a different problem. apollo run-local tries to access a file but fails:

jbrowse-download/tests/data/SL2.40_all_rna_seq.v1.bigwig

That file appears to exist, but it is actually a bad symlink to someone's external directory, which doesn't exist on my system.

nathandunn commented 8 years ago

What is the output from ./apollo run-local?

Also, did you install the updated git locally or globally? I think that it will use the system git by default.

jbhangoo commented 8 years ago

Here is the end of the output of apollo run-local:

BUILD FAILED
/usr/local/src/Apollo-2.0.2/build.xml:235: Failed to copy /usr/local/src/Apollo-2.0.2/jbrowse-download/tests/data/SL2.40_all_rna_seq.v1.bigwig to /usr/local/src/Apollo-2.0.2/web-app/jbrowse/tests/data/SL2.40_all_rna_seq.v1.bigwig due to java.io.FileNotFoundException /usr/local/src/Apollo-2.0.2/jbrowse-download/tests/data/SL2.40_all_rna_seq.v1.bigwig (No such file or directory)

... and when I looked for that missing file, I saw...

lrwxrwxrwx 1 root root     51 May  5 14:31 jbrowse-download/tests/data/SL2.40_all_rna_seq.v1.bigwig -> /home/rob/jbrowse/data/SL2.40_all_rna_seq.v1.bigwig
-rw-r--r-- 1 root root 426336 May  5 14:31 jbrowse-download/tests/data/SL2.40ch10_sample.gff3

I installed the new Git system-wide, replacing the previous version completely.

nathandunn commented 8 years ago

I see. Did you confirm the version of git it was running (git --version)?

Would also be helpful to get the full stack-trace from the top.

This is mine:

nathandunn:repositories% wget https://github.com/GMOD/Apollo/archive/2.0.2.tar.gz
--2016-05-05 14:50:38--  https://github.com/GMOD/Apollo/archive/2.0.2.tar.gz
Resolving github.com... 192.30.252.120
Connecting to github.com|192.30.252.120|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/GMOD/Apollo/tar.gz/2.0.2 [following]
--2016-05-05 14:50:39--  https://codeload.github.com/GMOD/Apollo/tar.gz/2.0.2
Resolving codeload.github.com... 192.30.252.162
Connecting to codeload.github.com|192.30.252.162|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-gzip]
Saving to: '2.0.2.tar.gz'

2.0.2.tar.gz                 [ <=>                              ]  77.78M  6.75MB/s    in 13s     

2016-05-05 14:50:54 (5.96 MB/s) - '2.0.2.tar.gz' saved [81558240]

nathandunn:repositories% tar xfz 2.0.2.tar.gz 
nathandunn:repositories% cd Apollo-2.0.2 
nathandunn:Apollo-2.0.2% ./apollo run-local 
Buildfile: /Users/nathandunn/repositories/Apollo-2.0.2/build.xml

copy.apollo.plugin.webapp:

setup-jbrowse:

git.clone:
^C%                                                                                                nathandunn:Apollo-2.0.2% git --version
git version 2.6.4 (Apple Git-63)
nathandunn:Apollo-2.0.2% ./apollo run-local 
Buildfile: /Users/nathandunn/repositories/Apollo-2.0.2/build.xml

copy.apollo.plugin.webapp:

setup-jbrowse:

git.clone:
     [exec] Submodule 'src/FileSaver' (git://github.com/dkasenberg/FileSaver.js.git) registered for path 'src/FileSaver'
     [exec] Submodule 'src/dbind' (git://github.com/rbuels/dbind.git) registered for path 'src/dbind'
     [exec] Submodule 'src/dgrid' (git://github.com/SitePen/dgrid.git) registered for path 'src/dgrid'
     [exec] Submodule 'src/dijit' (https://github.com/dojo/dijit.git) registered for path 'src/dijit'
     [exec] Submodule 'src/dojo' (https://github.com/dojo/dojo.git) registered for path 'src/dojo'
     [exec] Submodule 'src/dojox' (https://github.com/dojo/dojox.git) registered for path 'src/dojox'
     [exec] Submodule 'src/jDataView' (git://github.com/rbuels/jDataView.git) registered for path 'src/jDataView'
     [exec] Submodule 'src/json-schema' (git://github.com/kriszyp/json-schema.git) registered for path 'src/json-schema'
     [exec] Submodule 'src/jszlib' (git://github.com/rbuels/jszlib.git) registered for path 'src/jszlib'
     [exec] Submodule 'src/lazyload' (git://github.com/rbuels/lazyload.git) registered for path 'src/lazyload'
     [exec] Submodule 'src/put-selector' (git://github.com/kriszyp/put-selector.git) registered for path 'src/put-selector'
     [exec] Submodule 'src/util' (https://github.com/dojo/util.git) registered for path 'src/util'
     [exec] Submodule 'src/xstyle' (git://github.com/kriszyp/xstyle.git) registered for path 'src/xstyle'
     [exec] Submodule path 'src/FileSaver': checked out '4f79d8e66662b63df4928bfce769cdafae846600'
     [exec] Submodule path 'src/dbind': checked out '2b70834f3b88227274f158358a9c6f0cd09070ac'
     [exec] Submodule path 'src/dgrid': checked out '77901232965572a7e955ee17168751be9e19fb1d'
     [exec] Submodule path 'src/dijit': checked out '623c3737f5115807a1b1296c0aead855091d31dc'
     [exec] Submodule path 'src/dojo': checked out 'e92c9e4fa5e3e78253dfcc7e1b9b4621f04b180b'
   [exec] Submodule path 'src/dojox': checked out 'e99b2428d87f15d03e6a8ba1f78d15486be75fa2'
     [exec] Submodule path 'src/jDataView': checked out 'db0893d468ba034fadbbb5700d4ed79ab0dc236e'
     [exec] Submodule path 'src/json-schema': checked out 'f56cecb21b176755ad91f58c1ccfa55e5fb93186'
     [exec] Submodule path 'src/jszlib': checked out '18cd14e592652c10f7b494b1e29f891b0a5590a4'
     [exec] Submodule path 'src/lazyload': checked out 'd428749c5400213340d65d6156f43224ee477586'
     [exec] Submodule path 'src/put-selector': checked out 'f64ede2230f913c0ca224dce03106c1edc0ec542'
     [exec] Submodule path 'src/util': checked out '1448ca077eb7c702da4d7130541e74267f5c4daa'
     [exec] Submodule path 'src/xstyle': checked out '5c457b479db2cfd05d8a16a2f092ff82c4a7add6'

copy.apollo.plugin.jbrowse:
     [echo] copying plugin to jbrowse git ...
     [copy] Copying 190 files to /Users/nathandunn/repositories/Apollo-2.0.2/jbrowse-download/plugins/WebApollo
     [echo] Will be setting up jbrowse
     [copy] Copying 10405 files to /Users/nathandunn/repositories/Apollo-2.0.2/web-app/jbrowse

install.jbrowse.perl:
 [exec] Installing Perl prerequisites ... done.

copy-resources:

javac:
    [mkdir] Created dir: /Users/nathandunn/repositories/Apollo-2.0.2/target/classes/gwt
    [javac] Compiling 63 source files to /Users/nathandunn/repositories/Apollo-2.0.2/target/classes/gwt
    [javac] Note: /Users/nathandunn/repositories/Apollo-2.0.2/src/gwt/org/bbop/apollo/gwt/client/ReferenceSequenceOracle.java uses or overrides a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Note: Some input files use unchecked or unsafe operations.
    [javac] Note: Recompile with -Xlint:unchecked for details.
     [copy] Copying 102 files to /Users/nathandunn/repositories/Apollo-2.0.2/target/classes
     [copy] Copied 28 empty directories to 3 empty directories under /Users/nathandunn/repositories/Apollo-2.0.2/target/classes
gwtc:
     [java] Compiling module org.bbop.apollo.gwt.Annotator
     [java]    Compiling 5 permutations
     [java]       Compiling permutation 0...
     [java]       Compiling permutation 1...
     [java]       Compiling permutation 2...
     [java]       Compiling permutation 3...
     [java]       Compiling permutation 4...
     [java]    Compile of permutations succeeded
     [java]    Compilation succeeded -- 39.951s
     [java] Linking into /Users/nathandunn/repositories/Apollo-2.0.2/web-app/annotator
     [java]    Link succeeded
     [java]    Linking succeeded -- 0.333s
BUILD SUCCESSFUL
| Compiling 10 source files

| Compiling 218 source files
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
| Compiling 229 source files

| Compiling 229 source files.....
***
Metrics servlet injected into web.xml
Metrics Admin servlet-mapping (for /metrics/*) injected into web.xml
***
| Running Grails application
| Error Server failed to start for port 8080: Address already in use (Use --stacktrace to see the full trace)

Granted I was already running Apollo, but otherwise it would say "listening on port 8080" or something like that.

jbhangoo commented 8 years ago 
git --version
git version 2.8.0.GIT
Buildfile: build.xml

copy.apollo.plugin.webapp:

setup-jbrowse:

git.clone:
     [exec] Submodule 'src/FileSaver' (git://github.com/dkasenberg/FileSaver.js.git) registered for path 'src/FileSaver'
     [exec] Submodule 'src/dbind' (git://github.com/rbuels/dbind.git) registered for path 'src/dbind'
     [exec] Submodule 'src/dgrid' (git://github.com/SitePen/dgrid.git) registered for path 'src/dgrid'
     [exec] Submodule 'src/dijit' (https://github.com/dojo/dijit.git) registered for path 'src/dijit'
     [exec] Submodule 'src/dojo' (https://github.com/dojo/dojo.git) registered for path 'src/dojo'
     [exec] Submodule 'src/dojox' (https://github.com/dojo/dojox.git) registered for path 'src/dojox'
     [exec] Submodule 'src/jDataView' (git://github.com/rbuels/jDataView.git) registered for path 'src/jDataView'
     [exec] Submodule 'src/json-schema' (git://github.com/kriszyp/json-schema.git) registered for path 'src/json-schema'
     [exec] Submodule 'src/jszlib' (git://github.com/rbuels/jszlib.git) registered for path 'src/jszlib'
     [exec] Submodule 'src/lazyload' (git://github.com/rbuels/lazyload.git) registered for path 'src/lazyload'
     [exec] Submodule 'src/put-selector' (git://github.com/kriszyp/put-selector.git) registered for path 'src/put-selector'
     [exec] Submodule 'src/util' (https://github.com/dojo/util.git) registered for path 'src/util'
     [exec] Submodule 'src/xstyle' (git://github.com/kriszyp/xstyle.git) registered for path 'src/xstyle'
     [exec] Result: 1

copy.apollo.plugin.jbrowse:
     [echo] copying plugin to jbrowse git ...
     [copy] Copying 190 files to /usr/local/src/Apollo-2.0.2/jbrowse-download/plugins/WebApollo
     [echo] Will be setting up jbrowse
     [copy] Copying 1357 files to /usr/local/src/Apollo-2.0.2/web-app/jbrowse

BUILD FAILED
/usr/local/src/Apollo-2.0.2/build.xml:235: Failed to copy /usr/local/src/Apollo-2.0.2/jbrowse-download/tests/data/SL2.40_all_rna_seq.v1.bigwig to /usr/local/src/Apollo-2.0.2/web-app/jbrowse/tests/data/SL2.40_all_rna_seq.v1.bigwig due to java.io.FileNotFoundException /usr/local/src/Apollo-2.0.2/jbrowse-download/tests/data/SL2.40_all_rna_seq.v1.bigwig (No such file or directory)

Total time: 1 minute 10 seconds

ls -l jbrowse-download/tests/data/SL2.40*
lrwxrwxrwx 1 root root     51 May  5 14:31 jbrowse-download/tests/data/SL2.40_all_rna_seq.v1.bigwig -> /home/rob/jbrowse/data/SL2.40_all_rna_seq.v1.bigwig
-rw-r--r-- 1 root root 426336 May  5 14:31 jbrowse-download/tests/data/SL2.40ch10_sample.gff3
nathandunn commented 8 years ago

It looks like its not actually cloning. Let's try this, so we get the full output:

rm -rf jbrowse-download rm -rf web-app/jbrowse git clone --recursive -b 1.11.6-release --single-branch https://github.com/GMOD/jbrowse jbrowse-download

Let me know what this does. This is essentially what ant is doing, but its not telling us why its not cloning properly.

cmdcolin commented 8 years ago

Those bad symlinks were removed in more recent versions of JBrowse due to the fact that they are bad symlinks indeed. Currently Apollo 2.0.2 is pinned to JBrowse 1.11.6 though, so I guess that's why you see this. Still it is odd, since our deployments with 2.0.2 have worked thus far. I might suspect specific perhaps the "ant" version being used could cause you to specifically hit this error (I might check your "apache ant" version number just to see)

And, in any case, you could manually remove that symlink at jbrowse-download/tests/data/SL2.40_all_rna_seq.v1.bigwig

jbhangoo commented 8 years ago 
ant -version
Apache Ant version 1.7.1 compiled on April 26 2010

I manually removed the symlink, but got the same error. Maybe it regenerates the link.

Clone test results:

rm -fr jbrowse-download/
 rm -fr web-app/jbrowse/
 git clone --recursive -b 1.11.6-release --single-branch https://github.com/GMOD/jbrowse jbrowse-download
Cloning into 'jbrowse-download'...
remote: Counting objects: 44140, done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 44140 (delta 0), reused 3 (delta 0), pack-reused 44136
Receiving objects: 100% (44140/44140), 35.01 MiB | 7.95 MiB/s, done.
Resolving deltas: 100% (26492/26492), done.
Checking connectivity... done.
Note: checking out '201d441d352605be5ab446a5c7a16e979c66e912'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

Submodule 'src/FileSaver' (git://github.com/dkasenberg/FileSaver.js.git) registered for path 'src/FileSaver'
Submodule 'src/dbind' (git://github.com/rbuels/dbind.git) registered for path 'src/dbind'
Submodule 'src/dgrid' (git://github.com/SitePen/dgrid.git) registered for path 'src/dgrid'
Submodule 'src/dijit' (https://github.com/dojo/dijit.git) registered for path 'src/dijit'
Submodule 'src/dojo' (https://github.com/dojo/dojo.git) registered for path 'src/dojo'
Submodule 'src/dojox' (https://github.com/dojo/dojox.git) registered for path 'src/dojox'
Submodule 'src/jDataView' (git://github.com/rbuels/jDataView.git) registered for path 'src/jDataView'
Submodule 'src/json-schema' (git://github.com/kriszyp/json-schema.git) registered for path 'src/json-schema'
Submodule 'src/jszlib' (git://github.com/rbuels/jszlib.git) registered for path 'src/jszlib'
Submodule 'src/lazyload' (git://github.com/rbuels/lazyload.git) registered for path 'src/lazyload'
Submodule 'src/put-selector' (git://github.com/kriszyp/put-selector.git) registered for path 'src/put-selector'
Submodule 'src/util' (https://github.com/dojo/util.git) registered for path 'src/util'
Submodule 'src/xstyle' (git://github.com/kriszyp/xstyle.git) registered for path 'src/xstyle'
Cloning into '/usr/local/src/Apollo-2.0.2/jbrowse-download/src/FileSaver'...
fatal: unable to connect to github.com:
github.com[0: 192.30.252.123]: errno=Connection timed out

fatal: clone of 'git://github.com/dkasenberg/FileSaver.js.git' into submodule path '/usr/local/src/Apollo-2.0.2/jbrowse-download/src/FileSaver' failed
nathandunn commented 8 years ago

I think that this is a firewall problem, and its not letting git pull the submodules with the same command.

Here are two things to try:

(1) With the current code base remove the "--recursive" part and fetch them with a separate command:

git clone -b 1.11.6-release --single-branch https://github.com/GMOD/jbrowse jbrowse-download
cd jbrowse-download
git submodule update --init --recursive
cd ..
./apollo run-local

(2) If that doesn't work. clone the master branch where I've fixed some of this stuff and try ./apollo run-local

This is the master branch, which is "stable", but untested.

cmdcolin commented 8 years ago

We have a pre-requisite for ant 1.8 listed here

http://webapollo.readthedocs.io/en/2.0.2/Prerequisites.html

Also, as @nathandunn says above, you are running into blocking the use of the git:// protocol

In general, the git:// protocol issue is another problem that will be fixed after we synchronize with jbrowse 1.12.2, only affects older versions currently

jbhangoo commented 8 years ago

I upgraded ant to V1.9.7, but test (1) didn't work with the same result as before

So I tried (2) and clone the master branch. Reading the git documents seemed like this was the way:

git clone https://github.com/GMOD/Apollo.git
cd Apollo
git checkout master^^

Then ./apollo run-local gave this result which looks very different than before:


which: no gradle in (/root/.sdkman/candidates/grails/current/bin:/usr/local/google-cloud-sdk/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin)
Gradle not found using gradlew
Defining custom 'build' task when using the standard Gradle lifecycle plugins has been deprecated and is scheduled to be removed in Gradle 3.0
Defining custom 'clean' task when using the standard Gradle lifecycle plugins has been deprecated and is scheduled to be removed in Gradle 3.0
final JBrowse settings [git:[url:https://github.com/GMOD/jbrowse, branch:master, alwaysPull:false, alwaysRecheck:false], plugins:[WebApollo:[included:true], NeatHTMLFeatures:[included:true, linearGradient:0], NeatCanvasFeatures:[included:true], RegexSequenceSearch:[included:true], HideTrackLabels:[included:true]]]
final plugins [WebApollo:[included:true], NeatHTMLFeatures:[included:true, linearGradient:0], NeatCanvasFeatures:[included:true], RegexSequenceSearch:[included:true], HideTrackLabels:[included:true]]
Copying apollo plugin
handling jbrowse release [git:[url:https://github.com/GMOD/jbrowse, branch:master, alwaysPull:false, alwaysRecheck:false], plugins:[WebApollo:[included:true], NeatHTMLFeatures:[included:true, linearGradient:0], NeatCanvasFeatures:[included:true], RegexSequenceSearch:[included:true], HideTrackLabels:[included:true]]]
handling jbrowse [git:[url:https://github.com/GMOD/jbrowse, branch:master, alwaysPull:false, alwaysRecheck:false], plugins:[WebApollo:[included:true], NeatHTMLFeatures:[included:true, linearGradient:0], NeatCanvasFeatures:[included:true], RegexSequenceSearch:[included:true], HideTrackLabels:[included:true]]]
:evaluateJBrowseConfigs UP-TO-DATE
:installJBrowse
installing jbrowse [git:[url:https://github.com/GMOD/jbrowse, branch:master, alwaysPull:false, alwaysRecheck:false], plugins:[WebApollo:[included:true], NeatHTMLFeatures:[included:true, linearGradient:0], NeatCanvasFeatures:[included:true], RegexSequenceSearch:[included:true], HideTrackLabels:[included:true]]]
Cloning: https://github.com/GMOD/jbrowse into jbrowse-download.
using bower to install: GMOD/jbrowse#master
:installJBrowse FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':installJBrowse'.
> A problem occurred starting process 'command 'bower''

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 5.281 secs
nathandunn commented 8 years ago

Let's try this:

git config --global url."https://".insteadOf git://

and run bower install again. If that seems to work, you should be able to run the run-local target again without any problems.

From here: http://stackoverflow.com/a/15684898

jbhangoo commented 8 years ago

bower was not found on my system. So I installed that and got

bower ENOENT No bower.json present

So, I went back to V 2.0.2 and it works after entering that git config command -- that is very useful!

The apollo run-local ran without any errors or non-zero return codes. I now have the bin directory and all the contents. So, I am not sure why the master checkout didn't work, but V 2.0,2 works [after cpnam, etc]

The only problem is the next step in the instructions after running the "bin" scripts:

client/apollo/bin/add-webapollo-plugin.pl -i $JBROWSE_DATA_DIR/trackList.json

There is no "bin" direcoty under client/apollo. I can create a new issue for that to keep it separate.

cmdcolin commented 8 years ago

The add-webapollo-plugin command is not needed in Apollo2 versions :)

http://webapollo.readthedocs.io/en/2.0.2/Configure.html#upgrading-existing-instances

If you're able to navigate to http://localhost:8080/apollo (or 8085 if you use "apollo run-local 8085" for example) then you should be done!

jbhangoo commented 8 years ago

I ran the two scripts "prepare_refseqs" and "flatfile_to_json" on the .fas and .gff files in the pyu_data sample [respectively] but no tracks appear when I go to localhost:8085/apollo and log in. There are no errors in the browser console.

nathandunn commented 8 years ago

Did you add an organism and point the directory to where you added the files?

Nathan

On May 6, 2016, at 8:25 AM, Jasbir Bhangoo notifications@github.com wrote:

I ran the two scripts "prepare_refseqs" and "flatfile_to_json" on the .fas and .gff files in the pyu_data sample [respectively] but no tracks appear when I go to localhost:8085/apollo and log in. There are no errors in the browser console.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/GMOD/Apollo/issues/1020#issuecomment-217473262

jbhangoo commented 8 years ago

I was actually just copying and pasting the directions online.

On the webpage itself, there is an Organism tab, but when I try to create a new organism I get an error: Not authorized

I am logged in as the admin user. I checked the permission on the path to the data folder and I have read and execute permissions. I also changed the permission on the data folder itself to give write permission.

jbhangoo commented 8 years ago

UPdate: I just re-logged in as admin, and that fixed it!

nathandunn commented 8 years ago

Excellent. I’ll add that note about the git config as that seems to be a pretty reliable workaround.

Nathan

On May 6, 2016, at 8:40 AM, Jasbir Bhangoo notifications@github.com wrote:

UPdate: I just re-logged in as admin, and that fixed it!

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/GMOD/Apollo/issues/1020#issuecomment-217478289

jbhangoo commented 8 years ago

I would suggest adding information about cpanm too

nathandunn commented 8 years ago

I added it. Thanks.

Nathan

On May 6, 2016, at 9:13 AM, Jasbir Bhangoo notifications@github.com wrote:

I would suggest adding information about cpanm too

— You are receiving this because you modified the open/close state. Reply to this email directly or view it on GitHub https://github.com/GMOD/Apollo/issues/1020#issuecomment-217487355