search

ChristopherWilks / snaptron

fast webservices based query tool for large sets of genomic features
Other
25 stars 7 forks source link

pcat command not found inside docker container #13

Open klugem opened 3 years ago

klugem commented 3 years ago

The deploy_docker.sh script causes some errors as the pcat command is not found e.g.:

/deploy/srav3h/deploy//build_sqlite_db.sh: line 9: pcat: command not found

The command is used in deploy/build_sqlite_db.sh

podman run -v /usr/local/storage/snaptron/:/deploy --rm -it --entrypoint="/bin/bash" quay.io/broadsword/snaptron root@b8de4957a354:/usr/src# pcat bash: pcat: command not found

ChristopherWilks commented 3 years ago

@klugem thanks for the detailed report, I've switched to gzip -cd instead of pcat which is a custom wrapper around pigz (which I could have used but wanted a quick fix for this).

check for the latest image (1.0.1): https://quay.io/repository/broadsword/snaptron?tab=tags

Let me know if works/doesn't work for you now. Thanks

klugem commented 3 years ago

@ChristopherWilks Thanks for the fix. I think deploy works now.

But when I tried to start the server there is another problem as /run/lock/ is missing. However, I'm not sure if this is podman-related. Unfortunately, I can't test it with docker.

I tried to start it with: podman run --rm -p 1556:1556 -i -t --name snaptron --volume /path/to/deploy/:/deploy quay.io/broadsword/snaptron run srav2

+ DEPLOY_DIR=/deploy
+ '[' '!' -d /deploy ']'
+ compilation=srav2
+ '[' -z srav2 ']'
+ '[' run == deploy ']'
+ '[' run == run ']'
+ ln -fs //deploy/srav2/instances/proxy.conf /etc/apache2/mods-available/
+ ln -fs /etc/apache2/mods-available/proxy.conf /etc/apache2/mods-enabled/
+ ln -fs ../mods-available/proxy.load /etc/apache2/mods-enabled/
+ ln -fs ../mods-available/slotmem_shm.load /etc/apache2/mods-enabled/
+ ln -fs ../mods-available/proxy_balancer.conf /etc/apache2/mods-enabled/
+ ln -fs ../mods-available/proxy_balancer.load /etc/apache2/mods-enabled/
+ ln -fs ../mods-available/lbmethod_byrequests.load /etc/apache2/mods-enabled/
+ ln -fs ../mods-available/proxy_http.load /etc/apache2/mods-enabled/
+ /etc/init.d/apache2 start
[FAIL] Starting Apache httpd web server: apache2 failed!
[warn] The apache2 configtest failed. ... (warning).
Output of config test was:
/usr/sbin/apache2ctl: 99: ulimit: error setting limit (Operation not permitted)
Setting ulimit failed. See README.Debian for more information.
mktemp: failed to create directory via template '/var/lock/apache2.XXXXXXXXXX': No such file or directory
chmod: missing operand after '755'
Try 'chmod --help' for more information.
+ cd //deploy/srav2
+ python //deploy/srav2/snaptron_server --no-daemon
Logging to syslog
[24/Jun/2021:13:22:18] ENGINE Bus STARTING
[24/Jun/2021:13:22:18] ENGINE Started monitor thread 'Autoreloader'.
[24/Jun/2021:13:22:18] ENGINE Started monitor thread '_TimeoutMonitor'.
[24/Jun/2021:13:22:18] ENGINE Serving on 0.0.0.0:1556
[24/Jun/2021:13:22:18] ENGINE Bus STARTED

mktemp: failed to create directory via template '/var/lock/apache2.XXXXXXXXXX': No such file or directory

/var/lock is linked to /run/lock, which does not exist. When I fixed that (added RUN mkdir /run/lock in a Dockerfile based on your image), the apache2 server starts and I got results on my curl requests. :)

Another point in the README that should be fixed: curl http://localhost:1587/snaptron?regions:CD99 does not work. I think correct would be regions=CD99

klugem commented 3 years ago

@ChristopherWilks I also tried to deploy srav3h and srav1m but I can't get it running.

Content of the data folder for srav1m:

-rw-r--r-- 1 root root 763G Jul  2 02:27 exons.sqlite
-rw-r--r-- 1 root root  45M Oct  3  2017 gencode.v25.annotation.gff3.gz
-rw-r--r-- 1 root root  23M Jan 20  2019 gencode.vM15.basic.annotation.gff3.gz
-rw-r--r-- 1 root root  85G Jul  1 14:56 genes.sqlite
-rw-r--r-- 1 root root 311G Jul  1 13:45 junctions.sqlite
lrwxrwxrwx 1 root root   13 Jul  2 02:27 junctions_uncompressed.bgz -> junctions.bgz
lrwxrwxrwx 1 root root   17 Jul  2 02:27 junctions_uncompressed.bgz.tbi -> junctions.bgz.tbi
-rw-r--r-- 1 root root  11M Jun 20  2020 refseq_transcripts_by_hgvs.tsv
-rw-r--r-- 1 root root 6.1M Jun 20  2020 ucsc_known_canonical_transcript.tsv

Content of the data folder for srav3h:

-rw-r--r-- 1 root root 943355527168 Jun 26 06:01 exons.sqlite
-rw-r--r-- 1 root root  86702944256 Jun 25 14:31 genes.sqlite
-rw-r--r-- 1 root root 360285155328 Jun 25 13:24 junctions.sqlite
lrwxrwxrwx 1 root root           13 Jun 26 06:01 junctions_uncompressed.bgz -> junctions.bgz
lrwxrwxrwx 1 root root           17 Jun 26 06:01 junctions_uncompressed.bgz.tbi -> junctions.bgz.tbi

Error:

QUERY_STRING regions=CD99
Running: python .//snaptron.py regions=CD99
in _wait, found an error message
.//snaptron.py:Traceback (most recent call last):
.//snaptron.py:  File ".//snaptron.py", line 61, in <module>
.//snaptron.py:    import snample
.//snaptron.py:  File "./snample.py", line 86, in <module>
.//snaptron.py:    std_reader = DirectoryReader.open(SimpleFSDirectory(Paths.get(sc.LUCENE_STD_SAMPLE_DB)))
.//snaptron.py:lucene.JavaError: <super: <class 'JavaError'>, <JavaError object>>
.//snaptron.py:    Java stacktrace:
.//snaptron.py:org.apache.lucene.index.IndexNotFoundException: no segments* file found in SimpleFSDirectory@/deploy/srav1m/downloaded_data/lucene_full_standard lockFactory=org.apache.lucene.store.NativeFSLockFactory@78123e82: files: []
.//snaptron.py: at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:687)
.//snaptron.py: at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:77)
.//snaptron.py: at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
.//snaptron.py:
.//snaptron.py:
[02/Jul/2021:05:45:52] ENGINE Exception('.//snaptron.py failed on Traceback (most recent call last):\n  File ".//snaptron.py", line 61, in <module>\n    import snample\n  File "./snample.py", line 86, in <module>\n    std_reader = DirectoryReader.open(SimpleFSDirectory(Paths.get(sc.LUCENE_STD_SAMPLE_DB)))\nlucene.JavaError: <super: <class \'JavaError\'>, <JavaError object>>\n    Java stacktrace:\norg.apache.lucene.index.IndexNotFoundException: no segments* file found in SimpleFSDirectory@/deploy/srav1m/downloaded_data/lucene_full_standard lockFactory=org.apache.lucene.store.NativeFSLockFactory@78123e82: files: []\n\tat org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:687)\n\tat org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:77)\n\tat org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)\n\n',)
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/cherrypy/wsgiserver/wsgiserver2.py", line 1302, in communicate
    req.respond()
  File "/usr/local/lib/python2.7/dist-packages/cherrypy/wsgiserver/wsgiserver2.py", line 831, in respond
    self.server.gateway(self).respond()
  File "/usr/local/lib/python2.7/dist-packages/cherrypy/wsgiserver/wsgiserver2.py", line 2117, in respond
    for chunk in response:
  File "/deploy/srav1m/snaptronws.py", line 110, in next
    raise Exception("%s failed on %s" % (sc.SNAPTRON_APP, self.stderr))
Exception: .//snaptron.py failed on Traceback (most recent call last):
  File ".//snaptron.py", line 61, in <module>
    import snample
  File "./snample.py", line 86, in <module>
    std_reader = DirectoryReader.open(SimpleFSDirectory(Paths.get(sc.LUCENE_STD_SAMPLE_DB)))
lucene.JavaError: <super: <class 'JavaError'>, <JavaError object>>
    Java stacktrace:
org.apache.lucene.index.IndexNotFoundException: no segments* file found in SimpleFSDirectory@/deploy/srav1m/downloaded_data/lucene_full_standard lockFactory=org.apache.lucene.store.NativeFSLockFactory@78123e82: files: []
    at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:687)
    at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:77)
    at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)