search

iobio / minion

Server wrapper that turns command line tools into web services
MIT License
60 stars 9 forks source link

Local iobio bamstasAliveserver dies with example url #3

Closed ericenns closed 9 years ago

ericenns commented 9 years ago

I followed the instructions on the wiki to setup a local instance of bam.iobio

I installed all the dependencies and running from the tools from the command line work just fine.

But when trying to load the example bam url, the bamstatsAlive server fails with the following error.

prog process exited with code 1

events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: socket hang up
    at createHangUpError (http.js:1476:15)
    at Socket.socketOnEnd [as onend] (http.js:1572:23)
    at Socket.g (events.js:180:16)
    at Socket.emit (events.js:117:20)
    at _stream_readable.js:943:16
    at process._tickCallback (node.js:419:13)

And if I look at the samtools server output it hangs for a long time until finally the prog returns.

I think this error is basically that the samtools server connection is lost and I am at a loss on what to do.

chmille4 commented 9 years ago

Are all the executables (bamtools, bamstatsalive, etc) in the iobio/bin directory?

If so, you can test the different servers to see if they are all working

You can do this by curling an iobio command url on your local servers. Here is one for the samtools server using the default port (8060). 

curl "http://0.0.0.0:8060?cmd=view%20-h%20http://s3.amazonaws.com/1000genomes/data/HG04141/alignment/HG04141.mapped.ILLUMINA.bwa.BEB.low_coverage.20130415.bam%201:3000000-3100000"

You should get back a sam file. If that worked test the bamstatsalive server with this:

curl "http://0.0.0.0:7100?cmd=-u%203000%20ws%253A%252F%252F0.0.0.0%253A8060%253Fcmd%253D%252520view%252520-b%252520http%253A%252F%252Fs3.amazonaws.com%252F1000genomes%252Fdata%252FHG04141%252Falignment%252FHG04141.mapped.ILLUMINA.bwa.BEB.low_coverage.20130415.bam%2525201%253A5878601-5918601%2526encoding%253Dbinary"

Make sure your local samtools server is also running as the above command pipes output from the samtools server into the bamstatsalive server for analysis. Here you should get back some JSON that starts off something like this {"both_mates_mapped":2750,"duplicates"

Let me know how that goes.

ericenns commented 9 years ago

Yes, I symlinked all executables into the iobio/bin directory.

The command for the samtools server works correctly.

When I run the command for the bamstatsalive server, I get the following output.

$ curl "http://0.0.0.0:7100?cmd=-u%203000%20ws%253A%252F%252F0.0.0.0%253A8060%253Fcmd%253D%252520view%252520-b%252520http%253A%252F%252Fs3.amazonaws.com%252F1000genomes%252Fdata%252FHG04141%252Falignment%252FHG04141.mapped.ILLUMINA.bwa.BEB.low_coverage.20130415.bam%2525201%253A5878601-5918601%2526encoding%253Dbinary"
{"status":"error", "message":"Cannot open the specified file"}
ericenns commented 9 years ago

Here is the output for the bamstatsalive server

$ node services/bamstatsAliveServer.js 
iobio server started on port 7100
req.query.cmd = -u 3000 ws%3A%2F%2F0.0.0.0%3A8060%3Fcmd%3D%2520view%2520-b%2520http%3A%2F%2Fs3.amazonaws.com%2F1000genomes%2Fdata%2FHG04141%2Falignment%2FHG04141.mapped.ILLUMINA.bwa.BEB.low_coverage.20130415.bam%25201%3A5878601-5918601%26encoding%3Dbinary
params = {"cmd":"-u 3000 ws%3A%2F%2F0.0.0.0%3A8060%3Fcmd%3D%2520view%2520-b%2520http%3A%2F%2Fs3.amazonaws.com%2F1000genomes%2Fdata%2FHG04141%2Falignment%2FHG04141.mapped.ILLUMINA.bwa.BEB.low_coverage.20130415.bam%25201%3A5878601-5918601%26encoding%3Dbinary","protocol":"http"}
cmad = undefined
raw args = -u,3000,ws%3A%2F%2F0.0.0.0%3A8060%3Fcmd%3D%2520view%2520-b%2520http%3A%2F%2Fs3.amazonaws.com%2F1000genomes%2Fdata%2FHG04141%2Falignment%2FHG04141.mapped.ILLUMINA.bwa.BEB.low_coverage.20130415.bam%25201%3A5878601-5918601%26encoding%3Dbinary
mArg = ws%3A%2F%2F0.0.0.0%3A8060%3Fcmd%3D%2520view%2520-b%2520http%3A%2F%2Fs3.amazonaws.com%2F1000genomes%2Fdata%2FHG04141%2Falignment%2FHG04141.mapped.ILLUMINA.bwa.BEB.low_coverage.20130415.bam%25201%3A5878601-5918601%26encoding%3Dbinary
command: /home/ericenns/iobio/bin/bamstatsAlive -u,3000
http request
ws://0.0.0.0:8060?cmd=%20view%20-b%20http://s3.amazonaws.com/1000genomes/data/HG04141/alignment/HG04141.mapped.ILLUMINA.bwa.BEB.low_coverage.20130415.bam%201:5878601-5918601&encoding=binary
http://0.0.0.0:8060?cmd=%20view%20-b%20http://s3.amazonaws.com/1000genomes/data/HG04141/alignment/HG04141.mapped.ILLUMINA.bwa.BEB.low_coverage.20130415.bam%201:5878601-5918601&encoding=binary
prog process exited with code 1

events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: This socket is closed.
    at Socket._write (net.js:637:19)
    at doWrite (_stream_writable.js:225:10)
    at writeOrBuffer (_stream_writable.js:215:5)
    at Socket.Writable.write (_stream_writable.js:182:11)
    at Socket.write (net.js:615:40)
    at IncomingMessage.<anonymous> (/home/ericenns/iobio/minion.js:206:27)
    at IncomingMessage.emit (events.js:95:17)
    at IncomingMessage.<anonymous> (_stream_readable.js:764:14)
    at IncomingMessage.emit (events.js:92:17)
    at emitReadable_ (_stream_readable.js:426:10)
tonydisera commented 9 years ago

I wonder if port 8060 is restricted.

On Nov 28, 2014, at 1:52 PM, ericenns notifications@github.com wrote:

Yes, I symlinked all executables into the iobio/bin directory.

The command for the samtools server works correctly.

When I run the command for the bamstatsalive server, I get the following output.

$ curl "http://0.0.0.0:7100?cmd=-u%203000%20ws%253A%252F%252F0.0.0.0%253A8060%253Fcmd%253D%252520view%252520-b%252520http%253A%252F%252Fs3.amazonaws.com%252F1000genomes%252Fdata%252FHG04141%252Falignment%252FHG04141.mapped.ILLUMINA.bwa.BEB.low_coverage.20130415.bam%2525201%253A5878601-5918601%2526encoding%253Dbinary" {"status":"error", "message":"Cannot open the specified file"} — Reply to this email directly or view it on GitHub https://github.com/chmille4/iobio/issues/3#issuecomment-64927764.

ericenns commented 9 years ago

That wouldn't really make sense that port 8060 is restricted or that selinux is blocking the request, as the samtools server gets the request. I can see it in the logs, but bamstatsalive server seems to lose it's connection to samtools server. I can't unfortunately provide any more logs at the moment as I am back at work. But I can provide samtools server log on Monday if that would be helpful.

Thanks for all the help so far though.

Also note I am using Node v 0.10.33, and on Arch Linux. But my co-worker has been trying to get it work also and he was using Ubuntu 14.10 and experienced the same issues.

chmille4 commented 9 years ago

It looks like Yi has managed to recreate the problem and put it in a docker container. I'm going to load it up tomorrow and take a look.

tonydisera commented 9 years ago

Good to hear. Let me know if I can help.

Tony

On Friday, November 28, 2014, Chase Miller notifications@github.com wrote:

It looks like Yi has managed to recreate the problem and put it in a docker container. I'm going to load it up tomorrow and take a look.

— Reply to this email directly or view it on GitHub https://github.com/chmille4/iobio/issues/3#issuecomment-64935291.

yiq commented 9 years ago

@ericenns

It turns out that there was a configuration switch introduced to bamtools two years ago that we all forgot about, which fixes an issue regarding piping data in a nodejs environment. During the step of installing bamtools, instead of running "cmake ..", please run "cmake -DEnableNodeJS=true ..". The rest of the instruction should still be good. Let us know if this works for you.

ericenns commented 9 years ago

@yiq

Building bamtools with the -DEnableNodeJS=true, resolves this issue. Thanks again for your quick help.