TritonDataCenter / node-manta

Node.js SDK for Manta
75 stars 54 forks source link

mget -o does not handle multiple object arguments #299

Closed davepacheco closed 7 years ago

davepacheco commented 7 years ago

I created a test directory with three objects:

$ mmkdir /dap/stor/test
$ echo a | mput /dap/stor/test/1
/dap/stor/test/1                                                2B                  
$ echo b | mput /dap/stor/test/2
/dap/stor/test/2                                                2B                  
$ echo c | mput /dap/stor/test/3
/dap/stor/test/3                                                2B                  

You can fetch all of these concatenated by passing them all to "mget":

$ mget /dap/stor/test/1 /dap/stor/test/2 /dap/stor/test/3
a
b
c

and you can normally redirect mget's output to a file with -o:

$ mget -o bar /dap/stor/test/1 
/dap/stor/test/1      [===========================>] 100%       2B                  
$ cat bar
a

but if you combine these, it fails:

$ mget -o foo /dap/stor/test/1 /dap/stor/test/2 /dap/stor/test/3
/dap/stor/test/1      [===========================>] 100%       2B                  
mget: Error: write after end
$ cat foo 
a

I suspect what's going on here is that we pipe the stream for each object into the output file, including its "end" event. We need to not pipe the 'end' event, but instead apply our own .end() to the stream when we've finished with all of the objects.

jclulow commented 7 years ago

I have a change for review @ https://cr.joyent.us/1488

I added tests. The entire test suite passes, though there were no mget tests originally. There are now, though:

 $ make test TEST_FILTER=mget
...
# Running subset of tests matching TEST_FILTER=mget

mget.test.js
✔ setup: create test tree at /mantatest/stor/node-manta-test-mget-6d70d23a
✔ mget TESTDIR/02.data
✔ mget TESTDIR/01.data TESTDIR/02.data TESTDIR/03.data
✔ mget TESTDIR/01.txt TESTDIR/02.txt TESTDIR/03.txt
✔ mget -o TMPFILE TESTDIR/01.txt TESTDIR/02.txt TESTDIR/03.txt
✔ mget TESTDIR/01.txt TESTDIR/02.txt TESTDIR/XX.txt TESTDIR/03.txt
✔ mget -o TMPFILE TESTDIR/01.txt TESTDIR/02.txt TESTDIR/XX.txt TESTDIR/03.txt
✔ cleanup: rm test tree /mantatest/stor/node-manta-test-mget-6d70d23a

OK: 22 assertions (15382ms)