TritonDataCenter / node-manta

Node.js SDK for Manta
75 stars 54 forks source link

mfind --parallel does nothing (is never used) #300

Closed arekinath closed 6 years ago

arekinath commented 7 years ago

mfind currently does parallel operations (using a vasync.barrier), but never uses the value of the -p/--parallel switch to limit their concurrency.

bahamas10 commented 6 years ago

-p/--parallel is currently being used by client.ftw which mfind makes heavy use of. I verified this by manually printing out the value set before the queue was created.

# gd
diff --git a/lib/client.js b/lib/client.js
index ba0495a..837f5c3 100644
--- a/lib/client.js
+++ b/lib/client.js
@@ -1016,6 +1016,7 @@ MantaClient.prototype.ftw = function ftw(p, opts, cb) {

     function run() {
         var barrier = vasync.barrier();
+        console.log(opts.parallel);
         var q = new Queue({
             limit: opts.parallel || 50,
             worker: function (_opts, _cb) {

and then

# ./bin/mfind ~~/stor/manta-sync/lib
50
/dave.eddy@joyent.com/stor/manta-sync/lib/file.js
/dave.eddy@joyent.com/stor/manta-sync/lib/finder.js
/dave.eddy@joyent.com/stor/manta-sync/lib/glob.js
/dave.eddy@joyent.com/stor/manta-sync/lib/localfile.js
/dave.eddy@joyent.com/stor/manta-sync/lib/mantafile.js
# ./bin/mfind -p2 ~~/stor/manta-sync/lib
2
/dave.eddy@joyent.com/stor/manta-sync/lib/file.js
/dave.eddy@joyent.com/stor/manta-sync/lib/finder.js
/dave.eddy@joyent.com/stor/manta-sync/lib/glob.js
/dave.eddy@joyent.com/stor/manta-sync/lib/localfile.js
/dave.eddy@joyent.com/stor/manta-sync/lib/mantafile.js