teambition / merge2

Merge multiple streams into one stream in sequence or parallel (~119M/month downloads).
MIT License
170 stars 14 forks source link

Running streams in sequential #24

Closed mmajcica closed 6 years ago

mmajcica commented 6 years ago

I'm trying to execute a couple of streams sequential, run first stream, wait for it to complete, then run the next stream. The results I'm getting are not what I do expect.

Following is my gulp task:

gulp.task('build', ['clean', 'compile'], function () {
    var extension = gulp.src(['README.md', 'LICENSE.txt', 'images/**/*', '!images/**/*.pdn', 'vss-extension.json'], { base: '.' })
        .pipe(debug({ title: 'Copying extension files:' }))
        .pipe(gulp.dest(_buildRoot));

    const getDirectories = fs.readdirSync(_tasksRoot).filter(function (file) {
        return fs.statSync(_tasksRoot + '/' + file).isDirectory();
    });

    const stream = merge2(extension, { end: false })

    getDirectories.forEach(task => {
        var taskPath = path.join(_tasksRoot, task);
        var taskBuild = path.join(_buildRoot, task)

        gutil.log(taskPath);

        var taskFiles = path.join(task, '**/*');

        gutil.log(taskFiles);

        stream.add(gulp.src([taskFiles, '!**/*.ts', '!**/package.json'])
            .pipe(debug({ title: 'Task files copy for ' + task }))
            .pipe(gulp.dest(taskBuild)));

        //getExternalModules();
    });

    return stream.end();
});

The output I do get in the console is the following:

C:\Repos\tfs-ansible-tower-extension>gulp build
[15:22:56] Using gulpfile C:\Repos\tfs-ansible-tower-extension\gulpfile.js
[15:22:56] Starting 'clean'...
[15:22:56] Finished 'clean' after 28 ms
[15:22:56] Starting 'compile'...
[15:22:56] Compiling playbook-task
[15:22:56] Compiling run-playbook
[15:22:56] Finished 'compile' after 244 ms
[15:22:56] Starting 'build'...
[15:22:56] C:\Repos\tfs-ansible-tower-extension\Tasks\playbook-task
[15:22:56] playbook-task\**\*
[15:22:56] C:\Repos\tfs-ansible-tower-extension\Tasks\run-playbook
[15:22:56] run-playbook\**\*
[15:22:56] Finished 'build' after 7.64 ms
[15:22:56] Task files copy for playbook-task playbook-task\AnsibleTower.js
[15:22:56] Copying extension files: README.md
[15:22:56] Task files copy for run-playbook run-playbook\icon.png
[15:23:00] Task files copy for playbook-task playbook-task\icon.png
[15:23:00] Copying extension files: LICENSE.txt
[15:23:00] Task files copy for run-playbook run-playbook\task.json
[15:23:00] Task files copy for run-playbook 2 items
[15:23:00] Task files copy for playbook-task playbook-task\task.js
[15:23:00] Copying extension files: images\logo.png
[15:23:00] Task files copy for playbook-task playbook-task\task.json
[15:23:00] Task files copy for playbook-task playbook-task\test
[15:23:00] Task files copy for playbook-task 5 items
[15:23:00] Copying extension files: vss-extension.json
[15:23:00] Copying extension files: 4 items

Now this is not looking to be synced. Am I misunderstanding the usage of this library or not using it correctly? How do I get each of my copy operations executed sequentially one after another?

Thanks

zensh commented 6 years ago

Hi, you should use const stream = merge2(...extension, { end: false }).

merge2(stream1, stream2, ..., streamN, options) will run in sequence. merge2([stream1, stream2, ..., streamN], options) will run in parallel.

mmajcica commented 6 years ago

Is the stream.add method adding items to the collection for a sequential run or parallel?

zensh commented 6 years ago

@mmajcica

mergedStream.add(stream)
mergedStream.add(stream1, [stream2, stream3, ...], ...)

stream.add run as same as merge2 function.

streams in array will run in parallel, otherwise in sequence.