limited to 16 files - Githubissues

matogertel commented 9 years ago

This is the default for streams2 highWaterMark option. I've fixed it by adding {highWaterMark: 100000000} as a parameter. line 32: return through.obj({highWaterMark: 100000000}, function(file, enc, cb) {

jerrysu commented 9 years ago

Hi @matogertel, what are you piping into the gulp-rsync plugin? Do you have a stream with more than 16 globs? I'd like to verify the exact use case that's broken for you. Thanks for letting me know how you fixed it!

matogertel commented 9 years ago

Hi, form http://calv.info/an-introduction-to-nodes-new-streams/ "For streams in objectMode, this will be the number of objects instead of the buffer size.", which means that if you're trying to rsync more than 16 files, it will fail. Also would be good if you could fire an "end" event when rsync finished, so you can do more stuff after that.

jerrysu commented 9 years ago

Thanks for the feedback. I'll add some events.

I'm looking more specifically for the gulp code that you are using that is not working so that I can test the fix. I am able to deploy more than 16 files without any issues with the code as-is, but I suspect that maybe you have a case where you are using several globs?

For example:

  gulp.src(['fooA', 'fooB', 'fooC', 'fooD', ..., '!fooZ'])
    .pipe(rsync({
      hostname: 'example.com',
      destination: '/path/to/site'
    });

Is that what you mean?

matogertel commented 9 years ago

Yea, that's the one I mean. I'm trying to rsync a keystone.js install, so my sources are: ['./models/**/*.js','./routes/**/*.js', './templates/**/*.jade', './updates/**/*.js', './public/**/*.*', 'keystone.js', 'package.json'] The public folder contains several subfolders with a mixture of png, svg, css and js files.

paulmenzel commented 9 years ago

I hit the same issue, just with gulp-rsync 0.0.5.

    $ npm ls gulp-rsync
    dakoshop@0.0.0 /home/joey/myproject
    └── gulp-rsync@0.0.5

I want to copy the dist folder.

    $ ls -Rl dist | wc -l
    321

I use the following code for that.

    […]
    gulp.task('deploy', function() {
      gulp.src('dist/**')
        .pipe(rsync({
    […]

Adding the option @matogertel suggested fixes the issue on my end too.

paulmenzel commented 9 years ago

By the way, I am using Debian GNU/Linux. Maybe the different behavior is related to the operating system.

KutsenkoA commented 9 years ago

+1 Have a some problem. The worst is gulp-rsync just finished without any messages (

wethecode commented 9 years ago

+1 Same here. The fix by @matogertel resolved the issue.

macieklamberski commented 9 years ago

I also have this issue. Having stream with even one glob (return gulp.src(['something/**'])) doesn't work if there are more than 16 files matching the expression. Solution by @matogertel fix this.

chris-burgin commented 9 years ago

Also having this problem. @matogertel provided the fix.

paddingme commented 8 years ago

snip20151119_2

zhiyan commented 8 years ago

@jerrysu Can you provide the option of highWaterMark for fixing this?

errnesto commented 8 years ago

+1 when using an array in gulp.src(['something/**'])

goldenlove commented 8 years ago

+1, debug 4 a long time... then i change to use gulp-scp2 instead...

delijah commented 8 years ago

+1

ebaskoro commented 8 years ago

Thank you. We will have a look at fixing this at the next release. On 26 Apr 2016 8:32 pm, "delijah" notifications@github.com wrote:

+1

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/jerrysu/gulp-rsync/issues/5#issuecomment-214697024

jerrysu / gulp-rsync

limited to 16 files #5