ssh2 exec cat large file

[luctorQ]

Hi, I've problem with large size over 80MB. I'm not pretty sure that my test is correct but it seems that there is some problem handling buffers

I try issue simple cat on access.log (about 80MB) file and got strange behaviour. It's repeatable issue. Simply I try command: cat with line numbers, on connection created with user , password authentication. There are few things I've observed:

1. output_remote1 file is enormously large over 10GB , and inside I'can see repeated lines with rotated line numbers(first column) line no 2 after 283

   282  85.124.22.177 - - [01/Dec/2007:09:11:41 -0800] "GET /downloads/wso2-wsf-ruby-src-1.0.0-beta.tar.gz.md5 HTTP/1.0" 404 410 "http://dist.wso2.org/products/wsf/ruby/1.0.0-beta/" "Mozilla/5.0 (compatible; heritrix/1.12.0 +http://seekda.com)"
   283    66.249.70.72 - - [01/Dec/2007:09:12:25 -0800] "GET /maven2/org/wso2/wsas/wso2wsas-admin/SNAPSHOT/wso2wsas-admin-20070813.114027-9.pom.sha1 HTTP/1.1" 200 40 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
   222.165.132.170 - - [01/Dec/2007:06:51:35 -0800] "GET / HTTP/1.0" 302 - "-" "check_http/1.96 (nagios-plugins 1.4.5)"
   2  66.249.70.72 - - [01/Dec/2007:06:51:56 -0800] "GET /maven2/org/wso2/wsas/wso2wsas-admin/SNAPSHOT/wso2wsas-admin-20071025.114334-88.jar.md5 HTTP/1.1" 200 32 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
   3  66.249.70.72 - - [01/Dec/2007:06:52:26 -0800] "GET /maven2/org/wso2/wsas/wso2wsas-admin/SNAPSHOT/wso2wsas-admin-20071010.114839-73.jar.md5 HTTP/1.1" 200 32 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
   

2. In ssh log auth.log I can observe entry like this: fatal: buffer_append_space: len 1048608 not supported
3. I tried different node and ssh2 versions - all the same problem
4. when pty param set false strange log line rotating in output is on line numbers over 10000

   Test code

var fs = require('fs');
var ssh2 = require('ssh2');
var conn = new ssh2();
conn.on('ready', function () {
    conn.exec("cat -n access.log2", {
        pty: true
    }, function (err, stream) {
        stream.pipe(fs.createWriteStream('./output_remote1'));
        if (err) throw err;
        stream.on('end',function () {
            console.log('Stream :: EOF');
        }).on('exit',function (code, signal) {
            console.log('1 Stream :: exit :: code: ' + code + ', signal: ' + signal);
        }).on('close',function () {
            console.log('Stream :: close');
            conn.end();
        }).on('data', function (data, extended) {
            if (extended === 'stderr') {
                console.log('STDERR' + data)
            }
        })
        /*
         }).stderr.on('data', function(data) {
         console.log('STDERR: ' + data);
         })
         */
    })
conn.connect(connectionConfig.params);

environment params:

• ssh: OpenSSH_6.1p1 Debian-4, OpenSSL 1.0.1c 10 May 2012
• Node: windows 7 pro , 0.8.7 (tried also newest node - same effect)
• ssh2 version: npm list

  └─┬ ssh2@0.2.25
  ├── asn1@0.1.11
  └── streamsearch@0.1.2
  

[mscdex]

Try the master branch.

[luctorQ]

Now it works for my test, thanx for blazing fast response