No 'error' event

[gchauvet]

Hi,

This small unit test show that JSONStream don't return an "error" event :

var assert = require('assert');
var JSONStream = require('JSONStream');
var stream = require('stream');
var es = require('event-stream');

describe('JSONStream', function() {

  it('With unvalid stream', function(done) {

        var s = new stream.Readable();
        s._read = function noop() {};
        s.push('["foo":bar[');
        s.push(null);

        var parser = JSONStream.parse('*');
        parser
        .on('error', assert.ifError)
        .on('end', done);
        s.pipe(parser);
  });
});

returns me :

mocha --recursive

  JSONStream
    1) With unvalid stream

  0 passing (11ms)
  1 failing

  1) JSONStream With unvalid stream:
     Uncaught Error: Unexpected COLON(":") in state COMMA
      at Parser.proto.parseError (/home/gchauvet/Documents/jsonstreamtest/node_modules/JSONStream/node_modules/jsonparse/jsonparse.js:309:16)
      at Parser.proto.onToken [as _onToken] (/home/gchauvet/Documents/jsonstreamtest/node_modules/JSONStream/node_modules/jsonparse/jsonparse.js:394:12)
      at Parser.parser.onToken (/home/gchauvet/Documents/jsonstreamtest/node_modules/JSONStream/index.js:87:12)
      at Parser.proto.write (/home/gchauvet/Documents/jsonstreamtest/node_modules/JSONStream/node_modules/jsonparse/jsonparse.js:99:34)
      at Stream.<anonymous> (/home/gchauvet/Documents/jsonstreamtest/node_modules/JSONStream/index.js:21:12)
      at Stream.stream.write (/home/gchauvet/Documents/jsonstreamtest/node_modules/JSONStream/node_modules/through/index.js:26:11)
      at write (_stream_readable.js:583:24)
      at flow (_stream_readable.js:592:7)
      at _stream_readable.js:560:7
      at process._tickCallback (node.js:415:13)

[dominictarr]

hmm, this might need a try catch some where - would be happy to merge a PR for this

[gchauvet]

Yes, I work on a PR for this

[oleg-koval]

@gchauvet have You resolved this problem? thanks

[gchauvet]

Hi,

My apologies, I'm busy on another projects... The "jsonparse" library works synchronously (uses of 'throw' instructions), but theses exceptions are not catched by JSONStream who works in asynchronous mode (streaming).

[vanthome]

+1

I think the error handling is in place here:

https://github.com/dominictarr/JSONStream/blob/master/index.js#L108

and it emits an error but I can confirm that it does not work.

[dominictarr]

@vanthome can you make short runnable script to demonstrate this?

[gchauvet]

Note: jsonparse doesn't export "toknam" function

[smasher164]

I am also having this issue. @gchauvet just wondering if your fork emits the correct "error" event.

[gchauvet]

@smasher164 Did you have tested my fork ? In my use cases yes, but I waiting feedbacks from other users

[smasher164]

@gchauvet Sorry for the late response. I did test your fork, and I don't think an 'error' event is being emitted. Here is a test with an incorrect input for the parse method.

.pipe(JSONStream.parse('slkfsldf'))
.on('error', function(error) {
    console.log(error);
})

The program halts there immediately, without anything logged.

[gchauvet]

Okay, I've an error in my patch, I fixed it. Can you retry ? Note: If you have a fully corrupted stream, you will get multiple error events. (I've not tested JSONStream.parse with unvalid parameters)

[smasher164]

Ahh, that makes more sense now. I retried it with the latest commit, but as expected, I still don't see an error logged. But what is a test-case where the error is caused by JSONStream and not something else?

[gchauvet]

Take a look to issues.js. This test case contains only issue #66 (bad stream) for the moment.

Your case with unvalid parameters to JSONStream.parse seems not tested in other UT.

[smasher164]

Sorry about that! Just tested your commit with a gzip instead, and the error is in fact logged! :+1:

request.get("http://nodestreams.com/input/people.json.gz")
    .pipe(JSONStream.parse("*"))
    .on('error', function(error) {
        console.log(error);
    });

returns

[Error: Invalid JSON (Unexpected "\u001f" at position 0 in state START)]
[Error: Invalid JSON (Unexpected "‹" at position 1 in state START)]
[Error: Invalid JSON (Unexpected "\b" at position 2 in state START)]
[Error: Invalid JSON (Unexpected "\b" at position 3 in state START)]
[Error: Invalid JSON (Unexpected "Œ" at position 4 in state START)]
[Error: Invalid JSON (Unexpected "û" at position 5 in state START)]
[Error: Invalid JSON (Unexpected "~" at position 6 in state START)]
[Error: Invalid JSON (Unexpected "R" at position 7 in state START)]
[Error: Invalid JSON (Unexpected "\u0000" at position 8 in state START)]
[Error: Invalid JSON (Unexpected "\u0003" at position 9 in state START)]
[Error: Invalid JSON (Unexpected "m" at position 11 in state TRUE1)]
[Error: Invalid JSON (Unexpected "p" at position 12 in state TRUE1)]
[Error: Invalid JSON (Unexpected "." at position 13 in state TRUE1)]
[Error: Invalid JSON (Unexpected "j" at position 14 in state TRUE1)]
[Error: Invalid JSON (Unexpected "s" at position 15 in state TRUE1)]
[Error: Invalid JSON (Unexpected "o" at position 16 in state TRUE1)]
[Error: Invalid JSON (Unexpected "n" at position 17 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\u0000" at position 18 in state TRUE1)]
[Error: Invalid JSON (Unexpected "u" at position 19 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Ð" at position 20 in state TRUE1)]
[Error: Invalid JSON (Unexpected "M" at position 21 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\u000b" at position 22 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Â" at position 23 in state TRUE1)]
[Error: Invalid JSON (Unexpected "0" at position 24 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\f" at position 25 in state TRUE1)]
[Error: Invalid JSON (Unexpected "€" at position 26 in state TRUE1)]
[Error: Invalid JSON (Unexpected "á" at position 27 in state TRUE1)]
[Error: Invalid JSON (Unexpected "»" at position 28 in state TRUE1)]
[Error: Invalid JSON (Unexpected "¿" at position 29 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\"" at position 30 in state TRUE1)]
[Error: Invalid JSON (Unexpected "ô" at position 31 in state TRUE1)]
[Error: Invalid JSON (Unexpected "<" at position 32 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Ü" at position 33 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Ý" at position 34 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Û" at position 35 in state TRUE1)]
[Error: Invalid JSON (Unexpected "œ" at position 36 in state TRUE1)]
[Error: Invalid JSON (Unexpected "x" at position 37 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\u0010" at position 38 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\u0019" at position 39 in state TRUE1)]
[Error: Invalid JSON (Unexpected "¢" at position 40 in state TRUE1)]
[Error: Invalid JSON (Unexpected "(" at position 41 in state TRUE1)]
[Error: Invalid JSON (Unexpected "ˆ" at position 42 in state TRUE1)]
[Error: Invalid JSON (Unexpected "x" at position 43 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\b" at position 44 in state TRUE1)]
[Error: Invalid JSON (Unexpected "5" at position 45 in state TRUE1)]
[Error: Invalid JSON (Unexpected "¸" at position 46 in state TRUE1)]
[Error: Invalid JSON (Unexpected "î" at position 47 in state TRUE1)]
[Error: Invalid JSON (Unexpected "#" at position 48 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\u001d" at position 49 in state TRUE1)]
[Error: Invalid JSON (Unexpected "m" at position 50 in state TRUE1)]
[Error: Invalid JSON (Unexpected "p" at position 51 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\f" at position 52 in state TRUE1)]
[Error: Invalid JSON (Unexpected "ñ" at position 53 in state TRUE1)]
[Error: Invalid JSON (Unexpected "¿" at position 54 in state TRUE1)]
[Error: Invalid JSON (Unexpected "[" at position 55 in state TRUE1)]
[Error: Invalid JSON (Unexpected "‡" at position 56 in state TRUE1)]
[Error: Invalid JSON (Unexpected " " at position 57 in state TRUE1)]
[Error: Invalid JSON (Unexpected "V" at position 58 in state TRUE1)]
[Error: Invalid JSON (Unexpected "¼" at position 59 in state TRUE1)]
[Error: Invalid JSON (Unexpected "æ" at position 60 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\r" at position 61 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\u000f" at position 62 in state TRUE1)]
[Error: Invalid JSON (Unexpected "!" at position 63 in state TRUE1)]
[Error: Invalid JSON (Unexpected "§" at position 64 in state TRUE1)]
[Error: Invalid JSON (Unexpected "É" at position 65 in state TRUE1)]
[Error: Invalid JSON (Unexpected "]" at position 66 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\u0015" at position 67 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Ø" at position 68 in state TRUE1)]
[Error: Invalid JSON (Unexpected "’" at position 69 in state TRUE1)]
[Error: Invalid JSON (Unexpected "š" at position 70 in state TRUE1)]
[Error: Invalid JSON (Unexpected "©" at position 71 in state TRUE1)]
[Error: Invalid JSON (Unexpected "ƒ" at position 72 in state TRUE1)]
[Error: Invalid JSON (Unexpected "a" at position 73 in state TRUE1)]
[Error: Invalid JSON (Unexpected "M" at position 74 in state TRUE1)]
[Error: Invalid JSON (Unexpected "" at position 75 in state TRUE1)]
[Error: Invalid JSON (Unexpected "5" at position 76 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\r" at position 77 in state TRUE1)]
[Error: Invalid JSON (Unexpected "±" at position 78 in state TRUE1)]
[Error: Invalid JSON (Unexpected "J" at position 79 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Ô" at position 80 in state TRUE1)]
[Error: Invalid JSON (Unexpected "~" at position 81 in state TRUE1)]
[Error: Invalid JSON (Unexpected "»" at position 82 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\u000e" at position 83 in state TRUE1)]
[Error: Invalid JSON (Unexpected "³" at position 84 in state TRUE1)]
[Error: Invalid JSON (Unexpected "R" at position 85 in state TRUE1)]
[Error: Invalid JSON (Unexpected "¤" at position 86 in state TRUE1)]
[Error: Invalid JSON (Unexpected "›" at position 87 in state TRUE1)]
[Error: Invalid JSON (Unexpected "¥" at position 88 in state TRUE1)]
[Error: Invalid JSON (Unexpected "é" at position 89 in state TRUE1)]
[Error: Invalid JSON (Unexpected "í" at position 90 in state TRUE1)]
[Error: Invalid JSON (Unexpected "•" at position 91 in state TRUE1)]
[Error: Invalid JSON (Unexpected "ð" at position 92 in state TRUE1)]
[Error: Invalid JSON (Unexpected "U" at position 93 in state TRUE1)]
[Error: Invalid JSON (Unexpected "¦" at position 94 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Ú" at position 95 in state TRUE1)]
[Error: Invalid JSON (Unexpected "¶" at position 96 in state TRUE1)]
[Error: Invalid JSON (Unexpected "i" at position 97 in state TRUE1)]
[Error: Invalid JSON (Unexpected "È" at position 98 in state TRUE1)]
[Error: Invalid JSON (Unexpected "¹" at position 99 in state TRUE1)]
[Error: Invalid JSON (Unexpected "‘" at position 100 in state TRUE1)]
[Error: Invalid JSON (Unexpected "!" at position 101 in state TRUE1)]
[Error: Invalid JSON (Unexpected "ô" at position 102 in state TRUE1)]
[Error: Invalid JSON (Unexpected "¹" at position 103 in state TRUE1)]
[Error: Invalid JSON (Unexpected "³" at position 104 in state TRUE1)]
[Error: Invalid JSON (Unexpected "¶" at position 105 in state TRUE1)]
[Error: Invalid JSON (Unexpected "n" at position 106 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\u0006" at position 107 in state TRUE1)]
[Error: Invalid JSON (Unexpected "N" at position 108 in state TRUE1)]
[Error: Invalid JSON (Unexpected " " at position 109 in state TRUE1)]
[Error: Invalid JSON (Unexpected "8" at position 110 in state TRUE1)]
[Error: Invalid JSON (Unexpected "ª" at position 111 in state TRUE1)]
[Error: Invalid JSON (Unexpected "G" at position 112 in state TRUE1)]
[Error: Invalid JSON (Unexpected "ò" at position 113 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Á" at position 114 in state TRUE1)]
[Error: Invalid JSON (Unexpected "V" at position 115 in state TRUE1)]
[Error: Invalid JSON (Unexpected "È" at position 116 in state TRUE1)]
[Error: Invalid JSON (Unexpected "ä" at position 117 in state TRUE1)]
[Error: Invalid JSON (Unexpected "=" at position 118 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Â" at position 119 in state TRUE1)]
[Error: Invalid JSON (Unexpected "‚" at position 120 in state TRUE1)]
[Error: Invalid JSON (Unexpected "$" at position 121 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Æ" at position 122 in state TRUE1)]
[Error: Invalid JSON (Unexpected "¤" at position 123 in state TRUE1)]
[Error: Invalid JSON (Unexpected "7" at position 124 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\"" at position 125 in state TRUE1)]
[Error: Invalid JSON (Unexpected "ä" at position 126 in state TRUE1)]
[Error: Invalid JSON (Unexpected "F" at position 127 in state TRUE1)]
[Error: Invalid JSON (Unexpected "©" at position 128 in state TRUE1)]
[Error: Invalid JSON (Unexpected "B" at position 129 in state TRUE1)]
[Error: Invalid JSON (Unexpected "¾" at position 130 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Œ" at position 131 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\u001b" at position 132 in state TRUE1)]
[Error: Invalid JSON (Unexpected "o" at position 133 in state TRUE1)]
[Error: Invalid JSON (Unexpected "¯" at position 134 in state TRUE1)]
[Error: Invalid JSON (Unexpected " " at position 135 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\u001e" at position 136 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Ž" at position 137 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Ö" at position 138 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Õ" at position 139 in state TRUE1)]
[Error: Invalid JSON (Unexpected "?" at position 140 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Þ" at position 141 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\u0006" at position 142 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\u0005" at position 143 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\n" at position 144 in state TRUE1)]
[Error: Invalid JSON (Unexpected "¬" at position 145 in state TRUE1)]
[Error: Invalid JSON (Unexpected "±" at position 146 in state TRUE1)]
[Error: Invalid JSON (Unexpected "2" at position 147 in state TRUE1)]
[Error: Invalid JSON (Unexpected "-" at position 148 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Æ" at position 149 in state TRUE1)]
[Error: Invalid JSON (Unexpected "`" at position 150 in state TRUE1)]
[Error: Invalid JSON (Unexpected "‡" at position 151 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Â" at position 152 in state TRUE1)]
[Error: Invalid JSON (Unexpected "ï" at position 153 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\u0014" at position 154 in state TRUE1)]
[Error: Invalid JSON (Unexpected "" at position 155 in state TRUE1)]
[Error: Invalid JSON (Unexpected "·" at position 156 in state TRUE1)]
[Error: Invalid JSON (Unexpected "C" at position 157 in state TRUE1)]
[Error: Invalid JSON (Unexpected "†" at position 158 in state TRUE1)]
[Error: Invalid JSON (Unexpected "¥" at position 159 in state TRUE1)]
[Error: Invalid JSON (Unexpected "C" at position 160 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Ö" at position 161 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Æ" at position 162 in state TRUE1)]
[Error: Invalid JSON (Unexpected "k" at position 163 in state TRUE1)]
[Error: Invalid JSON (Unexpected "›" at position 164 in state TRUE1)]
[Error: Invalid JSON (Unexpected "@" at position 165 in state TRUE1)]
[Error: Invalid JSON (Unexpected "ž" at position 166 in state TRUE1)]
[Error: Invalid JSON (Unexpected "}" at position 167 in state TRUE1)]
[Error: Invalid JSON (Unexpected "™" at position 168 in state TRUE1)]
[Error: Invalid JSON (Unexpected ";" at position 169 in state TRUE1)]
[Error: Invalid JSON (Unexpected "t" at position 170 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\b" at position 171 in state TRUE1)]
[Error: Invalid JSON (Unexpected "y" at position 172 in state TRUE1)]
[Error: Invalid JSON (Unexpected "i" at position 173 in state TRUE1)]
[Error: Invalid JSON (Unexpected "º" at position 174 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Î" at position 175 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Ç" at position 176 in state TRUE1)]
[Error: Invalid JSON (Unexpected "¤" at position 177 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\u000f" at position 178 in state TRUE1)]
[Error: Invalid JSON (Unexpected "©" at position 179 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Ò" at position 180 in state TRUE1)]
[Error: Invalid JSON (Unexpected "c" at position 181 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Š" at position 182 in state TRUE1)]
[Error: Invalid JSON (Unexpected "È" at position 183 in state TRUE1)]
[Error: Invalid JSON (Unexpected "¿" at position 184 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\u0017" at position 185 in state TRUE1)]
[Error: Invalid JSON (Unexpected "®" at position 186 in state TRUE1)]
[Error: Invalid JSON (Unexpected "l" at position 187 in state TRUE1)]
[Error: Invalid JSON (Unexpected "É" at position 188 in state TRUE1)]
[Error: Invalid JSON (Unexpected "°" at position 189 in state TRUE1)]
[Error: Invalid JSON (Unexpected "%" at position 190 in state TRUE1)]
[Error: Invalid JSON (Unexpected "o" at position 191 in state TRUE1)]
[Error: Invalid JSON (Unexpected "®" at position 192 in state TRUE1)]
[Error: Invalid JSON (Unexpected "1" at position 193 in state TRUE1)]
[Error: Invalid JSON (Unexpected "F" at position 194 in state TRUE1)]
[Error: Invalid JSON (Unexpected "U" at position 195 in state TRUE1)]
[Error: Invalid JSON (Unexpected "H" at position 196 in state TRUE1)]
[Error: Invalid JSON (Unexpected "S" at position 197 in state TRUE1)]
[Error: Invalid JSON (Unexpected "ë" at position 198 in state TRUE1)]
[Error: Invalid JSON (Unexpected "®" at position 199 in state TRUE1)]
[Error: Invalid JSON (Unexpected "" at position 200 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\u001f" at position 201 in state TRUE1)]
[Error: Invalid JSON (Unexpected "7" at position 202 in state TRUE1)]
[Error: Invalid JSON (Unexpected "9" at position 203 in state TRUE1)]
[Error: Invalid JSON (Unexpected "?" at position 204 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\u0001" at position 205 in state TRUE1)]
[Error: Invalid JSON (Unexpected "Á" at position 206 in state TRUE1)]
[Error: Invalid JSON (Unexpected "0" at position 207 in state TRUE1)]
[Error: Invalid JSON (Unexpected "µ" at position 208 in state TRUE1)]
[Error: Invalid JSON (Unexpected "2" at position 209 in state TRUE1)]
[Error: Invalid JSON (Unexpected "ˆ" at position 210 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\u0001" at position 211 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\u0000" at position 212 in state TRUE1)]
[Error: Invalid JSON (Unexpected "\u0000" at position 213 in state TRUE1)]

[dominictarr]

hmm, that is not right! It should take stop on the first error!

[gchauvet]

Hmm, some parsers can manage many errors in a file, by using a look ahead error recovery mechanism. In this case, we have a "partial" stream (some data are parsed) and others emits an error event.

Moreover, I don't read in node.js handbook than one, and only one error must be emitted by a stream ?

Note: this doesn't concern jsonparser library ;)

[dominictarr]

json is not a parser that can accept errors. If you get unbalanced brackets it totally changes the meaning of the structure. Node streams can only have one error event. You should think of errors like another flavor of 'end' event.