search

jnordberg / dsteem

Steem blockchain RPC client
https://jnordberg.github.io/dsteem/
Other
82 stars 57 forks source link

Unexpected token < in JSON at position 0 - while stream.on() #4

Closed therealwolf42 closed 7 years ago

therealwolf42 commented 7 years ago

I understand that I have to handle promise rejections in the code (I'm still trying to figure out how I do this) but might it be possible that the underlying problem is somewhere else?

I get the error whenever I try to read from the stream at a random time. Sometimes after 1 minute sometimes after 10 minutes. Depending on the input that is generated.

The full error:

(node:64920) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): ResponseError: Unable to read response data: Unexpected token < in JSON at position 0 (node:64920) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

jnordberg commented 7 years ago

Looks like you're getting network errors, you need to handle them manually on the streams now (i might add something that handles stream errors in the future though).

You could do something like this (untested)


function run() {
  const stream = client.blockchain.createBlockStream()
  stream.on('data', (block) => { console.log('block', block) })
  stream.on('error', (error) => {
    console.log('stream error', error)
  })
  stream.on('end', () => {
    console.log('stream ended, recreating...')
    run()
  })
}
therealwolf42 commented 7 years ago

Restarting works but it often happens that I'm looking for e.g. an upvote or comment and right when it should be found the connection interrupts. This makes it unreliable. Are you sure that there is no problem with the JSON return? Isn't there something to make it just skip that JSON? And if its possible to output which JSON actually caused the error?

Edit: I've added a bit of commenting to your dsteem code to see what is going on and it seems that instead of json - an html element get's parsed.

undefined:1
<html>
^

SyntaxError: Unexpected token < in JSON at position 0
    at JSON.parse (<anonymous>)
    at IncomingMessage.stream.on (/Users/wolf/Bots/CommentVote/dsteem/lib/utils.js:132:81)
    at emitNone (events.js:110:20)
    at IncomingMessage.emit (events.js:207:7)
    at endReadableNT (_stream_readable.js:1059:12)
    at _combinedTickCallback (internal/process/next_tick.js:138:11)
    at process._tickCallback (internal/process/next_tick.js:180:9)
therealwolf42 commented 7 years ago

After researching more I found out that this the error. 

<html>
<head><title>504 Gateway Time-out</title></head>
<body bgcolor="white">
<center><h1>504 Gateway Time-out</h1></center>
</body>
</html>

Seems like you were right about network errors. But what is the best way to handle this? Because I'm looking for transactions and if transactions come in while I have that time-out error I'd lose transactions and that's very bad. I'd have to either start searching blocks from the moment it stopped or to watch for them in my transfer history (but is that even possible)?

Oh and by the way I'm using

const stream = client.blockchain.getOperationsStream({mode: dsteem.BlockchainMode.Irreversible});

and not 'createBlockStream' what you had in your example.

jnordberg commented 7 years ago

@therealwolf42 try updating to 0.8.1, it should have you seeing much fewer network errors. The new default behaviour is to retry requests for 1 minute. If they timeout you still need to handle the error but should only happen if there's a major outage now