mblackstock / node-red-contrib-influxdb

Node-RED nodes to save and query data from an influxdb time series database
Other
64 stars 44 forks source link

WARN: Write to InfluxDB failed (attempt: 1). y [RequestTimedOutError]: Request timed out #108

Closed janvda closed 2 years ago

janvda commented 2 years ago

When writing at a very high rate measurements to my influx DB (on another device) I am getting the following error. Moreover when having that error node-red seems to hang for a while.

WARN: Write to InfluxDB failed (attempt: 1). y [RequestTimedOutError]: Request timed out
    at ClientRequest.<anonymous> (/data/node_modules/@influxdata/influxdb-client/dist/index.js:16:10715)
    at ClientRequest.emit (events.js:400:28)
    at Socket.emitRequestTimeout (_http_client.js:790:9)
    at Object.onceWrapper (events.js:519:28)
    at Socket.emit (events.js:412:35)
    at Socket._onTimeout (net.js:495:8)
    at listOnTimeout (internal/timers.js:557:17)
    at processTimers (internal/timers.js:500:7)
25 Nov 15:28:44 - [info] [debug:catch all] 
{
  topic: '/miband-api/activity',
  payload: [
    [
      { category: 112, intensity: 0, steps: 0, time: 1637474340000 },
      { person: 'jan' }
    ]
  ],
  qos: 0,
  retain: false,
  _msgid: '6e57bd75ebca7689',
  ttl: 0,
  _queuetimestamp: 1637850474084,
  _queueCount: 4215,
  last_activity_time_ms: 1637474280000,
  expected_time_ms: 1637474340000,
  influx_error: {
    errorMessage: y [RequestTimedOutError]: Request timed out
        at ClientRequest.<anonymous> (/data/node_modules/@influxdata/influxdb-client/dist/index.js:16:10715)
        at ClientRequest.emit (events.js:400:28)
        at Socket.emitRequestTimeout (_http_client.js:790:9)
        at Object.onceWrapper (events.js:519:28)
        at Socket.emit (events.js:412:35)
        at Socket._onTimeout (net.js:495:8)
        at listOnTimeout (internal/timers.js:557:17)
        at processTimers (internal/timers.js:500:7)
  },
  error: {
    message: 'RequestTimedOutError: Request timed out',
    source: {
      id: 'b1dbea7902f83c6d',
      type: 'influxdb out',
      name: 'my influx db - activity measurement',
      count: 1
    },
    stack: 'RequestTimedOutError: Request timed out\n' +
      '    at ClientRequest.<anonymous> (/data/node_modules/@influxdata/influxdb-client/dist/index.js:16:10715)\n' +
      '    at ClientRequest.emit (events.js:400:28)\n' +
      '    at Socket.emitRequestTimeout (_http_client.js:790:9)\n' +
      '    at Object.onceWrapper (events.js:519:28)\n' +
      '    at Socket.emit (events.js:412:35)\n' +
      '    at Socket._onTimeout (net.js:495:8)\n' +
      '    at listOnTimeout (internal/timers.js:557:17)\n' +
      '    at processTimers (internal/timers.js:500:7)'
  }
}
janvda commented 2 years ago

I understand that this error might be normal when overloading the influx db with write requests but how can I best throttle this ?

One simple solution would be that I write the next measurement once the previous measurement has been sent to the influx DB but the influxdb out node I am using has no output, so that option is not possible.

Maybe instead of sending a single measurement, I should bundle them and write them in one call to the influx db.

mblackstock commented 2 years ago

One way might be to write the next measurement once the node indicates it's done. You could try wiring up a 'complete' node to do this. Once the node indicates it's done, you could trigger the next write. Hopefully this gives you enough to try that out. Here are some references.

Another thought may be to use the delay node:

https://cookbook.nodered.org/basic/rate-limit-messages

Hope that helps.