n8n-io / n8n

Free and source-available fair-code licensed workflow automation tool. Easily automate tasks across different services.
https://n8n.io
Other
48.32k stars 7.5k forks source link

AMQP node does not respect rhea buffer, which leads to a circular buffer overflow #6206

Closed marzn closed 1 year ago

marzn commented 1 year ago

Describe the bug AMQP node does not respect rhea buffer, which leads to a circular buffer overflow exception in case the node has more than 2048 items as input.

To Reproduce Steps to reproduce the behavior:
Send more than 2048 items as input to the AMQP node.
For example with a simple workflow, which is shown on the following screenshot.
Screenshot

The code node contains the following code, which just creates 2050 simple items:

const msgs = [];
for (var i = 0; i < 2050; i++) {
  msgs.push({
    key: 'value'
  });
}
return msgs.map(m => {return {json: m}});

Expected behavior There should be no limit in the amount of items.

Environment

Additional context

Documentation of the used library rhea for the relevant send method: https://github.com/amqp/rhea/blob/main/README.md#sendmsg

Pull request: https://github.com/n8n-io/n8n/pull/6211

Stack trace

Error: circular buffer overflow: head=0 tail=0 size=2048 capacity=2048
    at CircularBuffer.push (/app/node_modules/.pnpm/rhea@1.0.24/node_modules/rhea/lib/session.js:56:15)
    at Outgoing.send (/app/node_modules/.pnpm/rhea@1.0.24/node_modules/rhea/lib/session.js:172:21)
    at Session.send (/app/node_modules/.pnpm/rhea@1.0.24/node_modules/rhea/lib/session.js:627:27)
    at Sender.send (/app/node_modules/.pnpm/rhea@1.0.24/node_modules/rhea/lib/link.js:334:33)
    at Container.<anonymous> (/app/packages/nodes-base/nodes/Amqp/Amqp.node.ts:162:38)
    at Object.onceWrapper (node:events:628:26)
    at Container.emit (node:events:513:28)
    at Container.emit (node:domain:489:12)
    at Container.dispatch (/app/node_modules/.pnpm/rhea@1.0.24/node_modules/rhea/lib/container.js:41:33)
    at Connection.dispatch (/app/node_modules/.pnpm/rhea@1.0.24/node_modules/rhea/lib/connection.js:261:40)
    at Session.dispatch (/app/node_modules/.pnpm/rhea@1.0.24/node_modules/rhea/lib/session.js:452:41)
    at Sender.link.dispatch (/app/node_modules/.pnpm/rhea@1.0.24/node_modules/rhea/lib/link.js:62:38)
    at Sender.on_flow (/app/node_modules/.pnpm/rhea@1.0.24/node_modules/rhea/lib/link.js:324:18)
    at Session.on_flow (/app/node_modules/.pnpm/rhea@1.0.24/node_modules/rhea/lib/session.js:715:31)
    at Connection.<computed> [as on_flow] (/app/node_modules/.pnpm/rhea@1.0.24/node_modules/rhea/lib/connection.js:809:30)
    at c.dispatch (/app/node_modules/.pnpm/rhea@1.0.24/node_modules/rhea/lib/types.js:910:33)
    at Transport.read (/app/node_modules/.pnpm/rhea@1.0.24/node_modules/rhea/lib/transport.js:109:36)
    at SaslClient.read (/app/node_modules/.pnpm/rhea@1.0.24/node_modules/rhea/lib/sasl.js:328:26)
    at Connection.input (/app/node_modules/.pnpm/rhea@1.0.24/node_modules/rhea/lib/connection.js:543:35)
    at Socket.emit (node:events:513:28)
    at Socket.emit (node:domain:489:12)
    at addChunk (node:internal/streams/readable:324:12)
    at readableAddChunk (node:internal/streams/readable:297:9)
    at Socket.Readable.push (node:internal/streams/readable:234:10)
    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
Emitted 'error' event on Container instance at:
    at Container.dispatch (/app/node_modules/.pnpm/rhea@1.0.24/node_modules/rhea/lib/container.js:41:33)
    at Connection.dispatch (/app/node_modules/.pnpm/rhea@1.0.24/node_modules/rhea/lib/connection.js:261:40)
    at Connection.input (/app/node_modules/.pnpm/rhea@1.0.24/node_modules/rhea/lib/connection.js:561:18)
    at Socket.emit (node:events:513:28)
    [... lines matching original stack trace ...]
    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
Joffcom commented 1 year ago

Hey @marzn,

As you have put the PR in for this I am going to mark this part as closed, The PR is in Linear to be reviewed soon.