n8n-io / n8n

Free and source-available fair-code licensed workflow automation tool. Easily automate tasks across different services.
https://n8n.io
Other
45.7k stars 6.36k forks source link

Google Drive node fails with error "invalid array length" #4977

Closed germanbravouy closed 1 year ago

germanbravouy commented 1 year ago

Describe the bug When using the Google Drive node to upload a huge file, i get the error "invalid array length", it doesnt happen with small files. The file is 400MB in size.

Error thrown by node: ERROR: UNKNOWN ERROR - check the detailed error for more information Invalid array length

Stack: NodeApiError: UNKNOWN ERROR - check the detailed error for more information at Object.googleApiRequest (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/nodes/Google/Drive/GenericFunctions.ts:71:9) at Object.execute (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/nodes/Google/Drive/GoogleDrive.node.ts:2568:45) at Workflow.runNode (/usr/local/lib/node_modules/n8n/node_modules/n8n-workflow/src/Workflow.ts:1270:19) at /usr/local/lib/node_modules/n8n/node_modules/n8n-core/src/WorkflowExecute.ts:939:29

To Reproduce Steps to reproduce the behavior:

  1. Use the "Read binary file" to read a big file (400MB in my case) to the "data" variable
  2. Use the Google Drive node, to upload that file to a drive
  3. See error "invalid array length" when trying

Expected behavior Upload the file succesfully, since Google Drive accepts bigger files than 400MB.

Environment (please complete the following information):

Additional context N8N_DEFAULT_BINARY_DATA_MODE=filesystem

germanbravouy commented 1 year ago

I tried uploading a file of 47.7MB, and the Google Drive node fails with the following log in the K8S pod of n8n:

(the pods have the following resource limits: 2Gi mem, 300m CPU)

<--- Last few GCs --->

[964:0x7fb85d1c43b0] 193195 ms: Mark-sweep (reduce) 1026.5 (1028.4) -> 1026.5 (1030.4) MB, 14992.9 / 0.0 ms (average mu = 0.079, current mu = 0.000) allocation failure scavenge might not succeed [964:0x7fb85d1c43b0] 208499 ms: Mark-sweep (reduce) 1027.5 (1032.4) -> 1027.5 (1033.4) MB, 15298.2 / 0.0 ms (average mu = 0.041, current mu = 0.000) allocation failure scavenge might not succeed

<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason: ReferenceError: AbortSignal is not defined

Then I tried uploading a file of 976.6K in size, and it uploaded succesfully.

janober commented 1 year ago

Fix got released with n8n@0.210.0