typesense / typesense-js

JavaScript / TypeScript client for Typesense
https://typesense.org/docs/api
Apache License 2.0
373 stars 73 forks source link

Import multiple documents sometimes fail parsing response from Typesense server. #210

Open hortelanobruno-ol opened 1 month ago

hortelanobruno-ol commented 1 month ago

Description

Calling the import function with an array greater than 250 elements sometimes fails.

Steps to reproduce

Initialize a collection with more than 250 elements and import that collection repeatedly.

const documents = [...];
for(var i = 0; i < 1000;i++){
 await tsClient
  .collections<T>('testcollection')
  .documents()
  .import(documents); // this fails
}

Expected Behavior

It doesn't fail.

Actual Behavior

It fails with this error:

SyntaxError: Unexpected end of JSON input at JSON.parse () at /app/nodemodules/.pnpm/typesense@[1.4.1@babel](mailto:1.4.1_@babel)+runtime@7.21.0/node_modules/typesense/lib/Typesense/Documents.js:145:115 at Array.map () at Documents. (/app/nodemodules/.pnpm/typesense@[1.4.1@babel](mailto:1.4.1_@babel)+runtime@7.21.0/node_modules/typesense/lib/Typesense/Documents.js:145:84) at step (/app/nodemodules/.pnpm/typesense@[1.4.1@babel](mailto:1.4.1_@babel)+runtime@7.21.0/node_modules/typesense/lib/Typesense/Documents.js:48:23) at Object.next (/app/nodemodules/.pnpm/typesense@[1.4.1@babel](mailto:1.4.1_@babel)+runtime@7.21.0/node_modules/typesense/lib/Typesense/Documents.js:29:53) at fulfilled (/app/nodemodules/.pnpm/typesense@[1.4.1@babel](mailto:1.4.1_@babel)+runtime@7.21.0/node_modules/typesense/lib/Typesense/Documents.js:20:58) at runMicrotasks () at processTicksAndRejections (node:internal/process/task_queues:96:5)

Metadata

Typesense Version: 0.25.2

OS: Linux

jasonbosco commented 1 month ago

This is most likely because the code snippet you have makes 1K concurrent connections and Typesense is aborting those requests due to lack of CPU capacity.

Could you try using for-await...of when using promises inside of for loops?