connectionTimeoutSeconds seems to not be working

[ivanbacher]

Description

Init client and set connectionTimeoutSeconds to 10. Create a bulk data import. Time this using node.js timers.

importing into collection
time taken: 20.831406687006353s

No timeout error or warning

Steps to reproduce

import Typesense from 'typesense'
import Chance from 'chance'

const chance = new Chance()

const client = new Typesense.Client({
  nodes: [{
    host: 'localhost',
    port: 8110,
    protocol: 'http'
  }],
  apiKey: 'this@is@a@key',
  connectionTimeoutSeconds: 10
})

const schema = {
  name: 'test',
  enable_nested_fields: true,
  fields: [
    { name: 'name1', type: 'string', sort: true },
    { name: 'name2', type: 'string', sort: true },
    { name: 'name3', type: 'string', sort: true },
    { name: 'name4', type: 'string', sort: true },
    { name: 'name5', type: 'string', sort: true },

    { name: 'age1', type: 'int64', sort: true },
    { name: 'age2', type: 'int64', sort: true },
    { name: 'age3', type: 'int64', sort: true },
    { name: 'age4', type: 'int64', sort: true },
    { name: 'age5', type: 'int64', sort: true },

    { name: 'bio1', type: 'string' },
    { name: 'bio2', type: 'string' },
    { name: 'bio3', type: 'string' },
    { name: 'bio4', type: 'string' },
    { name: 'bio5', type: 'string' },

    { name: 'ip1', type: 'string' },
    { name: 'ip2', type: 'string' },
    { name: 'ip3', type: 'string' },
    { name: 'ip4', type: 'string' },
    { name: 'ip5', type: 'string' },

    { name: 'updated', type: 'int64', sort: true },
  ],
  default_sorting_field: 'updated'
}

const data = []

console.log('creating data')

for (let i = 0; i < 50000; i++) {
  data.push({
    name1: chance.name(),
    name2: chance.name(),
    name3: chance.name(),
    name4: chance.name(),
    name5: chance.name(),
    age1: chance.integer({ min: 0, max: 100 }),
    age2: chance.integer({ min: 0, max: 100 }),
    age3: chance.integer({ min: 0, max: 100 }),
    age4: chance.integer({ min: 0, max: 100 }),
    age5: chance.integer({ min: 0, max: 100 }),
    bio1: chance.paragraph({ sentences: 5 }),
    bio2: chance.paragraph({ sentences: 5 }),
    bio3: chance.paragraph({ sentences: 5 }),
    bio4: chance.paragraph({ sentences: 5 }),
    bio5: chance.paragraph({ sentences: 5 }),
    ip1: chance.ip(),
    ip2: chance.ip(),
    ip3: chance.ip(),
    ip4: chance.ip(),
    ip5: chance.ip(),
    updated: Date.now()
  })
}

try {
  await client.collections(schema.name).delete()
} catch(e) {}

console.log('creating collection')
await client.collections().create(schema)

console.log('importing into collection')
const start1 = performance.now();
await client.collections(schema.name).documents().import(data)
const end1 = performance.now();
console.log(`time taken: ${(end1 - start1) / 1000}s`);

console.log('deleting collection')
await client.collections(schema.name).delete()

output:

> creating data
> creating collection
> importing into collection
> time taken: 20.831406687006353s
> deleting collection

Expected Behavior

Raise timeout error

Actual Behavior

No timeout error or warning

Metadata

Typesense Version: 0.25.1

OS: Docker Container running on MacOs

[ivanbacher]

Setting the timeout to 1

Seems to be working for deleting a collection.

creating data
creating collection
importing into collection
time taken: 16.611135839000344s
deleting collection
Request #1695217761007: Request to Node 0 failed due to "ECONNABORTED timeout of 1000ms exceeded"
Request #1695217761007: Sleeping for 0.1s and then retrying request...
Request #1695217761007: Request to Node 0 failed due to "ECONNABORTED timeout of 1000ms exceeded"

[jasonbosco]

@ivanbacher As of 1.7.0-1, we've disabled timeouts from kicking in for imports, because you almost never want to interrupt an import API call as it's executing, since that leads to stale data on the Typesense server, and multiple duplicate imports.

Other endpoints like delete still use the timeout setting.

[ivanbacher]

Ah super. thanks for the info.

I still get this error though when doing a bulk import

[21-09-2023 08:04:07] - info: Typesense-Controller: Importing entries (3483) to collection documents
[21-09-2023 08:04:09] - info: Typesense-Controller: Importing entries (16646) to collection pages
[21-09-2023 08:04:09] - info: Typesense-Controller: Importing entries (18080) to collection annotations
Request #1695279850091: Request to Node 0 failed due to "ECONNRESET socket hang up"
Request #1695279850091: Sleeping for 0.1s and then retrying request...

Now I have double the data in the annotations collection

This only occurs on the VM I am running typesense on (2 cpus). If I run the same code on my local machine (macbook pro) then there is no error

[jasonbosco]

It's possible that CPU is being saturated... Could you try on a 4 core machine?

[ivanbacher]

Tried with a 4 core machine and still getting the same error

[jasonbosco]

Could you share about 100 lines from the Typesense logs around the time when the socket hang happens?