weaviate / typescript-client

Official Weaviate TypeScript Client
https://www.npmjs.com/package/weaviate-client
BSD 3-Clause "New" or "Revised" License
57 stars 21 forks source link

Warn when errors occurred during batch operations #61

Open dandv opened 1 year ago

dandv commented 1 year ago

I had forgotten to include the OpenAI API key in the headers, and noticed that batch imports appeared to succeed, but no objects were committed to the database.

Turned out that the response to batcher.do() included a result field with the error buried inside. I expected the batcher to throw.

import assert from 'assert';

import weaviate from 'weaviate-ts-client';
import fs from 'fs';
import csv from 'csv-parser';

const client = weaviate.client({
  scheme: 'http',
  host: 'localhost:8080',
  // no OpenAI API key
});

const classDefinition = {
  class: 'JeopardyQuestion',
  description: 'A Jeopardy! question',
  vectorizer: 'text2vec-openai',
};

try {
  await client.schema.classDeleter().withClassName('JeopardyQuestion').do();
} catch { }
finally {
  await client.schema.classCreator().withClass(classDefinition).do();
}

const qs = [
  { question: 'What color?', answer: 'Blue' },
  { question: 'Where?', answer: 'In the sky' },
]

let batcher = client.batch.objectsBatcher();

for (const q of qs)
  batcher = batcher.withObject({
    class: 'JeopardyQuestion',
    properties: q,
  });

const response = await batcher.do();
console.log(response.map(r => r.result.errors.error));  // <-- all objects OK, but nothing written to the DB
batcher = client.batch.objectsBatcher();

const result = await client.graphql.aggregate().withClassName('JeopardyQuestion').withFields('meta { count }').do();
assert.deepEqual(result.data['Aggregate']['JeopardyQuestion'], [{ meta: { count: 2 } }]);