typesense / typesense-mongodb

A Node.js CLI to sync documents from a MongoDB collection to Typesense.
https://typesense.org/docs/0.21.0/guide/mongodb-full-text-search.html
Apache License 2.0
28 stars 5 forks source link

How to enable facet for fields? #9

Open ghost opened 2 years ago

ghost commented 2 years ago

Description

harisarang commented 2 years ago

@dzango-roshan You need to create your own collection in typesense (with marking facets in it) and point that collection in this integration

yijia99 commented 1 year ago

@dzango-roshan You need to create your own collection in typesense (with marking facets in it) and point that collection in this integration

Do you mean manually create, like: `curl "${TYPESENSE_HOST}/collections" \ -X POST \ -H "Content-Type: application/json" \ -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" -d '{ "name": "books", "fields": [ {"name": "title", "type": "string" }, {"name": "authors", "type": "string[]", "facet": true },

      {"name": "publication_year", "type": "int32", "facet": true },
      {"name": "ratings_count", "type": "int32" },
      {"name": "average_rating", "type": "float" }       
    ],
    "default_sorting_field": "ratings_count"
  }'

Then run this tool with--typesense-collection=books` ?

I got an err: ✖ Create a new Typesense Collection → Request failed with HTTP code 409 | Server said: A collection with namebooksalready exists.

jzrchilel commented 1 year ago

Hey @harisarang, I'm having the same issue as @yijia99. I tried to create a collection manually to enable the facets and then tried to run typesense-mongodb and I'm getting the following error:

npx typesense-mongodb \
    --mongo-collection=books \
    --mongo-database=test \
    --typesense-collection=books \
    --mongo-url=mongodb://localhost:27017 \
    --typesense-url=http://localhost:8108 \
    --typesense-api-key=xyz
  ✔ Initialize Typesense Client
  ✔ Initialize Mongo Client
  ✔ Check for an existing typesense collection
  ✖ Create a new Typesense Collection
    → Request failed with HTTP code 409 | Server said: A collection with name `books` already exists.
    Index existing documents
    Open Change Stream

/Users/user/.npm/_npx/126aa40a701718f1/node_modules/typesense/lib/Typesense/ApiCall.js:378
      var customErrror = new CustomErrorKlass(errorMessage);
                         ^

ObjectAlreadyExists [Error]: Request failed with HTTP code 409 | Server said: A collection with name `books` already exists.
    at ApiCall._customErrorForResponse (/Users/user/.npm/_npx/126aa40a701718f1/node_modules/typesense/lib/Typesense/ApiCall.js:378:26)
    at ApiCall._callee$ (/Users/user/.npm/_npx/126aa40a701718f1/node_modules/typesense/lib/Typesense/ApiCall.js:186:70)
    at tryCatch (/Users/user/.npm/_npx/126aa40a701718f1/node_modules/@babel/runtime/helpers/regeneratorRuntime.js:45:16)
    at Generator.<anonymous> (/Users/user/.npm/_npx/126aa40a701718f1/node_modules/@babel/runtime/helpers/regeneratorRuntime.js:133:17)
    at Generator.next (/Users/user/.npm/_npx/126aa40a701718f1/node_modules/@babel/runtime/helpers/regeneratorRuntime.js:74:21)
    at asyncGeneratorStep (/Users/user/.npm/_npx/126aa40a701718f1/node_modules/@babel/runtime/helpers/asyncToGenerator.js:3:24)
    at _next (/Users/user/.npm/_npx/126aa40a701718f1/node_modules/@babel/runtime/helpers/asyncToGenerator.js:22:9)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  httpStatus: 409,
  context: [Object: null prototype] {}
}

@yijia99 were you able to solve it?

yijia99 commented 1 year ago

Hi @jzrchilel My tmp solution is to skip creating collection anyway, e.g: lib/main.js

` {

            title: "Create a new Typesense Collection",

            task: () => typesense.createCollection(options.typesenseCollectionName),

            skip: () => 

            // skip anyway

            true

                ? "Found an existing collection skipping create collection"

                : undefined,
        },

`

But this tool failed again when I process a 5k+ documents mongoDB collection. Same as: https://github.com/typesense/typesense-mongodb/issues/11

So finally I had to write my own connector scripts.

Babaktrad commented 4 months ago

https://github.com/typesense/typesense-mongodb/blob/de96e815eabdf41f9ee8498dcde0c0f41727007a/src/TypesenseClient.ts#L89-L101

https://github.com/typesense/typesense-mongodb/blob/de96e815eabdf41f9ee8498dcde0c0f41727007a/src/main.ts#L108-L115

These sections of code cause the issue. In a fresh install of mongodb and typesense, no documents exist. As a result, line 93 returns 0 and the task Create a new Typesense Collection will not be skipped. To work around this issue, we can follow the steps below:

  1. Create your own collection in typesense.
  2. Index at least one document into the collection created in step 1.

If you only attempt step 1, you still encounter the error. However, if you try with both steps, you'll encounter a new error A document with id xxx already exists..

Finally, I changed line 93 to result.name === collectionName which resolved both errors.