search

typesense / firestore-typesense-search

Firebase Extension to automatically push Firestore documents to Typesense for full-text search with typo tolerance, faceting, and more
https://extensions.dev/extensions/typesense/firestore-typesense-search
Apache License 2.0
159 stars 35 forks source link

Firestore Backfill Not Working #88

Closed tapangarg closed 2 months ago

tapangarg commented 2 months ago

Description

Firestore Extension installed, but backfill is not getting triggered.

Steps to reproduce

image image

Expected Behavior

Backfill to trigger and documents to be updated to my notes collection on Typesense

Actual Behavior

image

Metadata

*Typesense Version2.7*:

OS:

jasonbosco commented 2 months ago

Could you try deleting the trigger key completely from the document and create it again?

It should still log that missing key error when you delete the key, but when you add it back again, you should now see a 2nd log message stating the backfill is starting and either a success or error messages.

tapangarg commented 2 months ago

Thanks Jason. I did this and backfill started, but ended in a crash because of memory. I'm uploading 35 thousand records only.

Screenshot 2024-09-12 080535

jasonbosco commented 2 months ago

For large datasets, you would have to run the backfill using an admin script from your local computer.

We still need to look into batching during backfill as part of #70 to make a cloud function run a large backfill.

tapangarg commented 2 months ago

I see. Thank you. Can you point me to some documentation about this?