nextapps-de / flexsearch

Next-Generation full text search library for Browser and Node.js
Apache License 2.0
12.33k stars 489 forks source link

Does `export` really return a `Promise`? Are you *sure*? #393

Closed dhdaines closed 8 months ago

dhdaines commented 1 year ago

The export method does not really seem to behave as one would expect an async method to do. For instance, this:

import { Index } from "flexsearch";

(async () => {
  const index = new Index();
  await index.addAsync(42, "hello world");
  await index.export(async (key: string | number, data: any) => {
    console.log(`Exported key ${key}`);
  });
  console.log("Did we export everything yet?");
})();

Prints this:

Did we export everything yet?
Exported key reg
Exported key cfg
Exported key map
Exported key ctx

Whereas I would expect that calling await on the Promise that it supposedly returns would, you know, await the end of the export. Note that it makes no difference if the callback is not declared as async either. Same problem.

This is annoying because there is otherwise no way to know what all the keys are.

justin0mcateer commented 1 year ago

This problem basically caused us to have to stop using this library.

The saving and loading implementation makes it very difficult to integrate.

dhdaines commented 1 year ago

I think that in general flexsearch does not live up to its own hype, it is not at all clear to me what the scoring function is or how to get the scores, if it is so wonderful, one would expect there to be MAP/MRR numbers on a standard dataset, etc, etc. I'm switching to lunr.js which appears to be a real search engine.

fgatti675 commented 1 year ago

Same, moving away form this library due to all the messed up imports. I don't understand the hype either

xulihang commented 1 year ago

I choose FlexSearch over lunr.js because it has a better support for CJK. But the export function is a problem that I cannot know when it is completed.

xulihang commented 1 year ago

If we know how many keys to export, then we can know when it stops. Here is my code:

async function saveIndex(name){
  return new Promise(async function(resolve){
    let count = 0;
    let numberOfKeyToExport = Object.keys(documentIndex.index).length*3 + 3;
    documentIndex.export(async function(key, data){ 
      // you need to store both the key and the data!
      // e.g. use the key for the filename and save your data
      console.log("save"+key);
      await localforage.setItem(name+"-"+key, data);
      count = count + 1;
      if (count === numberOfKeyToExport) {
        resolve();
      }
    });
  });
}
iivvaannxx commented 11 months ago

If anyone ends up with this same issue, I provided a workaround in the above issue using a custom patch file.

ts-thomas commented 8 months ago

This should be fixed now. Please give me feedback when the issue still exists,