Closed lizozom closed 1 year ago
Thanks for the PR! A few points:
Note that the progress bar is gone, as the worker returns the final list.
Unfortunately I believe the progress bar is crucial. For the small demo it doesn't make much of a difference but for huge docs with 100's of pages of pdf you need to get a feeling about the right parameters. I always tried with different character length to balance between indexing speed and accuracy (in terms of actually finding what I was looking for). Without progress bar it would leave the user without information about progress and would likely lead to disappointment. Also - apart from the satisfying process of seeing the algorithm in action and retrieving results in real time - it had the advantage of showing the most important results as they came in. I usually already found the paragraph I was looking for before it was completely finished. Are you using threads already in the webworker? If so, do you think there is any way to monitor the progress?
I love the console log! This is great for debugging.
There might be a bug with the tokenizer. Select tokens
and click submit
- it will throw this error.
semantic.js:23 Uncaught (in promise) TypeError: tokenizer is not a function
at getTokens (semantic.js:23:18)
at splitByTokens (utils.js:43:82)
at splitText (utils.js:25:26)
at semanticHighlight (index.js:199:80)
at onSubmit (index.js:80:15)
at HTMLButtonElement.onclick ((index):49:110)
No problem, I'm pretty sure I could stream the results from the package, instead of returning them in one piece.
Regardless, I think we should decide whether we want to use a package or not (I will gladly make you guys co-admins @do-me @VarunNSrivastava ), otherwise we're going to get a bunch of conflicts.
And the table is @VarunNSrivastava 's doing!
No problem, I'm pretty sure I could stream the results from the package, instead of returning them in one piece.
That would be really nice!
As we have two PR's open at the moment and @VarunNSrivastava 's #23 is still building on the current implementation, I'd go ahead and merge his first. As soon as we have the progress bar logic in semantic-js, I guess we'd be ready to merge semantic-js and SemanticFinder (or split, depending on the perspective ;) ). Hence, for the moment, I'll just leave this PR open.
I used Xenova's demo for reference on how to do progress bars with WebWorkers. Once my PR is merged hopefully the progress bars look a little cleaner and we can implement this in semantic-js.
Replaced with https://github.com/do-me/SemanticFinder/pull/26
I extracted a semantic search package This is an implementation of the demo using that package.
Note that the progress bar is gone, as the worker returns the final list.