Use semantic-js package

[lizozom]

I extracted a semantic search package This is an implementation of the demo using that package.

Note that the progress bar is gone, as the worker returns the final list.

[do-me]

Thanks for the PR! A few points:

Note that the progress bar is gone, as the worker returns the final list.

• Unfortunately I believe the progress bar is crucial. For the small demo it doesn't make much of a difference but for huge docs with 100's of pages of pdf you need to get a feeling about the right parameters. I always tried with different character length to balance between indexing speed and accuracy (in terms of actually finding what I was looking for). Without progress bar it would leave the user without information about progress and would likely lead to disappointment. Also - apart from the satisfying process of seeing the algorithm in action and retrieving results in real time - it had the advantage of showing the most important results as they came in. I usually already found the paragraph I was looking for before it was completely finished. Are you using threads already in the webworker? If so, do you think there is any way to monitor the progress?

• I love the console log! This is great for debugging.

• There might be a bug with the tokenizer. Select tokens and click submit - it will throw this error.

semantic.js:23 Uncaught (in promise) TypeError: tokenizer is not a function
    at getTokens (semantic.js:23:18)
    at splitByTokens (utils.js:43:82)
    at splitText (utils.js:25:26)
    at semanticHighlight (index.js:199:80)
    at onSubmit (index.js:80:15)
    at HTMLButtonElement.onclick ((index):49:110)

[lizozom]

No problem, I'm pretty sure I could stream the results from the package, instead of returning them in one piece.

Regardless, I think we should decide whether we want to use a package or not (I will gladly make you guys co-admins @do-me @VarunNSrivastava ), otherwise we're going to get a bunch of conflicts.

[lizozom]

And the table is @VarunNSrivastava 's doing!

[do-me]

No problem, I'm pretty sure I could stream the results from the package, instead of returning them in one piece.

That would be really nice!

As we have two PR's open at the moment and @VarunNSrivastava 's #23 is still building on the current implementation, I'd go ahead and merge his first. As soon as we have the progress bar logic in semantic-js, I guess we'd be ready to merge semantic-js and SemanticFinder (or split, depending on the perspective ;) ). Hence, for the moment, I'll just leave this PR open.

[varunneal]

I used Xenova's demo for reference on how to do progress bars with WebWorkers. Once my PR is merged hopefully the progress bars look a little cleaner and we can implement this in semantic-js.

[lizozom]

Replaced with https://github.com/do-me/SemanticFinder/pull/26