Shazwazza / Examine

A .NET indexing and search engine powered by Lucene.Net
https://shazwazza.github.io/Examine/
380 stars 123 forks source link

Getting Searcher Synchronously? #381

Closed SerratedSharp closed 7 months ago

SerratedSharp commented 7 months ago

When in a .NET WASM project, Thread.Start() is not allowed. I believe _nrtReopenThread.Start(); in LuceneIndex.CreateSearcher() is doing this. I'm inside a using block .WithThreadingMode(IndexThreadingMode.Synchronous), but I take it that only affects indexing, and not searching.

Is there a work around that would allow me to perform searches synchrounously without threading?

Projected to ValueSets in 0.15048000000000006s
Indexing complete: 0.9268600000000001s
Operation is not supported on this platform.
System.PlatformNotSupportedException: Operation is not supported on this platform.
   at System.Threading.Thread.ThrowIfNoThreadStart(Boolean internalThread)
   at System.Threading.Thread.Start(Boolean captureContext, Boolean internalThread)
   at System.Threading.Thread.Start()
   at J2N.Threading.ThreadJob.Start()
   at Examine.Lucene.Providers.LuceneIndex.CreateSearcher()
   at System.Lazy`1[[Examine.Lucene.Providers.LuceneSearcher, Examine.Lucene, Version=3.2.0.0, Culture=neutral, PublicKeyToken=null]].ViaFactory(LazyThreadSafetyMode mode)
   at System.Lazy`1[[Examine.Lucene.Providers.LuceneSearcher, Examine.Lucene, Version=3.2.0.0, Culture=neutral, PublicKeyToken=null]].ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
   at System.Lazy`1[[Examine.Lucene.Providers.LuceneSearcher, Examine.Lucene, Version=3.2.0.0, Culture=neutral, PublicKeyToken=null]].CreateValue()
   at System.Lazy`1[[Examine.Lucene.Providers.LuceneSearcher, Examine.Lucene, Version=3.2.0.0, Culture=neutral, PublicKeyToken=null]].get_Value()
   at Examine.Lucene.Providers.LuceneIndex.get_Searcher()
   at Program.Main(String[] args)
using (var luceneIndex = (LuceneIndex)index)
using (var syncIndexContext = luceneIndex.WithThreadingMode(IndexThreadingMode.Synchronous))
{
    index.IndexItems(cardsValueSets);
    Console.WriteLine($"Indexing complete: {sw.Elapsed.TotalSeconds - lastTotalSeconds}s");

    try
    {     
        var searcher = index.Searcher; // Get a searcher
        var results = searcher.Search("vault");
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex.Message);
        Console.WriteLine(ex);
    }
}
Shazwazza commented 7 months ago

Hrm, this is a rather interesting question. Examine currently doesn't use Thread.Start but uses a queue of Tasks to perform the indexing. Lucene itself however I believe is quite dependent on Threads but this potentially depends on what is happening.

I think in this case you are correct and that it is the NRT functionality.

To be able to toggle this functionality in Examine will require some feature updates. We can add a boolean to LuceneIndexOptions to enable/disable NRT but this will have some cascading effects through the indexer along with adding a bunch more unit tests to ensure that the behavior between non-NRT and NRT has the expected results.

SerratedSharp commented 7 months ago

Thanks, if it's infeasible I understand. I was able to directly use Lucene.NET library instead and create/get IndexWriter/IndexSearcher's. So I will go that route for now. Thank you.

Shazwazza commented 7 months ago

Thanks, i'll close this for now but will look at creating this type of feature in the future.

SerratedSharp commented 7 months ago

Thanks, i'll close this for now but will look at creating this type of feature in the future.

FYI While researching I saw some discussion about adding multithreaded support for WASM: https://github.com/dotnet/runtime/issues/68162#issuecomment-1915098496

There'll probably be caveats even when supported, but it's possible this will just work in time.