azure-contrib / AzureDirectory

A Lucene Directory Provider for Azure Blob Storage
Microsoft Public License
77 stars 57 forks source link

Subfolders in directory structure brake the search index #42

Open sstraus opened 3 years ago

sstraus commented 3 years ago

I found a strange behavior using subfolders. My structure has a single container with many subfolders, one for each user. The first time the subfolders is created and filled everything works fine. But once reloaded, initializing the azure directory again, the index is empty. This doesn’t happen when I use root folder instead. I checked the paths in the lib but everything looks correct.

Any suggestion?

thank you.

richorama commented 3 years ago

interesting - I haven't seen this before, but if I get some time I'll try to track it down.

sstraus commented 3 years ago

Thank you, I'm going crazy. A sample to replicate the problem, with sub = null works, any other value will reset the index.

` static void ErrorTest(string sub = null) {

     var azureDirectory = new AzureDirectory(
         "DefaultEndpointsProtocol=xxx",
         "cachedir", "test", false, sub);

     var indexWriterConfig = new IndexWriterConfig(
         LuceneVersion.LUCENE_48,
         new StandardAnalyzer(LuceneVersion.LUCENE_48));

     var indexWriter = new IndexWriter(azureDirectory, indexWriterConfig);
     Document doc = new Document();
     doc.Add(new TextField("id", DateTime.Now.ToString(CultureInfo.InvariantCulture), Field.Store.YES));
     indexWriter.AddDocument(doc);
     indexWriter.Commit();
     var nd1 = indexWriter.NumDocs;
     indexWriter.Dispose();
     azureDirectory.Dispose();

     var indexWriterConfig2 = new IndexWriterConfig(
         LuceneVersion.LUCENE_48,
         new StandardAnalyzer(LuceneVersion.LUCENE_48));

     var azureDirectory2 = new AzureDirectory(
         "DefaultEndpointsProtocol=xxx",
         "cachedir", "test", false, sub); 

     var indexWriter2 = new IndexWriter(azureDirectory2, indexWriterConfig2);
     var nd2 = indexWriter2.NumDocs;
     indexWriter2.Dispose();
     azureDirectory2.Dispose();
     Console.WriteLine(nd1 == nd2);
 }`