search

apache / lucene

Apache Lucene open-source search software
https://lucene.apache.org/
Apache License 2.0
2.64k stars 1.02k forks source link

Autogenerated ID for LeafReaderContexts Within An IndexSearcher [LUCENE-8798] #9842

Open asfimport opened 5 years ago

asfimport commented 5 years ago

It would be good to be able to uniquely identify LeafReaderContext objects associated within a single IndexSearcher. This would allow storing of metadata around segments, such as demonstrated in #1954

The ID will be unique across the IndexSearcher instance and will make no guarantees of any semantic value outside the instance.

Migrated from LUCENE-8798 by Atri Sharma (@atris)

asfimport commented 5 years ago

Michael Sokolov (@msokolov) (migrated from JIRA)

@atris I glanced at the issue you referenced, but I don't see how it relates to this. Could you sketch out a use case where this would be better than using the ordinals that we use today to identify segments? Would these be persisted?

asfimport commented 5 years ago

Atri Sharma (@atris) (migrated from JIRA)

@msokolov Yes, as I commented in the other JIRA, this idea does not directly relate to the CollectorScorer patch, but came as an outcome of reading through it.

 

This is more of a readability and clarity perspective proposal – considering that LeafReaderContext allows initialization without assigning a specific ordinal, there should be an unique identifier which will be valid for any constructed LeafReaderContext – irrespective of which constructor initialized it.

 

My primary use case is to allow a secure way of storing in memory runtime metadata about segments during a search execution, hence the invariant that the ID has no relevance outside the scope of the owning IndexSearcher. Making persistent IDs that get written to segment blocks is an interesting idea, but is not what I planned to originally propose.

asfimport commented 5 years ago

Michael Sokolov (@msokolov) (migrated from JIRA)

I think what confused me was the link to the other JIRA seems to have a typo - it links to an ancient unrelated issue, but I guess you meant to link to one of your recent ones?

asfimport commented 5 years ago

Atri Sharma (@atris) (migrated from JIRA)

@msokolov Yes, sorry about that.