BitFunnel / mg4j-workbench

Java tools for evaluating BitFunnel performance compared to an mg4j baseline.
GNU Lesser General Public License v3.0
1 stars 2 forks source link

Support streams that include multiple segments #35

Closed jondgoodwin closed 6 years ago

jondgoodwin commented 6 years ago

It is possible for a document in a BitFunnel chunk to include multiple "segments" for the same stream id. For example, the BitFunnel filter -writer annotate option injects a shard term as a second segment of the text stream '00'.

This commit enables a chunk document to return a Reader that treats multiple segments for a given stream id as if they were a single concatenated InputStream.