Which index type does blast used? upside_down or scroch??

[Peipeilvcm]

It seems that scroch index type can be lighter(refer to bleve/issues/1186), which costs smaller index size.

If blast uses upside_down index type by default, how can I change to use scroch index type?

Thanks a lot.

[mosuka]

@Peipeilvcm Blast already supports scorch. You can use scorch for the index storage backend with the following command:

$ ./bin/blast indexer start \
    --grpc-address=:5000 \
    --grpc-gateway-address=:6000 \
    --http-address=:8000 \
    --node-id=indexer1 \
    --node-address=:2000 \
    --data-dir=/tmp/blast/indexer1 \
    --raft-storage-type=boltdb \
    --index-mapping-file=./example/wiki_index_mapping.json \
    --index-type=scorch \
    --index-storage-type=scorch

[Peipeilvcm]

@mosuka Blast already supports scorch. You can use scorch for the index storage backend with the following command:

$ ./bin/blast indexer start \
    --grpc-address=:5000 \
    --grpc-gateway-address=:6000 \
    --http-address=:8000 \
    --node-id=indexer1 \
    --node-address=:2000 \
    --data-dir=/tmp/blast/indexer1 \
    --raft-storage-type=boltdb \
    --index-mapping-file=./example/wiki_index_mapping.json \
    --index-type=scorch \
    --index-storage-type=scorch

Does index-storage-type need to be setted as scorch, I set it "boltdb" and work well,

[mosuka]

Hi @Peipeilvcm When upside_down is specified for --index_type, --index-storage-type is enabled. As you point out, if you specify scorch specified for --index_type, actually --index-storage-type will ignored. However, since bleve.newIndexUsing is called in index creation, an error will occur if --index-storage-type (kvstore) is omitted, so it is necessary to specify some string for convenience. Therefore I specified scorch for --index-storage-type .

https://github.com/blevesearch/bleve/blob/0bb7fb280a4cc7fa691add6809fdf0d9273f726a/index_impl.go#L75-L77

When specifying scorch for --index-type, modify so that --index-storage-type can be omitted.