search

alibaba / havenask

Apache License 2.0
1.6k stars 302 forks source link

[BUG] 直写表配置了向量索引却没写入对应向量字段的时候引擎会反复重新加载 #222

Open dyuyang opened 11 months ago

dyuyang commented 11 months ago
  1. 当aitheta发现用户插入的向量数据格式不正确或者缺字段，且schema里面没有配置ignore_invalid_doc=true时，首先会抛出错误 https://github.com/alibaba/havenask/blob/1c4d5e243821556b1c0ed0f9462d0f132f887d02/aios/storage/indexlib/index/ann/aitheta2/AithetaMemIndexer.cpp#L290-L309
  2. 然后引发直写表停止构建，不能再commit https://github.com/alibaba/havenask/blob/1c4d5e243821556b1c0ed0f9462d0f132f887d02/aios/suez/table/DirectBuilder.cpp#L321-L323

https://github.com/alibaba/havenask/blob/1c4d5e243821556b1c0ed0f9462d0f132f887d02/aios/suez/table/DirectBuilder.cpp#L185-L188

  1. 进而引发直写表管理类释放当前searcher的leader地位 https://github.com/alibaba/havenask/blob/1c4d5e243821556b1c0ed0f9462d0f132f887d02/aios/suez/table/ReadWriteTableManager.cpp#L133-L136

  2. 可以从suez中的状态机看到，无论之前是什么状态，一旦触发放弃leader事件，都会引发searcher的对应事件 https://github.com/alibaba/havenask/blob/1c4d5e243821556b1c0ed0f9462d0f132f887d02/aios/suez/table/LoadDecisionMaker.cpp#L156-L166