dosubot[bot]
commented
3 weeks ago @yw1991 Hello there! I'm a bot here to help you with any bugs, questions, or contributions while you wait for a human maintainer.
The error elasticsearch.BadRequestError: BadRequestError(400, 'parsing_exception', 'Unknown key for a START_OBJECT in [rank].' suggests an issue with the way the rank field is being handled in the Elasticsearch query. This is likely related to the hybrid mode in the AsyncDenseVectorStrategy.
To resolve this issue, ensure that the rank field is correctly mapped and recognized by Elasticsearch. Here is a potential fix:
- Ensure the rank field is correctly mapped in your Elasticsearch index.
- Modify the code to handle the rank field properly.
Check and potentially modify the following code snippet:
top_k_nodes = []
top_k_ids = []
top_k_scores = []
for hit in hits:
source = hit["_source"]
metadata = source.get("metadata", None)
text = source.get(self.text_field, None)
node_id = hit["_id"]
try:
node = metadata_dict_to_node(metadata)
node.text = text
except Exception:
# Legacy support for old metadata format
logger.warning(
f"Could not parse metadata from hit {hit['_source']['metadata']}"
)
node_info = source.get("node_info")
relationships = source.get("relationships", {})
start_char_idx = None
end_char_idx = None
if isinstance(node_info, dict):
start_char_idx = node_info.get("start", None)
end_char_idx = node_info.get("end", None)
node = TextNode(
text=text,
metadata=metadata,
id_=node_id,
start_char_idx=start_char_idx,
end_char_idx=end_char_idx,
relationships=relationships,
)
top_k_nodes.append(node)
top_k_ids.append(node_id)
top_k_scores.append(hit.get("_rank", hit["_score"]))
if (
isinstance(self.retrieval_strategy, AsyncDenseVectorStrategy)
and self.retrieval_strategy.hybrid
):
total_rank = sum(top_k_scores)
top_k_scores = [total_rank - rank / total_rank for rank in top_k_scores]
return VectorStoreQueryResult(
nodes=top_k_nodes,
ids=top_k_ids,
similarities=_to_llama_similarities(top_k_scores),
)Ensure the _rank field is correctly mapped and recognized by Elasticsearch. If _rank is not valid, use a different field or adjust the score calculations.
Additionally, ensure the mode matches the retrieval strategy. For AsyncDenseVectorStrategy(hybrid=True), the mode should be VectorStoreQueryMode.HYBRID. Here is a test case to ensure the mode matches the retrieval strategy:
def test_mode_must_match_retrieval_strategy() -> None:
# DEFAULT mode should never raise any exception
mode = VectorStoreQueryMode.DEFAULT
retrieval_strategy = AsyncBM25Strategy()
_mode_must_match_retrieval_strategy(mode, retrieval_strategy)
# AsyncSparseVectorStrategy with mode SPARSE should not raise any exception
mode = VectorStoreQueryMode.SPARSE
retrieval_strategy = AsyncSparseVectorStrategy()
_mode_must_match_retrieval_strategy(mode, retrieval_strategy)
# AsyncBM25Strategy with TEXT_SEARCH should not raise any exception
mode = VectorStoreQueryMode.TEXT_SEARCH
retrieval_strategy = AsyncBM25Strategy()
_mode_must_match_retrieval_strategy(mode, retrieval_strategy)
# AsyncDenseVectorStrategy(hybrid=True) with mode HYBRID should not raise any exception
mode = VectorStoreQueryMode.HYBRID
retrieval_strategy = AsyncDenseVectorStrategy(hybrid=True)
_mode_must_match_retrieval_strategy(mode, retrieval_strategy)
# unknown mode should raise NotImplementedError
for mode in [
VectorStoreQueryMode.SEMANTIC_HYBRID,
VectorStoreQueryMode.SVM,
VectorStoreQueryMode.LOGISTIC_REGRESSION,
VectorStoreQueryMode.LINEAR_REGRESSION,
VectorStoreQueryMode.MMR,
]:
retrieval_strategy = AsyncDenseVectorStrategy()
with pytest.raises(NotImplementedError):
_mode_must_match_retrieval_strategy(mode, retrieval_strategy)
# if mode is SPARSE and strategy is not AsyncSparseVectorStrategy, should raise ValueError
mode = VectorStoreQueryMode.SPARSE
retrieval_strategy = AsyncDenseVectorStrategy()
with pytest.raises(ValueError):
_mode_must_match_retrieval_strategy(mode, retrieval_strategy)
# if mode is HYBRID and strategy is not AsyncDenseVectorStrategy, should raise ValueError
mode = VectorStoreQueryMode.HYBRID
retrieval_strategy = AsyncSparseVectorStrategy()
with pytest.raises(ValueError):
_mode_must_match_retrieval_strategy(mode, retrieval_strategy)
# if mode is HYBRID and strategy is AsyncDenseVectorStrategy but hybrid is not enabled, should raise ValueError
mode = VectorStoreQueryMode.HYBRID
retrieval_strategy = AsyncDenseVectorStrategy(hybrid=False)
with pytest.raises(ValueError):
_mode_must_match_retrieval_strategy(mode, retrieval_strategy)Ensure your mode and retrieval strategy are correctly aligned to avoid such errors [1][2].
To continue talking to Dosu, mention @dosu.
yw1991
Documentation Issue Description
elasticsearch.BadRequestError: BadRequestError(400, 'parsing_exception', 'Unknown key for a START_OBJECT in [rank].'
Documentation Link
*