opensearch-project / user-behavior-insights

User Behavior Insights plugin for OpenSearch
Apache License 2.0
20 stars 5 forks source link

[FEATURE] Don't include UBI query details in the indexed `query` field #32

Closed jzonthemtn closed 3 days ago

jzonthemtn commented 1 month ago

When the plugin saves a query, it grabs the original query and persists it, too. This will include ubi information in the request that does not serve a purpose and just takes up space in the destination store.

      {
        "_index": "ubi_queries",
        "_score": 1.0,
        "_source": {
          "query_response_id": "39a29e90-324e-43f5-b449-7f79aee3fde7",
          "user_query": "",
          "query_id": "49140554-b9ae-4f12-825e-81bc73f140a8",
          "query_response_object_ids": [],
          "query": "{\"query\":{\"match_all\":{\"boost\":1.0}},\"ext\":{\"query_id\":\"49140554-b9ae-4f12-825e-81bc73f140a8\",\"user_query\":null,\"client_id\":null,\"object_id_field\":null,\"query_attributes\":{}}}",
          "query_attributes": {},
          "client_id": "",

This issue is to remove the ubi attributes from the query prior to storing it.

sandeshkr419 commented 1 month ago

[Search Triage] There seems some relevance in storing both the original and transformed query. Is this information redundant and already present elsewhere? @jzonthemtn Can you please add more details if so.

jzonthemtn commented 1 month ago

Yes, keep storing the original query and the user query, but remove this part from the query:

"ext\":{\"query_id\":\"49140554-b9ae-4f12-825e-81bc73f140a8\",\"user_query\":null,\"client_id\":null,\"object_id_field\":null,\"query_attributes\":{}}

This is all duplicate information that's already stored in the other properties:

          "user_query": "",
          "query_id": "49140554-b9ae-4f12-825e-81bc73f140a8",
          "query_response_object_ids": [],
          "query_attributes": {},
          "client_id": "",

So we can remove it to save space and not lose any understanding.

epugh commented 3 days ago

This has been completed!