Update for Kibana 8.3.3

[ndmitch311]

Include in this update receiving and passing along query parameters for logging and tracking:

-X-Opaque-ID -User info data

[maihde]

The X-Opaque-Id is a value that is passed into Elasticsearch queries by the client that allows you to correlated specific client requests with specific queries/tasks within Elasticsearch. In the ElasticSearch client this can be passed on each search (https://github.com/elastic/elasticsearch-py/issues/847).

The intention is that each tile request would generate a new X-Opaque-Id so that if we see a hung-task or a slow query we can directly track it down to the specific tile request in the Datashader .datashader_tiles index. This will require that X-Opaque-Id be stored in .datashader_tiles so that it can be easily cross-referenced.

[desean1625]

Ok, I was thinking that this was something like x-proxied-entities to track who is running queries.

Would it make sense to set it to something like sha1 of the url, and log the url/hash so we could look up and recreate the exact query?

sha_1 = hashlib.sha1()
sha_1.update(url)
query_hash = sha_1.hexdigest()
queryid = "DS-"+query_hash 
logger.info(queryid,url)
es = Elasticsearch(["http://localhost:9200"], headers={"x-opaque-id": queryid})

[maihde]

I don't have a preference on how the opaque-id is created, but there are two considerations:

1. The user who ran the query, the URL, and the full query are stored in .datashader_tiles already
2. Using the hash will cause two identical requests to have the same opaque-id, so if two identical URLs are accessed we cannot differentiate them.

[desean1625]

Looks like the user information on who ran the query isn't in the .datashader_tiles. Nothing from the header is in there. I can add that also if we want. right now I have the x-opaque-id at

"params.x-opaque-id": [
    "e83066df-fe66-4a27-a508-de8f85d9bf99"
  ],

I can also move the x-opaque-id up to the document root if we have a preference.

[desean1625]

Fixed and merged.