I've been developing an API using elasticsearch and their golang client, upon trying to do a Search request using the v8 TypedClient (and normal client) I get unexpected behavior, this mainly consists of extremely high CPU usage on all nodes and high latency (20+ seconds), my cluster is using elasticsearch v8.15.2.
high CPU usage (essentially using all resources on both nodes)
The strange part of this is when I recreate the same request using the REST API directly (say with curl, or even in go with http), the latency is 10ms-300ms, which is to be expected.
simple test code that shows the difference between everything
it is also worth noting that the query itself is what's taking long, in the screenshot i highlight the took value of the client, the Search request took 20 seconds.
And of course here is the aggregation I'm trying to run, it is grouping all documents by a field 'sort' (this field is mapped), there are a large amount of documents (10b+) but the cluster is running on three powerful servers, the aggregation returns ~800 documents.
I think it's safe to rule out the aggregation itself for this performance issue because when I run it through the API directly (like in the screenshot) there is no performance issue, it only has issues when I run it through a client.
and here is the code for performing the Search through the typed API, this code would be the code displaying the info that I've highlighted as pink in the previous screenshot.
If there are any questions feel free to reply and I'll reply to them ASAP!
Hello!
I've been developing an API using elasticsearch and their golang client, upon trying to do a Search request using the v8 TypedClient (and normal client) I get unexpected behavior, this mainly consists of extremely high CPU usage on all nodes and high latency (20+ seconds), my cluster is using elasticsearch v8.15.2.
high CPU usage (essentially using all resources on both nodes)
The strange part of this is when I recreate the same request using the REST API directly (say with curl, or even in go with http), the latency is 10ms-300ms, which is to be expected.
simple test code that shows the difference between everything
it is also worth noting that the query itself is what's taking long, in the screenshot i highlight the
tookvalue of the client, the Search request took 20 seconds.And of course here is the aggregation I'm trying to run, it is grouping all documents by a field 'sort' (this field is mapped), there are a large amount of documents (10b+) but the cluster is running on three powerful servers, the aggregation returns ~800 documents. I think it's safe to rule out the aggregation itself for this performance issue because when I run it through the API directly (like in the screenshot) there is no performance issue, it only has issues when I run it through a client.
and here is the code for performing the Search through the typed API, this code would be the code displaying the info that I've highlighted as pink in the previous screenshot.
If there are any questions feel free to reply and I'll reply to them ASAP!
Thank you for your time!!