Improve data plugin performance

elastic / kibana

Your window into the Elastic Stack

https://www.elastic.co/products/kibana

Other

19.6k stars 8.22k forks source link

Improve data plugin performance #84923

Open mshustov opened 3 years ago

mshustov commented 3 years ago

The data plugin, being one of the most fundamental plugins in the Kibana ecosystem, is the critical part of improving the overall Kibana performance.

Network

If you run Kibana with APM enabled (see instruction), you can notice that the plugin performs some amount of duplicated network requests, which can be optimized: 2020-12-03_13-52-08 2020-12-03_14-30-36

Note: I've got these numbers running Kibana and Elasticsearch locally - the network latency might be even higher on Cloud.

Code

re-use API created by the core. https://github.com/elastic/kibana/blob/b3430e3f09deac0ed3e577d4f27ef7f9865f2b01/src/plugins/data/server/search/search_service.ts#L243-L255 here RequestHandlerContext can be used.
extend RequestHandlerContext with lazy instantiated API https://github.com/elastic/kibana/blob/bc8a1dac99af3ea16bbcf5f2825d0f90724fa3b3/src/core/server/core_route_handler_context.ts#L32-L58

elasticmachine commented 3 years ago

Pinging @elastic/kibana-app-services (Team:AppServices)

lukeelmers commented 3 years ago

here RequestHandlerContext can be used

Awhile back there was a discussion around avoiding using RequestHandlerContext in cases like this because of some of the complexities it creates for folks authenticating via API keys (for example alerting would need to not only generate a fake KibanaRequest to create an alert that relies on the search service, but now also a fake RequestHandlerContext).

However, performance did not come up as a potential issue in that discussion.

@streamich has suggested a caching layer in the data plugin's services. Another option would be to go down the path of restructuring everything to accept scoped services, which was an alternative approach proposed in that thread.

lukeelmers commented 3 years ago

Also on the same topic of data plugin performance, there's this related issue about whether we can further decrease the page load bundle size: https://github.com/elastic/kibana/issues/84788

lizozom commented 2 years ago

@lukeelmers @mshustov

This issue has been without activity for a while now. Is it still relevant? Can it be made more actionable? If it is releveant, we can add it to the Kibana Performance project and groom it.

mshustov commented 2 years ago

Is it still relevant? Can it be made more actionable?

It's on AppServices team to answer these questions as they own data plugin

elasticmachine commented 1 year ago

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)