[Console] loadJSONSpecInDir() takes 250ms to run at Kibana startup

[pmuellr]

While doing some CPU profiling of Kibana startup, I noticed that the loadJSONSpecInDir() function took 250ms to run.

https://github.com/elastic/kibana/blob/51f6eeccefca5b927f96727be39181e0c4119275/src/plugins/console/server/services/spec_definitions_service.ts#L118-L143

At first I thought it might be the readFileSync(), but it doesn't appear to be, nor the globby calls that don't even show up in the profile. Thinking it might be the reduce() call with the spread, which can be problematic, in terms of performance.

[elasticmachine]

Pinging @elastic/platform-deployment-management (Team:Deployment Management)

[alisonelizabeth]

Hi @pmuellr, thanks for bringing this to our attention. We will try to take a look.

[pgayvallet]

FWIW, I observed the same things while performing some deep startup time profiling this week.

On our benchmark machine, it's ~ 70ms (which translates to ~ 150ms to 200ms on serverless)

Most of the time is spent in the internal readFileUtf8 method.

Given we own the tool that generates those JSON specs, I wonder if generating a single file with all the specs instead of one file per spec might speed the loading time (I assume less file handlers and a single file access may be faster)

[rudolf]

@pgayvallet correct me if I'm wrong... in terms of prioritization for the management team, it does not feel like 200ms startup time improvement is enough to justify spending a lot of time on this. So unless we can make an improvement in a day's of work it probably would not be worth it.

[pgayvallet]

Yeah that's correct. I mostly replied to the issue for the record and to keep track of the timings I found when I did my profiling.

Unless we can think of a quick win to reduce this json parsing duration, we shouldn't spend time on this right now