Preload catalog at build time for native executables

[apupier]

Is your feature request related to a problem? Please describe.

Currently on start of Kaoto backend, the embedded catalogs are taking several seconds (between 4 and 5 seconds usually on my machine).

Describe the solution you'd like

it would be nice if the warmup is done at build time so that it takes few millliseconds to warmup.

Describe alternatives you've considered

Additional context

It is especially interesting in context of VS Code where the Kaoto backend is started on each VS Code start. When Kaoto editor are automatically opened it will allow a faster startup. (it currently needs to await warmup before openign otherwise the whole content is wiped out)

[Delawen]

Interesting. My tests were down to milliseconds but seems my machine is too fast then :)

[apupier]

for reference, logs of startup with native binary for Linux on my machine:

__  ____  __  _____   ___  __ ____  ______ 
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ 
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/   
2023-03-06 10:57:45,800 INFO  [io.kao.bac.api.met.cat.AbstractCatalog] (ForkJoinPool.commonPool-worker-1) Parser class io.kaoto.backend.metadata.parser.JarParseCatalog processed in 13ms.
2023-03-06 10:57:45,800 INFO  [io.kao.bac.api.met.cat.AbstractCatalog] (ForkJoinPool.commonPool-worker-1) Catalog class io.kaoto.backend.api.metadata.catalog.ViewDefinitionCatalog_Subclass warmed up in 13ms.
2023-03-06 10:57:48,980 INFO  [io.kao.bac.api.met.cat.AbstractCatalog] (ForkJoinPool.commonPool-worker-3) Parser class io.kaoto.backend.metadata.parser.JarParseCatalog processed in 4ms.
2023-03-06 10:57:49,031 INFO  [io.quarkus] (main) kaoto-backend 0.7.0 native (powered by Quarkus 2.15.2.Final) started in 3.299s. Listening on: http://0.0.0.0:8081
2023-03-06 10:57:49,031 INFO  [io.quarkus] (main) Profile prod activated. 
2023-03-06 10:57:49,031 INFO  [io.quarkus] (main) Installed features: [cdi, config-yaml, jgit, jsch, kubernetes-client, micrometer, opentelemetry, opentelemetry-otlp-exporter, quartz, resteasy-reactive, resteasy-reactive-jackson, scheduler, smallrye-context-propagation, smallrye-openapi, vertx]
2023-03-06 10:57:49,045 INFO  [io.kao.bac.api.met.cat.AbstractCatalog] (ForkJoinPool.commonPool-worker-1) Parser class io.kaoto.backend.metadata.parser.JarParseCatalog processed in 69ms.
2023-03-06 10:57:49,151 INFO  [io.kao.bac.api.met.cat.AbstractCatalog] (ForkJoinPool.commonPool-worker-7) Parser class io.kaoto.backend.metadata.parser.JarParseCatalog processed in 175ms.
2023-03-06 10:57:49,218 INFO  [io.kao.bac.api.met.cat.AbstractCatalog] (ForkJoinPool.commonPool-worker-4) Parser class io.kaoto.backend.metadata.parser.JarParseCatalog processed in 242ms.
2023-03-06 10:57:49,218 INFO  [io.kao.bac.api.met.cat.AbstractCatalog] (ForkJoinPool.commonPool-worker-4) Catalog class io.kaoto.backend.api.metadata.catalog.StepCatalog_Subclass warmed up in 3341ms.

[apupier]

(base) [apupier@localhost kaoto-ui]$ docker run --rm -p 8080:8081 --name kaoto kaotoio/standalone:0.7.2
__  ____  __  _____   ___  __ ____  ______ 
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ 
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/   
2023-03-13 12:17:17,563 WARN  [io.fab.kub.cli.Config] (main) Error reading service account token from: [/var/run/secrets/kubernetes.io/serviceaccount/token]. Ignoring.
2023-03-13 12:17:17,564 WARN  [io.fab.kub.cli.Config] (main) Error reading service account token from: [/var/run/secrets/kubernetes.io/serviceaccount/token]. Ignoring.
2023-03-13 12:17:32,724 INFO  [io.quarkus] (main) kaoto-backend 0.7.2 native (powered by Quarkus 2.15.2.Final) started in 15.185s. Listening on: http://0.0.0.0:8081
2023-03-13 12:17:32,724 INFO  [io.quarkus] (main) Profile prod activated. 
2023-03-13 12:17:32,724 INFO  [io.quarkus] (main) Installed features: [cdi, config-yaml, jgit, jsch, kubernetes-client, micrometer, opentelemetry, opentelemetry-otlp-exporter, quartz, resteasy-reactive, resteasy-reactive-jackson, scheduler, smallrye-context-propagation, smallrye-openapi, vertx]
2023-03-13 12:17:32,732 INFO  [io.kao.bac.api.met.cat.AbstractCatalog] (ForkJoinPool.commonPool-worker-3) Parser class io.kaoto.backend.metadata.parser.JarParseCatalog processed in 32ms.
2023-03-13 12:17:32,735 INFO  [io.kao.bac.api.met.cat.AbstractCatalog] (ForkJoinPool.commonPool-worker-6) Parser class io.kaoto.backend.metadata.parser.JarParseCatalog processed in 18ms.
2023-03-13 12:17:32,735 INFO  [io.kao.bac.api.met.cat.AbstractCatalog] (ForkJoinPool.commonPool-worker-6) Catalog class io.kaoto.backend.api.metadata.catalog.ViewDefinitionCatalog_Subclass warmed up in 19ms.
2023-03-13 12:17:32,768 INFO  [io.kao.bac.api.met.cat.AbstractCatalog] (ForkJoinPool.commonPool-worker-2) Parser class io.kaoto.backend.metadata.parser.JarParseCatalog processed in 68ms.
2023-03-13 12:17:32,852 INFO  [io.kao.bac.api.met.cat.AbstractCatalog] (ForkJoinPool.commonPool-worker-1) Parser class io.kaoto.backend.metadata.parser.JarParseCatalog processed in 153ms.
2023-03-13 12:17:32,873 INFO  [io.kao.bac.api.met.cat.AbstractCatalog] (ForkJoinPool.commonPool-worker-4) Parser class io.kaoto.backend.metadata.parser.JarParseCatalog processed in 174ms.
2023-03-13 12:17:32,873 INFO  [io.kao.bac.api.met.cat.AbstractCatalog] (ForkJoinPool.commonPool-worker-4) Catalog class io.kaoto.backend.api.metadata.catalog.StepCatalog_Subclass warmed up in 15308ms.
2023-03-13 12:17:33,193 WARN  [io.qua.ope.run.exp.otl.LateBoundBatchSpanProcessor] (vert.x-eventloop-thread-1) No BatchSpanProcessor delegate specified, no action taken.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Delawen]

Looking at doing a quarkus extension or https://quarkus.io/guides/lifecycle#what-is-the-difference-from-initializedapplicationscoped-class-and-destroyedapplicationscoped-class

[lburgazzoli]

@Delawen @apupier I can take this one if no one is already working on it

[lburgazzoli]

can anyone assign it to me ? or grant me the right to self assign stuffs ?