search

kapicorp / kapitan

Generic templated configuration management for Kubernetes, Terraform and other things
https://kapitan.dev
Apache License 2.0
1.8k stars 197 forks source link

Large performance regression comparing to using jsonnet from command line #611

Closed joseffilzmaier closed 2 weeks ago

joseffilzmaier commented 3 years ago

Describe the bug/feature I use kube-prometheus as part of my kapitan project. When using time jsonnet -J ../../lib/kube-prometheus -m manifests monitoring.jsonnet to compile the project with plain jsonnet i do get a compile time of around 6 Seconds. If i, however use kapitan compile the compilation takes about 85 Seconds. This discrepancy is too large IMO and i would like to find out why the compilation takes so much longer using kapitan

To Reproduce Steps to reproduce the behavior: Integrate kube-prometheus as a kapitan component with the vendor directory inside libs run kapitan compile

Expected behavior The compilation with jsonnet using kapitan should also take around 6 Seconds

If it's a bug (please complete the following information):

sbarzowski commented 3 years ago

(Jsonnet dev here)

@joseffilzmaier Are you perhaps using https://github.com/google/go-jsonnet when calling Jsonnet from cmdline? IIUC Kapitan still uses cpp implementation under the hood, which is known to be way slower.

joseffilzmaier commented 3 years ago

@sbarzowski Hi, Thanks! - Yes, i'm using the golang version of jsonnet.

Is there any way to use the golang version of jsonnet within kapitan (or is it planned to be integrated)? The long compile time is quite annoying ...

ramaro commented 3 years ago

@joseffilzmaier thanks for reporting. Can you share a bit more details please: is this a regression - did you have better performance with a previous version of kapitan and what version? How many targets are you compiling and are you just compiling this one kube-prometheus component? Golang jsonnet requires a few non obvious changes to kapitan but is still work in progress...

joseffilzmaier commented 3 years ago

@ramaro Hi! I do not know if this is a regression - version 0.29.1 is the only one i tried. I am compiling 10 targets, but all of them compile in <1 sec except the kube-prometheus which takes ~85 secs. Also, if i just compile kube-prometheus with the --targets selector, the performance is the same

ramaro commented 3 years ago

@joseffilzmaier thanks! Yep, sounds like a cpp jsonnet issue, unfortunately. I'll let you know when a functional golang jsonnet version of kapitan is availabe in pre-release or even in a branch so you can try it out, if that helps?

joseffilzmaier commented 3 years ago

@ramaro Yeah, thanks, that would be nice! The functionality isn't lacking - however the long compile time is somewhat annoying. I'd subscribe as a tester if the go-jsonnet branch emerges :+1:

joulaud commented 3 years ago

cf.https://github.com/deepmind/kapitan/issues/144 also.

github-actions[bot] commented 2 weeks ago

This issue is stale because it has been open for 1 year with no activity. Remove the stale label or comment if this issue is still relevant for you. If not, please close it yourself.