Separate 3rd party libraries into separate classpaths

[scholzj]

Today, Strimzi has the 3rd party libraries mechanism to add additional libraries to the Kafka classpath. They are downloaded separately and copied into the Kafka's libs directory when building the container images. This is used to add for example:

• Prometheus JMX Exporter
• Tracing libraries
• OAuth plugins
• and more

This seems to work fine. But one of the sideeffects to how this is designed is that all these libraries are always in the classpath even when they are not really used. For example, the Jaeger client and its dependencies are in the Kafka classpath even when tracing is not enabled. It does not affect the functionality, but for example in case of CVEs, having affected libraries int he classpath makes it harder to decide whether the CVE impacts Strimzi or not.

Maybe, we can change this mechanism to download the different 3rd party libraries into different folders and include them into the class path only when they are really needed. E.g. we would have a structure like this:

├── LICENSE
├── 3rd-party
│   ├── jmx-exporter
│   ├── cruise-control
│   ├── oauth
│   └── jaeger-tracing
├── bin
├── config
├── libs
├── licenses
└── site-docs

And in the shell scripts reponsible for the Kafka startup, we would include these subdirectories into the classpath as required. E.g. when metrics are enabled, 3rd-party/jmx-exporter will be added to the classpath. When Cruise Control is enabled, 3rd-party/cruise-control will be added to the classpath. (Adding to the classpath would mean just configuring the variables, not copying the files.) This would help to reduce number of libraries in the classpath to be the minimum needed by the user and might make the deployment more secure.

To implement this, we wound need to think through:

• How complicated will this mechanism be
• How much does it really help (e.g. would security people be really happier about it? Or would they not care as long as the library is still present in the image?)

[scholzj]

Triaged on 25.8.2022: Seems useful, but there might be some edge cases to deal with. For example, what if a multiple plugins share the same dependencies? They would need to be there multiple times and we would need to make sure they are in the same version for example to avoid some conflicts. We should have a proposal to clarify all these possible issues and mechanisms to protect against them before an actual implementation PR.

[aidan-melen]

https://github.com/strimzi/strimzi-kafka-operator/issues/5473