Log4j incompatible with recent kubernetes-client

Amya28 commented 10 months ago

Description

While using the latest log4j release version ; we encountered the following error :

ERROR StatusConsoleListener Unable to create Lookup for k8s java.lang.NoSuchMethodError: 'io.fabric8.kubernetes.client.ConfigFluent io.fabric8.kubernetes.client.ConfigBuilder.withRollingTimeout(long)'

Debugging we found that in latest version of kubernetes-client-api they removed the getRollingTimeout calls which is still referred in latest version of log4j-Kubernetes jar.

https://github.com/fabric8io/kubernetes-client/commit/f12e14b2b9cebe38efae60a910d849767fd45cb4

Even tough the Release notes https://github.com/apache/logging-log4j2/releases specify the Kubernetes client to version 5.12.4

Can anyone have a look at it. we need to specify the version explicitly for kubernetes client under log4j pom.xmls.

[A clear and concise description of what the bug is.]

Configuration

Version: [[2.22.0] (https://github.com/apache/logging-log4j2/releases/tag/rel%2F2.22.0)]

Operating system: [Windows 10]

JDK: [17]

Logs


[Stacktraces, errors, etc. relevant applications logs.]

ERROR StatusConsoleListener Unable to create Lookup for k8s
 java.lang.NoSuchMethodError: 'io.fabric8.kubernetes.client.ConfigFluent io.fabric8.kubernetes.client.ConfigBuilder.withRollingTimeout(long)'
        at org.apache.logging.log4j.kubernetes.KubernetesClientBuilder.kubernetesClientConfig(KubernetesClientBuilder.java:69)
        at org.apache.logging.log4j.kubernetes.KubernetesClientBuilder.createClient(KubernetesClientBuilder.java:35)
        at org.apache.logging.log4j.kubernetes.KubernetesLookup.initialize(KubernetesLookup.java:89)
        at org.apache.logging.log4j.kubernetes.KubernetesLookup.<init>(KubernetesLookup.java:70)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
        at org.apache.logging.log4j.core.util.ReflectionUtil.instantiate(ReflectionUtil.java:189)
        at org.apache.logging.log4j.core.lookup.Interpolator.<init>(Interpolator.java:86)
        at org.apache.logging.log4j.core.lookup.Interpolator.<init>(Interpolator.java:105)
        at org.apache.logging.log4j.core.config.AbstractConfiguration.<init>(AbstractConfiguration.java:135)
        at org.apache.logging.log4j.core.config.DefaultConfiguration.<init>(DefaultConfiguration.java:46)
        at org.apache.logging.log4j.core.LoggerContext.<init>(LoggerContext.java:84)
        at org.apache.logging.log4j.core.selector.ClassLoaderContextSelector.createContext(ClassLoaderContextSelector.java:254)
        at org.apache.logging.log4j.core.selector.ClassLoaderContextSelector.locateContext(ClassLoaderContextSelector.java:218)
        at org.apache.logging.log4j.core.selector.ClassLoaderContextSelector.getContext(ClassLoaderContextSelector.java:136)
        at org.apache.logging.log4j.core.selector.ClassLoaderContextSelector.getContext(ClassLoaderContextSelector.java:123)
        at org.apache.logging.log4j.core.selector.ClassLoaderContextSelector.getContext(ClassLoaderContextSelector.java:117)
        at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:150)
        at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:47)
        at org.apache.logging.log4j.LogManager.getContext(LogManager.java:196)
        at org.apache.commons.logging.LogAdapter$Log4jLog.<clinit>(LogAdapter.java:146)
        at org.apache.commons.logging.LogAdapter$Log4jAdapter.createLog(LogAdapter.java:113)
        at org.apache.commons.logging.LogAdapter.createLog(LogAdapter.java:95)
        at org.apache.commons.logging.LogFactoryService.getInstance(LogFactoryService.java:52)
        at org.apache.commons.logging.LogFactoryService.getInstance(LogFactoryService.java:47)
        at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:655)

## Reproduction

[An isolated test reproducing the test.
JUnit tests similar to the ones in the code base are extremely appreciated.]

Amya28 commented 10 months ago

https://github.com/apache/logging-log4j2/blob/rel/2.22.1/log4j-kubernetes/pom.xml

This doesnot specify the kubernetes-client version hence, it is fetching to the latest version of kubernetes-client : 6.9.2

ppkarwasz commented 10 months ago

@Amya28,

thank you for the issue report. Currently log4j-kubernetes is compiled against version 5.12.4:

https://github.com/apache/logging-log4j2/blob/8469975a4f2b1f8f1bd4f25ca6d1989a52aefc1b/log4j-parent/pom.xml#L124

but it compiles fine up to version 6.1.1 of the Kubernetes libraries.

Kubernetes seems to have performed several breaking API changes in minor releases without waiting for a major release:

in version 6.2.0 the method io.fabric8.kubernetes.api.model.ObjectMeta#getClusterName disappeared,
as you noticed in version 6.5.0 the methods io.fabric8.kubernetes.client.Config#getRollingTimeout and io.fabric8.kubernetes.client.ConfigBuilder#withRollingTimeout disappeared.

I have reported the problem upstream.

Amya28 commented 10 months ago

@Amya28,

thank you for the issue report. Currently log4j-kubernetes is compiled against version 5.12.4:

https://github.com/apache/logging-log4j2/blob/8469975a4f2b1f8f1bd4f25ca6d1989a52aefc1b/log4j-parent/pom.xml#L124

but it compiles fine up to version 6.1.1 of the Kubernetes libraries.

Kubernetes seems to have performed several breaking API changes in minor releases without waiting for a major release:

in version 6.2.0 the method io.fabric8.kubernetes.api.model.ObjectMeta#getClusterName disappeared,

as you noticed in version 6.5.0 the methods io.fabric8.kubernetes.client.Config#getRollingTimeout and io.fabric8.kubernetes.client.ConfigBuilder#withRollingTimeout disappeared.

I have reported the problem upstream.

Thanks @ppkarwasz for the prompt reply. Yes, Kubernetes have performed several breaking API changes in their latest release.

But it does not seem that log4j-kubernetes is compiled against version 5.12.4 The pom.xml of log4j-kubernetes does not specify the version tag and hence it is fetching the latest 'kubernetes-client' version.

Dependency tree:


[INFO] +- org.apache.logging.log4j:log4j-kubernetes:jar:2.21.1:compile
[INFO] |  \- io.fabric8:kubernetes-client:jar:6.9.2:compile
[INFO] |     +- io.fabric8:kubernetes-client-api:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-core:jar:6.9.2:compile
[INFO] |     |  |  \- io.fabric8:kubernetes-model-common:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-gatewayapi:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-resource:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-rbac:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-admissionregistration:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-apps:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-autoscaling:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-apiextensions:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-batch:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-certificates:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-coordination:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-discovery:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-events:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-extensions:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-flowcontrol:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-networking:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-metrics:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-policy:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-scheduling:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-storageclass:jar:6.9.2:compile
[INFO] |     |  +- io.fabric8:kubernetes-model-node:jar:6.9.2:compile

We need to use the<kubernetes-client.version>5.12.4</kubernetes-client.version> throughout.

ppkarwasz commented 10 months ago

@Amya28,

In Maven you need to specify an artifact's version, it throws an error if you don't. The reason you don't see it in log4j-kubernetes/pom.xml is that we define it in log4j-parent/pom.xml dependency management section.

Your project can of course override that version, which is probably what happens.

vy commented 10 months ago

@Amya28, are there any particular reasons you choose KubernetesLookup over exporting associated Kubernetes configuration to environment variables and accessing them using EnvironmentLookup, e.g., ${env:KUBERNETES_CLUSTER_NAME}?

I am curious about your use case for following reasons:

We considered dropping KubernetesLookup in Log4j 3.0.0
Using EnvironmentLookup helps you avoid any kubernetes-client dependencies
Judging from my personal experience, injecting K8s config into apps through env vars is a common industry practice

Hence, I want to understand your rationale for deviating from this.

ppkarwasz commented 10 months ago

@vy,

After the explanations provided by @manusa in fabric8io/kubernetes-client#5682, I think that the best solution would be to either move log4j-kubernetes to a Fabric8 repo or to a repo, where both our teams have access.

The lifecycle of log4j-kubernetes should clearly follow the lifecycle of kubernetes-client instead of log4j-core.

vy commented 10 months ago

@ppkarwasz, yes, I have seen fabric8io/kubernetes-client#5682 discussion. Thanks for reporting the issue to the upstream. I agree that if kubernetes-client wants to adopt log4j-kubernetes, they should.

Yet I still want to understand the use case of @Amya28. Because, maybe, there should not be a KubernetesLookup at all? I don't feel comfortable with moving the maintenance responsibility of a component that should not have existed in the first place on to somebody else's shoulders.

Amya28 commented 10 months ago

@ppkarwasz @vy There's no KubernetesLookup.

In our case, we are working with spring boot and through Spring cloud the kubernetes version is set to latest 6.9.2 by framework which is causing above mentioned issue.

ppkarwasz commented 10 months ago

@Amya28,

If you don't use the KubernetesLookup (i.e. you don't have ${k8s:...} placeholders in your config file), you can remove the log4j-kubernetes dependency, since it does not provide anything else.

ppkarwasz commented 7 months ago

I am closing this, since a new kubernetes-log4j lookup is available since version 6.11.0 of Kubernetes Client (cf. announcement).

apache / logging-log4j2