proxy-injector doesn't respect JSON logging configuration for all log messages

[woz5999]

What is the issue?

When JSON logging is enabled, the proxy-injector uses the proper JSON format for some log messages, but not all.

How can it be reproduced?

• Enable JSON logging for the proxy-injector (we use helm and set controllerLogFormat: json
• Ensure you had pods in your cluster that are configured NOT to be injected by the injector
• View proxy-injector logs

Logs, error output, etc

This is an example of mix-format log messages from a proxy-injector pod:

{"level":"info","msg":"received admission review request \"83a0ce4d-ab81-42c9-abe4-e0ade0f926e2\"","time":"2024-10-10T21:06:18Z"}
time="2024-10-10T21:06:18Z" level=info msg="received pod/mypod"
time="2024-10-10T21:06:18Z" level=info msg="skipped pod/mypod: neither the namespace nor the pod have the annotation \"linkerd.io/inject:enabled\""
{"level":"info","msg":"received admission review request \"c2321ce9-7c3d-4f37-86c7-33a4a337ded0\"","time":"2024-10-10T21:06:28Z"}
time="2024-10-10T21:06:28Z" level=info msg="received pod/mypod2"
time="2024-10-10T21:06:28Z" level=info msg="skipped pod/mypod2: neither the namespace nor the pod have the annotation \"linkerd.io/inject:enabled\""
{"level":"info","msg":"received admission review request \"ba4d8337-ec51-46d1-931e-610c067fbc58\"","time":"2024-10-10T21:08:11Z"}
time="2024-10-10T21:08:11Z" level=info msg="received pod/mypod3"
time="2024-10-10T21:08:11Z" level=info msg="injection patch generated for: pod/mypod3"

output of linkerd check -o short

linkerd-version
---------------
‼ cli is up-to-date
    unsupported version channel: stable-2.14.10
    see https://linkerd.io/2.14/checks/#l5d-version-cli for hints

control-plane-version
---------------------
‼ control plane and cli versions match
    control plane running edge-24.10.2 but cli running stable-2.14.10
    see https://linkerd.io/2.14/checks/#l5d-version-control for hints

linkerd-control-plane-proxy
---------------------------
‼ control plane proxies and cli versions match
    linkerd-destination-5cf894ff9c-5jshx running edge-24.10.2 but cli running stable-2.14.10
    see https://linkerd.io/2.14/checks/#l5d-cp-proxy-cli-version for hints

linkerd-ha-checks
-----------------
‼ pod injection disabled on kube-system
    kube-system namespace needs to have the label config.linkerd.io/admission-webhooks: disabled if injector webhook failure policy is Fail
    see https://linkerd.io/2.14/checks/#l5d-injection-disabled for hints

linkerd-viz
-----------
‼ viz extension proxies and cli versions match
    metrics-api-86dd7f89dd-n6dvp running edge-24.10.2 but cli running stable-2.14.10
    see https://linkerd.io/2.14/checks/#l5d-viz-proxy-cli-version for hints

Status check results are √

Environment

• Server Version: version.Info{Major:"1", Minor:"29+", GitVersion:"v1.29.8-eks-a737599"
• EKS
• Bottlerocket

Possible solution

No response

Additional context

No response

Would you like to work on fixing this bug?

None

[VISHESH0932]

Hey @woz5999 , I would love to work on this issue.