I've observed that the CPU utilization from the Kafka provider pod has been steadily growing over time. I've observed this in three different configurations:
provider -> mTLS proxy -> Kafka
provider w/ mTLS -> Kafka
provider w/o mTLS -> Kafka
I recently took a look at the metrics, and I can see the the number of goroutines is increasing over time, which I believe explains the CPU issues.
Interestingly, with the mTLS proxy in place, it also had growth in CPU usage over time. I believe this suggests that the goroutine leak involves the Kafka connection. I haven't done enough of a deep dive to identify the issue yet though.
How can we reproduce it?
As far as I can tell, you just have to use this provider-kafka component and take a look at the number of goroutines or CPU utilization. I don't think there are any special steps for repro.
What environment did it happen in?
Crossplane version: 1.11.0-rc.0.195.g3ce766b3
provider-kafka: xpkg.upbound.io/crossplane-contrib/provider-kafka:v0.2.0 (I also saw this behavior with 0.1.0)
What happened?
I've observed that the CPU utilization from the Kafka provider pod has been steadily growing over time. I've observed this in three different configurations:
I recently took a look at the metrics, and I can see the the number of goroutines is increasing over time, which I believe explains the CPU issues.
Interestingly, with the mTLS proxy in place, it also had growth in CPU usage over time. I believe this suggests that the goroutine leak involves the Kafka connection. I haven't done enough of a deep dive to identify the issue yet though.
How can we reproduce it?
As far as I can tell, you just have to use this provider-kafka component and take a look at the number of goroutines or CPU utilization. I don't think there are any special steps for repro.
What environment did it happen in?
Crossplane version: 1.11.0-rc.0.195.g3ce766b3 provider-kafka: xpkg.upbound.io/crossplane-contrib/provider-kafka:v0.2.0 (I also saw this behavior with 0.1.0)