NATS Jetstream: Ability to use `natsServerMonitoringEndpoint` directly to fetch metrics of a consumer

[debajyoti-truefoundry]

Proposal

Right now, the Consumer metrics are fetched using the direct URL of the NATS Consumer Leader Node.

Considering a situation where Keda ScaledObject and NATS are in two different K8s clusters, it may not be possible to expose the individual Nodes over the internet.

package main

import (
    "context"
    "encoding/json"
    "fmt"
    "time"

    "github.com/kedacore/keda/v2/pkg/scalers"
)

func main() {
    config := &scalers.ScalerConfig{
        GlobalHTTPTimeout: 2000 * time.Millisecond,
        TriggerMetadata: map[string]string{"account": "example",
            "activationLagThreshold":       "1",
            "consumer":                     "example-consumer",
            "lagThreshold":                 "1",
            "natsServerMonitoringEndpoint": "nats-metrics.example.org",
            "stream":                       "example",
            "useHttps":                     "true"},
    }
    scaler, err := scalers.NewNATSJetStreamScaler(config)
    if err != nil {
        panic(err)
    }
    ctx := context.TODO()

    for i := 1; i <= 5; i++ {
        m, _, err := scaler.GetMetricsAndActivity(ctx, "")
        if err != nil {
            panic(err)
        }
        j, _ := json.Marshal(m)
        if err != nil {
            panic(err)
        }
        fmt.Println(string(j))
    }

}

In the above scenario, it will try to get the metrics from https://nats-1.nats-metrics.example.org/jsz?acc=example&consumers=true&config=true URL, assuming nats-1 is the Consumer Leader.

The above API call fails as I only have https://nats-metrics.example.org exposed.

I see that with the following patch, I can get the Consumer metrics directly from the natsServerMonitoringEndpoint.

diff --git a/pkg/scalers/nats_jetstream_scaler.go b/pkg/scalers/nats_jetstream_scaler.go
index 3e4de2d4..a0c778ea 100644
--- a/pkg/scalers/nats_jetstream_scaler.go
+++ b/pkg/scalers/nats_jetstream_scaler.go
@@ -287,6 +287,7 @@ func (s *natsJetStreamScaler) getNATSJetstreamMonitoringData(ctx context.Context
                                        return err
                                }
                        }
+                      natsJetStreamMonitoringNodeURL = s.metadata.monitoringURL

                        jetStreamAccountResp, err = s.getNATSJetstreamMonitoringRequest(ctx, natsJetStreamMonitoringNodeURL)
                        if err != nil {

I understand that the Consumer Leader will have the most up-to-date metrics information. But is it possible to add a configuration in the Trigger Metadata, forcing the scaler code to use the natsServerMonitoringEndpoint directly? I would also like to know if this is a bad idea.

Use-Case

No response

Is this a feature you are interested in implementing yourself?

Maybe

Anything else?

No response

[zroubalik]

Hi this is definitely reasonable, we just need to be sure that we don't break setup for existing users.

@debajyoti-truefoundry are you willing to contribute this feature?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed due to inactivity.

[atif-saddique-deel]

I think this should not be closed as its still a valid issue