martymarron / xk6-output-prometheus-pushgateway

This is a k6 extension for publishing test-run metrics to Prometheus via Pushgateway.
Apache License 2.0
10 stars 5 forks source link

Errors after merge to tag v0.2.0 #3

Closed NorbertChrzaszcz closed 1 year ago

NorbertChrzaszcz commented 1 year ago

When you add new func (labels) my tests run but always get error. When I start them up, I immediately get an error display: level=error msg="Could not add to pushgateway" component=output-prometheus-pushgateway error="1008 error(s) occurred: collected metric \"checks\" { gauge: } was collected before with the same name and label values .... for all metrics. I deleted all custom metrics from my code and got same errors.

I set my dockerfile and set up build to tag v0.1.1 - tests and custom metrics work.

Additional question how labels work, if I want to have for example for custom metrics label named "aaa" to have the status of each request sent, it is possible to do this? Needs this for my grafana :)

martymarron commented 1 year ago

Hi @NorbertChrzaszcz, Thank you for reaching out, and sorry for bothering you after my updates.

As I tried to reproduce them, that sometimes occurs on my local. However, I haven't figured out the root cause yet. As a workaround, can you try specifying the value of K6_PUSH_INTERVAL smaller like 1s, 2s, ...etc? On my local, this error looks to be occurring for longer push intervals.

By the way, here is the execution commands w/ specifying labels, and example metrics sent observed on the Pushgateway.

Command:

K6_PUSHGATEWAY_URL=https://your.pushgateway.com:9091/ \
K6_LABEL_APP=myapp K6_LABEL_ENV=myenv \
k6 run \
-o output-prometheus-pushgateway \
script.js

Metrics on the Pushgateway: Screen Shot 2023-02-17 at 14 39 40

Hope this helps 😉

NorbertChrzaszcz commented 1 year ago

Hello @martymarron, thank you for fast answer. I have already managed to add the labels to my pushgateway but my question is a little different.

import http from "k6/http"
import { check, sleep} from 'k6';
// import { Counter } from 'k6/metrics';

export let options = {
    insecureSkipTLSVerify: true,
     ext: {
         "pushgateway": {
             status: " ",
         }
     },
    stages: [ 
        { duration: "1m", target: 100}, // simulate ramp-up of traffic from 1 to 100 users over 5 minutes.
        { duration: "1m", target: 100}, // stay at 100 users for 10 minutes
        { duration: "1m", target: 0}, // ramp_down to 0 users
    ]
}

 let failedRequestCount = new Counter("failed_request_count")

export default function () {
    const pages = [
        "page$&100",
    ]
    for (const page of pages) {
        const r = http.get("http://localhost:8000" + page);
        check(r, {
            "status was 200": (r) => r.status == 200,
            "duration was <= 1000ms": (r) => r.timings.duration <= 2000
        });

        let labelStatus = JSON.stringify(r.status)
        options.ext.pushgateway.status=labelStatus

        // build custom metrics
         if (r.status >= 400) { 
             failedRequestCount.add(1);
         }

        sleep(1);
    }
}

Why it doesn't work in this situation. I wanted to send status metrics depending on the request response. In prometheus, depending on the response, metrics are created with different statuses, which can then be translated into graphane to show the results. Example: image

I set K6_PUSH_INTERVAL=2s and check too for 1s and it still doesn't work :(

Have a nice day :)

martymarron commented 1 year ago

@NorbertChrzaszcz

Additional question how labels work, if I want to have for example for custom metrics label named "aaa" to have the status of each request sent, it is possible to do this? Needs this for my grafana :) I have already managed to add the labels to my pushgateway but my question is a little different.

Sorry, I misunderstood what you want. So far, this extension doesn't support configuring labels per request sent. Because the options.ext.pushgateway.status option is configured only at the start of the test execution.

It's very difficult for me to reserve time to support this capability. Hence I'll really appreciate your contribution if possible.

I set K6_PUSH_INTERVAL=2s and check too for 1s and it still doesn't work :(

I'll keep investigating this problem, and update you if any progress :)

martymarron commented 1 year ago

@NorbertChrzaszcz Allow me to close this issue because the reported issue below has been resolved.

When you add new func (labels) my tests run but always get error. When I start them up, I immediately get an error display: level=error msg="Could not add to pushgateway" component=output-prometheus-pushgateway error="1008 error(s) occurred: collected metric "checks" { gauge:value:1 } was collected before with the same name and label values .... for all metrics. I deleted all custom metrics from my code and got same errors.

If you really want the new feature mentioned in the previous comment, it would be appreciated to create a new issue for it 😊