edamame-load-test / edamame

A distributed load testing framework for real-time collaboration apps (HTTP and WebSockets)
5 stars 1 forks source link

Adding Commander to manage CLI commands #64

Closed elguaposalsero closed 1 year ago

elguaposalsero commented 1 year ago

Added commander to manage CLI commands. Also added some error handling to the getTestIds function.

elguaposalsero commented 1 year ago

Resolves #27

gcpinckert commented 1 year ago

I think there might be a bug in the SQL queries for Grafana, although I'm not entirely sure where. When I run a test, I am no longer seeing results for any metrics other than request duration (99th percentile). There's no reported statistics for number of VUs, number of requests made, or HTTP failures.

When I exec into the postgres pod, these values are all reflected in the database, so the problem is happening in the communication between postgres and Grafana.

When I check the logs for the Grafana pod, I am getting a number of error messages:

2023-03-14 23:58:14.900 UTC [1] LOG:  database system is ready to accept connections
2023-03-14 23:59:39.959 UTC [76] ERROR:  syntax error at or near "AND" at character 142
2023-03-14 23:59:39.959 UTC [76] STATEMENT:  SELECT ts AS "time",
        SUM(value)
        FROM samples
        WHERE
          ts BETWEEN '2023-03-14T00:51:34.767Z' AND '2023-03-14T00:55:35.787Z'
          AND test_id =
          AND metric = 'counts.http_reqs'
        GROUP BY 1
        ORDER BY 1 ASC
2023-03-14 23:59:39.961 UTC [77] ERROR:  syntax error at or near "AND" at character 143
2023-03-14 23:59:39.961 UTC [77] STATEMENT:  SELECT ts AS time,
        metric,
        value
        FROM samples
        WHERE
          ts BETWEEN '2023-03-14T00:51:34.767Z' AND '2023-03-14T00:55:35.787Z'
          AND test_id =
          AND metric IN ('vus', 'counts.http_reqs')
2023-03-14 23:59:39.964 UTC [78] ERROR:  syntax error at or near "AND" at character 142
2023-03-14 23:59:39.964 UTC [78] STATEMENT:  SELECT ts AS "time",
        MAX(value)
        FROM samples
        WHERE
          ts BETWEEN '2023-03-14T00:51:34.767Z' AND '2023-03-14T00:55:35.787Z'
          AND test_id =
          AND metric = 'counts.counts.http_req_failed'
        GROUP BY 1
        ORDER BY 1 ASC
2023-03-14 23:59:44.670 UTC [75] ERROR:  syntax error at or near "AND" at character 142
2023-03-14 23:59:44.670 UTC [75] STATEMENT:  SELECT ts AS "time",
        SUM(value)
        FROM samples
        WHERE
          ts BETWEEN '2023-03-14T00:51:34.767Z' AND '2023-03-14T00:55:35.787Z'
          AND test_id =
          AND metric = 'counts.http_reqs'
        GROUP BY 1
        ORDER BY 1 ASC
2023-03-14 23:59:44.672 UTC [75] ERROR:  syntax error at or near "AND" at character 142
2023-03-14 23:59:44.672 UTC [75] STATEMENT:  SELECT ts AS "time",
        MAX(value)
        FROM samples
        WHERE
          ts BETWEEN '2023-03-14T00:51:34.767Z' AND '2023-03-14T00:55:35.787Z'
          AND test_id =
          AND metric = 'counts.counts.http_req_failed'
        GROUP BY 1
        ORDER BY 1 ASC
2023-03-14 23:59:44.673 UTC [75] ERROR:  syntax error at or near "AND" at character 143
2023-03-14 23:59:44.673 UTC [75] STATEMENT:  SELECT ts AS time,
        metric,
        value
        FROM samples
        WHERE
          ts BETWEEN '2023-03-14T00:51:34.767Z' AND '2023-03-14T00:55:35.787Z'
          AND test_id =
          AND metric IN ('vus', 'counts.http_reqs')
2023-03-14 23:59:49.679 UTC [76] ERROR:  syntax error at or near "AND" at character 142
2023-03-14 23:59:49.679 UTC [76] STATEMENT:  SELECT ts AS "time",
        SUM(value)
        FROM samples
        WHERE
          ts BETWEEN '2023-03-14T00:51:34.767Z' AND '2023-03-14T00:55:35.787Z'
          AND test_id =
          AND metric = 'counts.http_reqs'
        GROUP BY 1
        ORDER BY 1 ASC
2023-03-14 23:59:49.680 UTC [76] ERROR:  syntax error at or near "AND" at character 142
2023-03-14 23:59:49.680 UTC [76] STATEMENT:  SELECT ts AS "time",
        MAX(value)
        FROM samples
        WHERE
          ts BETWEEN '2023-03-14T00:51:34.767Z' AND '2023-03-14T00:55:35.787Z'
          AND test_id =
          AND metric = 'counts.counts.http_req_failed'
        GROUP BY 1
        ORDER BY 1 ASC
2023-03-14 23:59:49.684 UTC [75] ERROR:  syntax error at or near "AND" at character 143
2023-03-14 23:59:49.684 UTC [75] STATEMENT:  SELECT ts AS time,
        metric,
        value
        FROM samples
        WHERE
          ts BETWEEN '2023-03-14T00:51:34.767Z' AND '2023-03-14T00:55:35.787Z'
          AND test_id =
          AND metric IN ('vus', 'counts.http_reqs')

Any ideas what the issue might be?

gcpinckert commented 1 year ago

Hey @elguaposalsero would you mind taking a look at the conflicts here and then merging this into main? Now that we've determined the issue is coming from elsewhere? Thank you!