I suggest a few improvements after reviewing the latest Cloud APM changes:
1 - Export by default the http_req_failed metric.
When a Cloud APM integration is enabled, the Default APM metrics are exported by default. The current list of metrics are: data_sent, data_received, http_req_duration, http_reqs, iterations, and vus.
Starting v0.31, k6 reports the http_req_failed metric that provides the rate of failed requests.
http_req_failed is an important metric, and it would be good to be added to the Default APM metrics list to be sent by default to Cloud APM tools.
2 - Add a new includeCustomMetrics option. Default true.
The Cloud APM feature provides two settings - includeDefaultMetrics and metrics - to control which metrics will be exported to the APM provider.
Given these settings, custom metrics are not exported by default. At this moment, the user has to include each custom metric name in the metrics array option. I think that is not the best DX.
I suggest providing an includeCustomMetrics option that works like includeDefaultMetrics for all the custom metrics.
Enabling the includeCustomMetrics option will automatically export all the custom metrics, making the work of exporting custom metrics easier and less error-prone. Finally, I suggest enabling it by default.
3 - Enable Cloud APM integration per project to automatically export test result metrics
With the new update, users can enable a Cloud APM integration for an organization.
Today, the user must enable the integration for each test - either with the test builder or in the k6 script.
Ideally, an admin/manager could enable the APM integration for one or multiple projects. Any test running within those projects will automatically export the metrics to the configured APM solution.
I think this feature could improve the process to configure Cloud APM tests significantly:
Zero-configuration for new tests. Users don't need to set up the APM setting for each test.
Other testers or developers scripting their tests might not know the APM settings: credentials, URLs, options, etc.
It encourages creating scopes for particular projects.
I suggest a few improvements after reviewing the latest Cloud APM changes:
1 - Export by default the
http_req_failed
metric.When a Cloud APM integration is enabled, the
Default APM metrics
are exported by default. The current list of metrics are:data_sent
,data_received
,http_req_duration
,http_reqs
,iterations
, andvus
.Starting v0.31, k6 reports the
http_req_failed
metric that provides the rate of failed requests.http_req_failed
is an important metric, and it would be good to be added to theDefault APM metrics
list to be sent by default to Cloud APM tools.2 - Add a new
includeCustomMetrics
option. Default true.The Cloud APM feature provides two settings -
includeDefaultMetrics
andmetrics
- to control which metrics will be exported to the APM provider.Given these settings, custom metrics are not exported by default. At this moment, the user has to include each custom metric name in the
metrics
array option. I think that is not the best DX.I suggest providing an
includeCustomMetrics
option that works likeincludeDefaultMetrics
for all the custom metrics.Enabling the
includeCustomMetrics
option will automatically export all the custom metrics, making the work of exporting custom metrics easier and less error-prone. Finally, I suggest enabling it by default.3 - Enable Cloud APM integration per project to automatically export test result metrics
With the new update, users can enable a Cloud APM integration for an organization.
Today, the user must enable the integration for each test - either with the test builder or in the k6 script.
Ideally, an admin/manager could enable the APM integration for one or multiple projects. Any test running within those projects will automatically export the metrics to the configured APM solution.
I think this feature could improve the process to configure Cloud APM tests significantly: