Mattihew / k6-to-junit

tool to convert k6 output to junit xml
18 stars 6 forks source link

Support for group duration threshold #2

Closed burtoni closed 4 years ago

burtoni commented 4 years ago

Hi, I can't get group duration thresholds to appear as a test case in the JUnit output, despite appearing as a failed check in the k6 output.

I've set up a group based on https://k6.io/docs/using-k6/thresholds#copy-paste-threshold-examples:

// Thresholds on global error rate and response time, and group duration for specific group
thresholds: {
    errors: ["rate==0.0"],    // 0% errors
    'http_req_duration': ['p(90)<100'],
    'group_duration{group:::get_user}': ['p(90) < 50'], 
  }

group('get_user', function() {
  // Do stuff
});

The group fails the response time validation in the k6 output:

    group_duration.......................: avg=157.66ms min=34.99ms med=184.26ms max=395.97ms p(90)=278.91ms p(95)=305.41ms
    ✗ { group:::get_user }...............: avg=250.91ms min=189ms   med=242.64ms max=395.97ms p(90)=305.54ms p(95)=332.19ms

However, only the errors and http_req_duration thresholds have test cases created; the group duration is omitted.

Is group syntax supported?

Mattihew commented 4 years ago

appears to be an issue with the regex i'm using to detect the failed lines. the colon(:) character is messing it up. should be a simple fix, will try and get this done today after i've finished work.

burtoni commented 4 years ago

I had a dig as well and thought that was what it might be. I’ll have a look and see whether there are any other cases that trigger it eg tags.

Mattihew commented 4 years ago

should be fixed now in version 1.0.3 @burtoni let me know if this works for you now aswell

burtoni commented 4 years ago

Confirmed this works for both tags (name=get_user) and groups (group:::get_transfer_reasons). Thanks for the quick turnaround!

    <testcase name="{ group:::get_transfer_reasons }">
      <failure message="    ✗ { group:::get_transfer_reasons }...: avg=101.46ms min=53.83ms med=70.8ms   max=348ms    p(90)=204.87ms p(95)=287.26ms">    ✗ { group:::get_transfer_reasons }...: avg=101.46ms min=53.83ms med=70.8ms   max=348ms    p(90)=204.87ms p(95)=287.26ms</failure>
      <system-out>    ✗ { group:::get_transfer_reasons }...: avg=101.46ms min=53.83ms med=70.8ms   max=348ms    p(90)=204.87ms p(95)=287.26ms</system-out>
    </testcase>
...
    <testcase name="{ name:get_user_request }">
      <failure message="    ✗ { name:get_user_request }..........: avg=69.85ms  min=48.1ms  med=64.27ms  max=130.89ms p(90)=98.27ms  p(95)=104.89ms">    ✗ { name:get_user_request }..........: avg=69.85ms  min=48.1ms  med=64.27ms  max=130.89ms p(90)=98.27ms  p(95)=104.89ms</failure>
      <system-out>    ✗ { name:get_user_request }..........: avg=69.85ms  min=48.1ms  med=64.27ms  max=130.89ms p(90)=98.27ms  p(95)=104.89ms</system-out>
    </testcase>