artilleryio / artillery

The complete load testing platform. Everything you need for production-grade load tests. Serverless & distributed. Load test with Playwright. Load test HTTP APIs, GraphQL, WebSocket, and more. Use any Node.js module.
https://www.artillery.io
Mozilla Public License 2.0
7.9k stars 502 forks source link

Collect non 200 responses #190

Open danielo515 opened 7 years ago

danielo515 commented 7 years ago

Hello,

I'm not sure if this is currently possible, but if it is, please instruct me how. I want to see what uris failed with which statusCode, which for me is basically any status different from 200.

I'm currently trying with this:

module.exports.logErrors = function logArtilleryErrors(requestParams, response, context, ee, next) {
    if (response.statusCode !== 200) {
        ee.emit('nonOkResponse', {uri: requestParams.uri, statusCode: response.statusCode})
    }
    next();
}

But nothing appears on the log. What I'm doing wrong?

Regards

hassy commented 7 years ago

Not possible at the moment. What I've done before myself is use console.error() and then redirect stderr to a file when running Artillery with something like artillery run something.yaml 2>errors.txt

This will eventually be solved properly - there's an open issue here: https://github.com/shoreditch-ops/artillery/issues/161

danielo515 commented 7 years ago

I was fearing an answer like that. Thanks for the workaround by the way.

Logging histograms should be possible I'm curious about that already implemented feature. What does it refers to?

paraspatidar commented 5 years ago

This is something which might help to those who are looking for similar thing :
pre-requisite to log in file : npm install -g fs

testconfig.yaml -->

config:
  target: 'http://192.168.1.2'
  phases:
    - duration: 1
      arrivalRate: 1
  defaults:
    headers:
      x-my-service-auth: '987401838271002188298567'
  processor: "my-functions.js"
scenarios:
  - flow:
    - get:
        url: "/fnempty01"
        afterResponse: "logBody"

related my-functions.js in same folder as this config :

const fs = require('fs');

 module.exports = {
    setJSONBody: setJSONBody,
    logBody: logBody
  }

  function setJSONBody(requestParams, context, ee, next) {
    return next(); // MUST be called for the scenario to continue
  }

  function logBody(requestParams, response, context, ee, next) {
console.log(response.body);
      let body=response.body+"\r\n"; //this enabled new line in windows , use \n for linux
    fs.appendFile('temp.txt', body, function(err, body){
    if (err) console.log(err);
    //console.log("Successfully Written to File.");
    });
    return next(); // MUST be called for the scenario to continue
  }

now just do artillery run testconfig.yaml

this will log the response body in text file , same goes for headers based on implementation of logBody , same way you can write logHeaders.

hassy commented 5 years ago

There's no need to npm install the fs package - it's a built in module in Node.js.

Also bear in mind that writing to a file for every response will slow down your tests unless the RPS is very low.

Otherwise, it's a decent solution e.g. if you're running a single virtual user to help debug why something might be failing.

paraspatidar commented 5 years ago

@hassy , Yeah thats exactly is requirement to verify if something is failing then for debug purpose. as fiddler or any other tracing tool was unable to capture the same load test traffic which artillery is making. Can you help on that one on how to capture same traffic on standard tools like fiddler / wire-shark.

Or Shall I open any issue for separate documentation ?

hassy commented 5 years ago

@paraspatidar there's nothing special about how Artillery sends its traffic, so using Wireshark as normal should just work. It would be good to have an example blog post perhaps, but having it as part of the documentation would be out-of-scope for that reason.

CRathod1220 commented 1 year ago

Hey you can save the error log record in csv file format as well Step 1: Just install : npm i csv-writer

Step 2 : go to node_modules/artillery_plugin_expect/index.js file and add the below code in function(expectationsPluginCheckExpectations)

Changes in function

requestExpectations.results.forEach((e) => { if (e.ok) { events.emit('counter', plugins.expect.ok, 1); events.emit('counter', plugins.expect.ok.${e.type}, 1); } else { events.emit('counter', plugins.expect.failed, 1); events.emit('counter', plugins.expect.failed.${e.type}, 1);

 <-- add the below code in else part 
  console.log('error',res).  
  console.log('statusMessage',res.statusMessage) 
  console.log('statusCode',res.statusCode) 
  console.log('timings start',res.timings.start)
  console.log('timings end',res.timings.end) 
  console.log('requestUrl',res.requestUrl) 
  console.log('options',res.request.options.name) 
  console.log('options',res.request.options.url.origin) 

  const data = [       
    {
      host: res.request.options.url.origin,
      request: res.request.options.name,
      url:res.requestUrl,
      statusCode: res.statusCode,
      statusMessage: res.statusMessage
    }
  ];
  saveConsoleLogResult(data). <-- call the function to save error log in csv

}

});

// end here

Step 3 : And Then create function saveConsoleLogResult and add the below code

const createCsvWriter = require('csv-writer').createObjectCsvWriter;
const csvWriter = createCsvWriter({
path: 'result.csv',
header: [
  {id: 'host', title: 'Host'},
  {id: 'request', title: 'Request'},
  {id: 'url', title: 'URL'},
  {id: 'statusCode', title: 'StatusCode'},
  {id: 'statusMessage', title: 'Message'},
]
});

function saveConsoleLogResult(data){
csvWriter
.writeRecords(data)
.then(()=> console.log('The CSV file was written successfully'));
}