matteofigus / api-benchmark

A node.js tool to benchmark APIs
MIT License
311 stars 35 forks source link

In "compare" mode HTML output only displays stats of first server #30

Closed pensierinmusica closed 4 years ago

pensierinmusica commented 9 years ago

Hi,

I'm using this code:

'use strict';

var fs = require('fs');
var open = require('open');
var apiBenchmark = require('api-benchmark');

var services = {
  server1: "http://localhost:4000/",
  server2: "http://localhost:9999/"
};

var routes = { route1: '/create/email' };

var opts = {
  runMode: 'parallel',
  minSamples: 5000
}

apiBenchmark.compare(services, routes, opts, function(err, results){
  apiBenchmark.getHtml(results, function(error, html){
    fs.writeFile('./output.html', html, function () {
      open('./output.html');
    });
  });
});

The HTML output opens correctly in the browser, but it only displays stats from "server1". Is this the expected behavior? Shouldn't it compare the performance of both (possibly wuth different colored lines)? Also, the "Request details" and "Response details" tabs don't seem to work in this case.

screenshot

Thanks!

oharlem commented 9 years ago

same here

dmyers commented 9 years ago

Yeah you are correct, currently only the first service is shown on the chart. This would certainly be a great pull request and has also been mentioned in #16.

arufian commented 5 years ago

Is this still unresolved ? I got the same issue, plus I couldn't go to the other menu beside stats. I got this error on my console log.

benchmark.html:175 Uncaught TypeError: Cannot read property 'end' of undefined
    at Object.route (benchmark.html:175)
    at HTMLDocument.<anonymous> (benchmark.html:294)
    at l (jquery.min.js:4)
    at Object.fireWith [as resolveWith] (jquery.min.js:4)
    at Function.ready (jquery.min.js:4)
    at HTMLDocument.S (jquery.min.js:4)
stollr commented 5 years ago

I have the same problem as @arufian. The problem ist, that the benchmark stats are rendered into the result html like this:

// this is only an extract showing the issue
var data={
    "benchmark":{
        "server1":{
            "api_method_1":{
                // sevel information, stats and so on
                "options": {"method":"post","concurrencyLevel":1,"start":"2019-01-25T09:47:27.828Z","end":"2019-01-25T09:47:39.315Z"} 
                "isSlowest":true
            },
            "api_method_2":{
                // sevel information, stats and so on
                "options": {} 
                "isSlowest":true
            },
            "isSlowest":true // <-- THIS IS WRONG
        },
        "server2":{
            "api_method_1":{
                // sevel information, stats and so on
                "options": {"method":"post","concurrencyLevel":1,"start":"2019-01-25T09:47:39.415Z","end":"2019-01-25T09:47:42.853Z"} 
                "isFastest":true
            },
            "api_method_2":{
                // sevel information, stats and so on
                "options": {} 
                "isSlowest":true
            },
            "isFastest":true // <-- THIS IS WRONG
        }
    },
};

For each route the report tries to access the options property. But the problem is, that there are the isSlowest/isFastest properties on the same level where the route stats are. And this produces the error @arufian mentioned.

matteofigus commented 5 years ago

I think this is still unresolved. Happy if anyone wants to attempt a Pull Request and/or Review existing ones :)