search

hobbit-project / platform

HOBBIT benchmarking platform
GNU General Public License v2.0
23 stars 9 forks source link

Experiment results GUI does not show benchmark parameters #11

Closed MichaelRoeder closed 7 years ago

MichaelRoeder commented 7 years ago

From the JSON transferred to the GUI it can be seen that the GUI backend does not send the needed information:

[{
        "benchmark": {
            "description": "This benchmark emulates the traffic created by benchmarking a SPARQL store to determine the maximal throughput of the platform.",
            "id": "http://w3id.org/hobbit/platform-benchmark/vocab#PlatformBenchmark",
            "name": "HOBBIT Platform Benchmark"
        },
        "id": "1487869724763",
        "kpis": [{
                "description": "The average time from the moment a query is sent to the benchmarked system until its response is received.",
                "id": "http://w3id.org/hobbit/platform-benchmark/vocab#msPerQuery",
                "name": "Average query runtime (in ms)",
                "datatype": "xsd:double",
                "range": "http://www.w3.org/2001/XMLSchema#double",
                "value": "59274.730430000003253"
            }, {
                "description": "Average number of processed queries (=result sets) that are received by the evaluation storage per second.",
                "id": "http://w3id.org/hobbit/platform-benchmark/vocab#responsesPerSecond",
                "name": "Queries per second",
                "datatype": "xsd:double",
                "range": "http://www.w3.org/2001/XMLSchema#double",
                "value": "802.16263045169773704"
            }, {
                "description": "The number of errors identified by either missing expected responses or missing result sets or both.",
                "id": "http://w3id.org/hobbit/platform-benchmark/vocab#errorCount",
                "name": "Error count",
                "datatype": "xsd:integer",
                "range": "http://www.w3.org/2001/XMLSchema#int",
                "value": "0"
            }, {
                "description": "The overall runtime, i.e., the time from the first query sent to the last response received.",
                "id": "http://w3id.org/hobbit/platform-benchmark/vocab#runtime",
                "name": "Runtime",
                "datatype": "xsd:string",
                "range": "http://www.w3.org/2001/XMLSchema#integer",
                "value": "124850"
            }, {
                "description": "The standard deviation of the time between the moment a query is sent to the benchmarked system and the point in time at which its response is received.",
                "id": "http://w3id.org/hobbit/platform-benchmark/vocab#queryRuntimeStdDev",
                "name": "Query runtime standard deviation",
                "datatype": "xsd:double",
                "range": "http://www.w3.org/2001/XMLSchema#double",
                "value": "31312.219011738958216"
            }
        ],
        "system": {
            "description": "The system needed to run the platform benchmark.",
            "id": "http://w3id.org/hobbit/platform-benchmark/vocab#PlatformBenchmarkSystem",
            "name": "HOBBIT Platform Benchmark System"
        }
    }
]

This feature is added in the https://github.com/hobbit-project/platform/tree/feature-bench-param-in-exp-result-gui branch

MichaelRoeder commented 7 years ago

In the feature branch https://github.com/hobbit-project/platform/tree/feature-bench-param-in-exp-result-gui the gui-serverbackend sends the needed data to the gui-client.

The JSON of an example experiment is

[  
   {  
      "benchmark":{  
         "description":"This benchmark emulates the traffic created by benchmarking a SPARQL store to determine the maximal throughput of the platform.",
         "id":"http://w3id.org/hobbit/platform-benchmark/vocab#PlatformBenchmark",
         "name":"HOBBIT Platform Benchmark",
         "configurationParamNames":[  
            "Number of queries",
            "Number of data generators",
            "Number of task generators",
            "Seed"
         ],
         "configurationParams":[  
            {  
               "description":"The number of SPARQL queries that are sent.",
               "id":"http://w3id.org/hobbit/platform-benchmark/vocab#numberOfQueries",
               "name":"Number of queries",
               "datatype":"xsd:unsignedInt",
               "defaultValue":"1000",
               "feature":true,
               "range":"http://www.w3.org/2001/XMLSchema#unsignedInt",
               "required":false
            },
            {  
               "description":"The number of data generators that will be created.",
               "id":"http://w3id.org/hobbit/platform-benchmark/vocab#numberOfDataGenerators",
               "name":"Number of data generators",
               "datatype":"xsd:unsignedInt",
               "defaultValue":"2",
               "feature":true,
               "range":"http://www.w3.org/2001/XMLSchema#unsignedInt",
               "required":false
            },
            {  
               "description":"The number of task generators that will be created.",
               "id":"http://w3id.org/hobbit/platform-benchmark/vocab#numberOfTaskGenerators",
               "name":"Number of task generators",
               "datatype":"xsd:unsignedInt",
               "defaultValue":"1",
               "feature":true,
               "range":"http://www.w3.org/2001/XMLSchema#unsignedInt",
               "required":false
            },
            {  
               "description":"The seed of the random number generators used by this bechmark.",
               "id":"http://w3id.org/hobbit/platform-benchmark/vocab#seed",
               "name":"Seed",
               "datatype":"xsd:string",
               "defaultValue":"31",
               "feature":false,
               "range":"http://www.w3.org/2001/XMLSchema#integer",
               "required":false
            }
         ],
         "configurationParamValues":[  
            {  
               "description":"The number of task generators that will be created.",
               "id":"http://w3id.org/hobbit/platform-benchmark/vocab#numberOfTaskGenerators",
               "name":"Number of task generators",
               "datatype":"xsd:unsignedInt",
               "range":"http://www.w3.org/2001/XMLSchema#unsignedInt",
               "value":"1"
            },
            {  
               "description":"The number of data generators that will be created.",
               "id":"http://w3id.org/hobbit/platform-benchmark/vocab#numberOfDataGenerators",
               "name":"Number of data generators",
               "datatype":"xsd:unsignedInt",
               "range":"http://www.w3.org/2001/XMLSchema#unsignedInt",
               "value":"2"
            },
            {  
               "description":"The number of SPARQL queries that are sent.",
               "id":"http://w3id.org/hobbit/platform-benchmark/vocab#numberOfQueries",
               "name":"Number of queries",
               "datatype":"xsd:unsignedInt",
               "range":"http://www.w3.org/2001/XMLSchema#unsignedInt",
               "value":"1000"
            },
            {  
               "description":"The seed of the random number generators used by this bechmark.",
               "id":"http://w3id.org/hobbit/platform-benchmark/vocab#seed",
               "name":"Seed",
               "datatype":"xsd:string",
               "range":"http://www.w3.org/2001/XMLSchema#integer",
               "value":"31"
            }
         ]
      },
      "id":"1487680214721",
      "kpis":[  
         {  
            "description":"The average time from the moment a query is sent to the benchmarked system until its response is received.",
            "id":"http://w3id.org/hobbit/platform-benchmark/vocab#msPerQuery",
            "name":"Average query runtime (in ms)",
            "datatype":"xsd:double",
            "range":"http://www.w3.org/2001/XMLSchema#double",
            "value":"17079.675999999999476"
         },
         {  
            "description":"Average number of processed queries (=result sets) that are received by the evaluation storage per second.",
            "id":"http://w3id.org/hobbit/platform-benchmark/vocab#responsesPerSecond",
            "name":"Queries per second",
            "datatype":"xsd:double",
            "range":"http://www.w3.org/2001/XMLSchema#double",
            "value":"41.689248342852380347"
         },
         {  
            "description":"The number of errors identified by either missing expected responses or missing result sets.",
            "id":"http://w3id.org/hobbit/platform-benchmark/vocab#errorCount",
            "name":"Error count",
            "datatype":"xsd:integer",
            "range":"http://www.w3.org/2001/XMLSchema#int",
            "value":"0"
         },
         {  
            "description":"The overall runtime, i.e., the time from the first query sent to the last response received.",
            "id":"http://w3id.org/hobbit/platform-benchmark/vocab#runtime",
            "name":"Runtime",
            "datatype":"xsd:string",
            "range":"http://www.w3.org/2001/XMLSchema#integer",
            "value":"24000"
         },
         {  
            "description":"The standard deviation of the time between the moment a query is sent to the benchmarked system and the point in time at which its response is received.",
            "id":"http://w3id.org/hobbit/platform-benchmark/vocab#queryRuntimeStdDev",
            "name":"Query runtime standard deviation",
            "datatype":"xsd:double",
            "range":"http://www.w3.org/2001/XMLSchema#double",
            "value":"3234.3673228351790385"
         }
      ],
      "system":{  
         "type":"systemBean",
         "description":"The system needed to run the platform benchmark.",
         "id":"http://w3id.org/hobbit/platform-benchmark/vocab#PlatformBenchmarkSystem",
         "name":"HOBBIT Platform Benchmark System"
      }
   }
]
MichaelRoeder commented 7 years ago

@romankorf the new version looks much better. When selecting one experiment, it shows the expected behaviour. singleresult

However, there seems to be a small bug when selecting several experiments. As shown in the screenshot, the parameters are listed multiple times although the parameters names/URIs are the same for all experiments. multipleresults

Unfortunately, I didn't had the time to check whether the same problem occurs with the KPIs.