Blank graphs in cross-run-ID comparison

GMLC-TDC / helics_benchmark_results

Repo containing helics_benchmark results and processing code

BSD 3-Clause "New" or "Revised" License

0 stars 0 forks source link

Blank graphs in cross-run-ID comparison #6

Closed trevorhardy closed 4 years ago

trevorhardy commented 4 years ago

The current version of the cross-run-ID comparison code ("cross_run_id.py") generates blank graphs for the following comparisons using our test case (Zu60n and zUZF6):

echoBenchmark - core_type = "unspecified"
messageSendBenchmark
- message_size, core_type = "inproc"
- message_count, core_type = "inproc"
- message_size, core_type = "ipc"
- message_count, core_type = "ipc"
- message_size, core_type = "singleFed"
- message_count, core_type = "singleFed"
- message_size, core_type = "singleCore"
- message_count, core_type = "singleCore"
- message_size, core_type = "unspecified"
- message_count, core_type = "unspecified"
- message_size, core_type = "zmq"
- message_count, core_type = "zmq"
- message_size, core_type = "zmqss"
- message_count, core_type = "zmqss"
ringBenchmark
- core_type = singleCore
- core_type = unspecified

trevorhardy commented 4 years ago

@corinnegroth it looks like there is still an inconsistency in the naming of core types: "singleFed" and "singleCore" are both showing up. I've created a separate issue (#7 ) and will be working that one myself.

corinnegroth commented 4 years ago

@trevorhardy we discovered that earlier because there was a name inconsistency in the data itself (I believe). Did that get fixed?

trevorhardy commented 4 years ago

@corinnegroth I thought I had but apparently not. It should be a quick fix.

trevorhardy commented 4 years ago

I've cleaned up some of these by pre-filtering the dataframe to only results of the given benchmark prior to iterating over the core types to generate the graphs. There are still some blank graphs (all "messageSendBenchmark") that needs to be looked into.

trevorhardy commented 4 years ago

For reasons that are somewhat mysterious, the pre-filtering I added and the existing filtering of the dataframe in the graphing functions were making a null set for the messageSend benchmarks. Corinne removed hers from the graphing functions and that caused all the graphs for messageSend to be generated.

We also found a special case in ringBenchmark where only a single point is tested for singleCore. This was causing the line graph to be blank. We added logic to check for singleCore and generated a bar graph instead to compare that single point.