search

cucapra / calyx-evaluation

4 stars 3 forks source link

Bug in systolic array data processing #3

Closed sgpthomas closed 3 years ago

sgpthomas commented 3 years ago

In this file, there is a bug with how we process the systolic array data.

We gathered data on gemm sizes for 1x1 to 8x8. However, we only report the even sizes: 2x2, 4x4, 6x6, 8x8. We implement this by filtering out the unwanted data from the pandas dataframe.

# Remove systolic array data for which VHLS implementation failed
systolic_data = systolic_data.drop([0, 4, 6]).reset_index(drop=True)
systolic_data['latency'] = systolic_lat['latency']

The bug with this code is that we filter the data frame before merging it. Pandas doesn't merge by keys, so it merged the wrong latency information for each benchmark name. For example the benchmark 4x4 got the latency for 3x3.

rachitnigam commented 3 years ago

We caught this bug during ASPLOS 2021 artifact evaluation and fixed it.