Open gussmith23 opened 6 months ago
If we just cut out the logic ops, Lakeroad's results stay proportionally the same (if a little better), but Xilinx's of course get better too:
diff --git a/python/robustness_experiments.py b/python/robustness_experiments.py
index db6eb76..aaa5a90 100644
--- a/python/robustness_experiments.py
+++ b/python/robustness_experiments.py
@@ -823,6 +823,11 @@ def _visualize_succeeded_vs_failed_xilinx(
== 0
)
+ # Remove mul+logic benchmarks
+ df = df[~df["identifier"].str.match(".*mulxor.*", case=False)]
+ df = df[~df["identifier"].str.match(".*muland.*", case=False)]
+ df = df[~df["identifier"].str.match(".*mulor.*", case=False)]
+
Path(cleaned_data_filepath).parent.mkdir(parents=True, exist_ok=True)
df.to_csv(cleaned_data_filepath)
here's the full figure:
In light of all of this, it would be ideal if we could add more benchmarks -- it's likely that we are still much better at mapping than the above figure captures, if only we enumerated more valid benchmarks.
Also, it should be noted that, from a cursory glance, Vivado does not mul+logic onto a single DSP.
One question would be, how do we write the constraint in Lakeroad to outlaw this? Also, not a question for us, but why isn't this prevented in the DRC checks?
There is a single line in both manuals about this being the case. However, Lakeroad can successfully synthesize mul+logic designs, as it isn't outlawed by the simulation model. It would be worth checking whether this is a feature that's actually allowed in some cases, and Xilinx is just being defensive about people not using it, or if it truly doesn't work once flashed to an actual FPGA.