search

verilog-to-routing / vtr-verilog-to-routing

Verilog to Routing -- Open Source CAD Flow for FPGA Research
https://verilogtorouting.org
Other
1.01k stars 388 forks source link

Add Koios benchmark QoR example to the vtr developer guide, and add koios to CI #1782

Open vaughnbetz opened 3 years ago

vaughnbetz commented 3 years ago

The Koios benchmarks have been integrated into the vtr repo, but to have the most impact, we should do two more things:

  1. Add a short description (and links to the benchmark README) to the VTR developer guide under the QoR Collection section: https://docs.verilogtorouting.org/en/latest/dev/developing/#collecting-qor-measurements It should come after the Titan benchmarks (or maybe after the VTR benchmarks if that flows better as they go all the way through odinII ...)_ and have a similar style.

  2. The vtr_reg_nightly4 suite that runs them should be added to CI. @ArashAhmadian can work with you to make the necessary changes and pin Tim Ansell to make the eventual google-side changes.

aman26kbm commented 3 years ago

Thanks for filing this, Vaughn.

I will add to task #2. I need to add protection of benchmarks without the "`define complex_dsp". And I need to update the documentation to mention how to run benchmarks with and without complex_dsp (the new include file method).

I will work on these within next week.

aman26kbm commented 3 years ago

I've created a new PR which includes the changes for task #1 mentioned in this issue.

@ArashAhmadian , can you please guide me on what changes to make changes for task #2? I'll make those changes and push them to this new PR I've created.

[Edit: Fix typo in task number]

ArashAhmadian commented 3 years ago

@aman26kbm Sure, do you mean changes to add vtr_reg_nightly_test4 to CI? Your previous PR actually added the .cfg file and made the necessary changes in vtr_test.sh so I think everything is good from our side.

aman26kbm commented 3 years ago

Right. That's great. (Sorry for the typo in my previous comment. I meant to point to task #2 in the question I asked :P. Fixing it now) So, that means the Koios tests have been running in nightly regressions over the last week?

ArashAhmadian commented 3 years ago

I don't see them being run. I think @mithro needs to add the configs on the Google side of things for them to start running.

aman26kbm commented 3 years ago

Ok. Thanks. We can discuss this with @mithro in this week's VTR meeting on Thursday.

ArashAhmadian commented 3 years ago

Sounds good.

ArashAhmadian commented 3 years ago

@aman26kbm What's your email? I am drafting the email to Arnaud and I can put you on it.

vaughnbetz commented 3 years ago

Aman is at aman.kbm@utexas.edu