Post-Routing Annotations Is Not Done When Flat-Router Is Enabled

[amin1377]

There is a function named "sync_netlists_to_routing." This function synchronizes the results of packing and routing. During the routing, the pin that a net is connected to may change, and this would make the routing and packing results inconsistent.

When flat-routing is enabled, this function is skipped. This may cause problems for people who use the results of this function to generate a bitstream.

[tangxifan]

@amin1377

Post-routing annotation is required when there is a local routing architecture, which allows router to swap nets freely between the IPINs that are defined as logic_equivalent in architecture file. Since router may apply such optimization, the routing traces in clustering results should be synchronized accordingly. Otherwise, we see incorrect bitstreams.

For bitstream generators in OpenFPGA, when decoding bitstreams of clusters, we always refer to the ClusteringContext as the final results of any routing traces inside any cluster. Therefore, we need post-routing annotation so that we can always get correct results from ClusteringContext. As a system-level, VPR's date structure should guarantee the correctness of their content. Otherwise, it is very hard to trace bugs in bitstream if any happens.

When flat-routing is enabled, the current post-routing annotation step is not enough, because it only consider the correction on the routing traces of crossbars (only touch nodes at depth of 1 in the local routing resource graph). If flat-routing is enabled, we need to thoroughly update the routing traces of each clusters, because router may change every routing traces, due to

• the nature logic equivalent pins of LUT inputs
• the logic equivalent pins of CLB inputs etc.

So, currently OpenFPGA cannot support flat router. But once the post-routing annotation is upgraded, I believe the problem should be resolved.

[vaughnbetz]

Adding @rs-dhow

[rs-dhow]

@amin1377 @vaughnbetz

Thanks for adding me to this conversation. This came about since I asked about (i) writing out a new .net file after routing equivalent inputs and implicitly changing the packing and (ii) would we also do that after flat routing.

Specific to what we are doing, I think there are actually three possibly relevant cases:

1. Writing out an updated .net file, after routing, but without flat routing. This captures the result of sync_netlists_to_routing.
2. Writing out an updated .net file, after routing WITH flat routing. Assume in-grid configurable routing is only 1 mux deep.
3. Writing out an updated .net file, after routing WITH flat routing. In-grid configurable routing is arbitrary. We understand each of us has separate priorities, but on our end, we cannot afford to wait for this issue to percolate through the code base.

Consequently, I'm going to address (1) on my end and see if it can also handle (2). I'm not interested in (3), nor does that happen in US+ nor Agilex. What we are doing here focuses even more on verification, viz, we have scripting which validates the Verilog for the fabric array against the routing graph (in general, w/o a user design), and we have other scripting which validates the fabric array plus bitstream against the original user design netlist. We use simulation more for block verification. In any event, I will update this scripting, which includes a new bitstream generator, to address (1) and hopefully (2) given our simple in-grid "local routing", and let you know how it goes. In any event, I still think it would be valuable for VPR to be able to completely write out all details of its routing decisions (say, by calling the .net writer a 2nd time), whether flat routing is involved or not. Hopefully I can bridge this gap here until that ability arrives. Thanks.

(Note: we have a new bitstream generator since we estimated the unwieldy "looping" technique in section 9 of our 2024 FPGA paper would take 1-2 man-months to reliably implement to support an even more general version of our "overlay" for commercial routing. The bitstream generator (and accompanying scripts) took 2 man-weeks to write and similarly lets us use all graph features of VPR's router without being subject to the requirements of "island-style" routing.)

[tangxifan]

@vaughnbetz As far as I know, @tpagarani 's team is working on a solution. The flat routing results will be back-annotated to clustering context. I believe this is the way to avoid massive changes in downstream flows.