Adding analog macros including manually routed interconnects to digital macros shows congestion errors

[3x10e8]

Description

I am trying to incorporate analog macros as blackbox'd verilog with LEF/GDS into caravel_user_project using openlane for top-level integration. These analog macros include mimcaps utilizing metal4 and metal5, therefore PDN cannot go on top of them to prevent shorting supplies and grounds. The analog macro also includes manual routing aimed to interface the analog block to corresponding digital pins, which is possibly giving congestion errors.

Expected behavior

I did not expect congestion errors for blackbox'd macros internally connecting analog block ports to their corresponding digital pins (which is what I suspect is causing the higher congestion at the analog/digital interface below:) Zooming in:

I also expected the PDNgen to completely avoid the analog macro areas. I have tried painting OBS layers but it doesn't quite seem to work (not included in this issue's submission). In the current reproducible, we still see power lines entering the analog macro analog_core_Q_0:

Environment

Kernel: Linux v3.10.0-1160.76.1.el7.x86_64
Distribution: centos 7
Python: v3.6.8 (OK)
Container Engine: docker v20.10.17 (OK)
OpenLane Git Version: 87173889682b67c490c96ded79b93aa2bb570a48
pip: INSTALLED
python-venv: INSTALLED
---
PDK Version Verification Status: OK
---
Git Log (Last 3 Commits)

8717388 %cI Point to `open_pdks` upstream (#1372) - Kareem Farid -  (%D)
---
Git Remotes

origin  https://github.com/The-OpenROAD-Project/OpenLane (fetch)
origin  https://github.com/The-OpenROAD-Project/OpenLane (push)

Reproduction Material

Issue Reproducible: https://github.com/3x10e8/fossi_cochlea/tree/main/openlane/user_project_wrapper/runs/22_09_19_22_38/issue_reproducible

Logs

[STEP 11]
[INFO]: Running Global Routing Resizer Timing Optimizations (log: ../local_disk/fossi_cochlea/openlane/user_project_wrapper/runs/22_09_19_22_38/logs/routing/11-resizer.log)...
[ERROR]: during executing openroad script /openlane/scripts/openroad/resizer_routing_timing.tcl
[ERROR]: Log: ../local_disk/fossi_cochlea/openlane/user_project_wrapper/runs/22_09_19_22_38/logs/routing/11-resizer.log
[ERROR]: Last 10 lines:

[INFO GRT-0101] Running extra iterations to remove overflow.
[INFO GRT-0197] Via related to pin nodes: 147
[INFO GRT-0198] Via related Steiner nodes: 0
[INFO GRT-0199] Via filling finished.
[INFO GRT-0111] Final number of vias: 424
[INFO GRT-0112] Final usage 3D: 14698
[ERROR GRT-0118] Routing congestion too high. Check the congestion heatmap in the GUI.
Error: resizer_routing_timing.tcl, 34 GRT-0118
child process exited abnormally

[ERROR]: Creating issue reproducible...

[maliberty]

You have no OBS on metal5 so pdn sees that space as empty and has no reason to avoid it:

[maliberty]

Where you have OBS the rails are cut:

[maliberty]

Did you notice the numerous warnings, e.g.: [WARNING GRT-0036] Pin wbs_dat_o[9] is outside die area. [WARNING GRT-0040] Net vssa2 has wires outside die area.

You can see all the objects outside the die area:

The router doesn't want to route outside the die area.

[maliberty]

The congested areas are all in the center of your analog block. Moving the pins to the edges and blocking the core should resolve this.

[3x10e8]

You have no OBS on metal5 so pdn sees that space as empty and has no reason to avoid it:

Thank you for taking a look, @maliberty! Actually those power lines in the screenshot above are good, they are trying to power up the digital macro underneath, so its great to have PDNgen handle that :) I have labelled these as (A) in the new screenshot below.

The confusing part is the PDN over the analog block (B). This macro has mimcaps using M4/M5, and previously I had runs with a smaller macro on older openlane where the PDN would go through and short out the power lines with the caps until I manually painted obsm4,obsm5 and exported LEF/GDS to be then called in with openlane. For this run, however, I took out the OBS, but PDN still successfully avoided the analog block except in region (C), where see a broken line trying to go across...

I don't think I manually added OBS here however, but it is nice that PDN is avoiding this area anyway (but not consistently, as some lines are bleeding in -- but that's safely away from the m4/m5 mimcaps):

Where you have OBS the rails are cut:

For the chip area warnings, I believe these warnings would show up for all caravel_user_projects? I have not attempted to modify or debug this part of the wrapper:

Did you notice the numerous warnings, e.g.: [WARNING GRT-0036] Pin wbs_dat_o[9] is outside die area. [WARNING GRT-0040] Net vssa2 has wires outside die area.

You can see all the objects outside the die area:

The router doesn't want to route outside the die area.

Finally, do you have a suggestion on how to handle the routing congestion? I suspect it is from my attempting to manually connect the analog macro's pins to the digital pins: But on my verilogs for the digital macro and the blackbox'd analog macro, I didn't define any interconnects (the ports are left as disconnected) -- could that be causing the congestion error? Should I make openlane do the interconnects? Or is the congestion coming from somewhere else?

[maliberty]

The partial pdn lines could possibly be cleaned up by pdngen (@arlpetergadfort ) but it would be simplest to just OBS to whole area. They aren't actually a violation though.

My suggestion remains to bring the pins to the edge and simplify the macro. Your abstract is overly detailed and trying to route through that narrow area is leading to congestion.

[3x10e8]

The pins I was asking openlane to connect through the blackbox'd verilog are at the edge:

For the interface pins between analog and digital, I brought those out to the edges as well, such that they would short between the analog/digital macros just by placement, and won't need any additional routing through openlane. For these pins, I left their verilog ports disconnected so openlane won't try to route them.

Do you think this could still cause congestion?

[gadfort]

The partial pdn lines could possibly be cleaned up by pdngen (@arlpetergadfort ) but it would be simplest to just OBS to whole area. They aren't actually a violation though.

My suggestion remains to bring the pins to the edge and simplify the macro. Your abstract is overly detailed and trying to route through that narrow area is leading to congestion.

I looks like pdngen is doing what it should and avoiding pins and obstructions. If you don't want PDN to do anything, I think the correct thing to do would be to add the OBS as this is already supported. Not being able to see the pdngen script, I would assume that the inconsistency mentioned relates to the missing strap, which is missing because it was not connected to anything and therefore would not have been needed there.

[3x10e8]

The congested areas are all in the center of your analog block. Moving the pins to the edges and blocking the core should resolve this.

@maliberty One thing I realized was that the metal segments I had defined as ports weren't quite coming to the edge. I fixed that now, and stretched the wires by about 10um (similar to how digital macros bring their ports out by 10um). Unfortunately I still see congestion errors.

What I don't understand is, why is the tool finding congestion in the blackbox'd analog macro being imported with EXTRA_LEF/GDS? The macro itself is DRC clean in magic, and I am not asking openlane to connect any pins in that area (the pins are already being shorted to the correct digital pins by macro placement alone)

Updated issue_reproducible: https://github.com/3x10e8/fossi_cochlea/tree/main/openlane/user_project_wrapper/runs/22_09_21_21_34/issue_reproducible

If you don't want PDN to do anything, I think the correct thing to do would be to add the OBS as this is already supported.

@arlpetergadfort Agreed, adding OBS towards the edge of the analog core is able to take PDN out completely. However there's seem to be a problem with painting metal OBS in magic -- it seems to take some vias out as well, introducing DRC errors. And when I pulled the analog core away from digital by 10um, it provided enough room to complete the PDN strip, so its not broken anymore.

[3x10e8]

Hi @arlpetergadfort, I talked to Tim Edwards and the fix is to write lef out from magic using a -hide flag: lef write -hide I do not need to manually paint OBS layers now. This part of the issue is resolved.

Hi @maliberty, I made some progress on resolving congestion. I went back to testing with a smaller macro and found that some ports on the east-side of the analog blackbox were rather close, and possibly causing the congestion. Removing those ports made the congestion go away, so I ended up just spacing them out, and congestion issues went away when placing one such macro.

So I proceeded with attempting to add more of these blackbox'd macros, and congestion struck again!

[INFO GRT-0101] Running extra iterations to remove overflow.
[INFO GRT-0197] Via related to pin nodes: 1711
[INFO GRT-0198] Via related Steiner nodes: 0
[INFO GRT-0199] Via filling finished.
[INFO GRT-0111] Final number of vias: 1802
[INFO GRT-0112] Final usage 3D: 17690
[ERROR GRT-0118] Routing congestion too high. Check the congestion heatmap in the GUI.
Error: resizer_routing_timing.tcl, 34 GRT-0118
child process exited abnormally

[ERROR]: Creating issue reproducible...

This time, however, I am unable to see where the congestion is. My debug procedure this far has been to run the OpenROAD GUI and then call:

read_db issue_reproducible/results/placement/user_project_wrapper.buffers_removed.odb
global_route

But I don't see a congestion error on the GUI...which is odd, because I see it when running openlane in non-interactive mode. Any thoughts on what might be going on differently between the GUI and the usual openlane flow?

Issue reproducible: https://github.com/3x10e8/fossi_cochlea/tree/main/openlane/user_project_wrapper/runs/22_09_22_22_32/issue_reproducible

[maliberty]

You can run the global router with -congestion_report_file to dump where the congestion issues are. You can load that into the GUI's DRC viewer to see it graphically. That's where I would start to debug to see where the congestion is.

[maliberty]

The overflowing gcells all look like:

I believe the confusion is over where to access the very long pins. Is it possible to short them to just the portion near the edge?

[maliberty]

@eder-matheus @luis201420 is there an easy to get grt to choose to access the pin near the macro edge?

[eder-matheus]

@eder-matheus @luis201420 is there an easy to get grt to choose to access the pin near the macro edge?

I think we can add a check for very long pins (e.g., that goes over multiple groute grid positions) and choose the one closest to the macro edge. Just notice that for macro pins, we apply the fake pin approach to avoid routing near the macro edge.

[maliberty]

@eder-matheus in a very narrow channel like this the fake pins won't work well

[3x10e8]

@maliberty I was able to run the -congestion_report_file with global_route from the gui: global_route -congestion_report_file <path to file> But I somehow couldn't find the report in the docker container...

Still following your conversation with @eder-matheus I tried making my analog macro's ports just be flat edges instead of being boxes.

This alongwith spacing out the macros some more has made us pass global_route (yay!), but detailed routing is unable to access one of the pins towards the edge...any suggestions for this one? (Sorry but it looks like we are chasing a moving target now...)

[STEP 16]
[INFO]: Running Detailed Routing (log: openlane/user_project_wrapper/runs/22_09_23_13_09/logs/routing/16-detailed.log)...
[ERROR]: during executing openroad script /openlane/scripts/openroad/droute.tcl
[ERROR]: Log: openlane/user_project_wrapper/runs/22_09_23_13_09/logs/routing/16-detailed.log
[ERROR]: Last 10 lines:
[INFO DRT-0033] met5 shape region query size = 2770.
[INFO DRT-0165] Start pin access.
[INFO DRT-0076]   Complete 100 pins.
[INFO DRT-0076]   Complete 200 pins.
[ERROR DRT-0073] No access point for analog_core_I_1/fb1[0].
[ERROR DRT-0073] No access point for analog_core_I_0/fb1[0].
[ERROR DRT-0073] No access point for analog_core_Q_0/fb1[0].
[ERROR DRT-0073] No access point for analog_core_Q_1/fb1[0].
Error: droute.tcl, 38 DRT-0073
child process exited abnormally

[ERROR]: Creating issue reproducible...

Zooming in, we see that the fb1[0] pin just needs a vertical strip to short the digital and analog ports together. In the past I had the macros abutting but that gave congestion errors. Should I try spacing them out even more?

Issue reproducible: https://github.com/3x10e8/fossi_cochlea/tree/main/openlane/user_project_wrapper/runs/22_09_23_13_09/issue_reproducible

[maliberty]

You've gone too far the other way. The pin needs to be at least min width in both directions.

[3x10e8]

And that would be the min width for the metal used for the port? So ideally what shape should I make these ports? Rectangles? I have enough room I think.

[maliberty]

Yes the min width for the layer the shape is on. Rects are fine. Its fine to be bigger than min width, just not smaller.

[donn]

Staled out. Do feel free to re-open if the issue persists.