With tightly packed macros, LVS fails due to hanging strips of metal on the PDN

mattvenn commented 2 years ago

Description

Environment

Kernel: Linux v5.13.0-40-generic
Distribution: ubuntu 20.04
Python: v3.8.10 (OK)
Container Engine: docker v20.10.7 (OK)
OpenLane Git Version: -dev
pip:click: INSTALLED
pip:pyyaml: INSTALLED
pip:venv: INSTALLED
---
PDK Version Verification Status: OK
---
Git Log (Last 3 Commits)

e4bfdd7 2022-02-22T16:59:14-03:00 Remove pip install, no longer needed (#953) - Vitor Bandeira -  (grafted, HEAD, tag: 2022.02.23_02.50.41)

Reproduction Material

tarball not easy as this is Caravel - see repo: https://github.com/mattvenn/tapeout_100
List the commands used to run the design:
make install (installs caravel submodule)
make uncompress (uncompresses gds files)
user_project_wrapper

Expected behavior

LVS shouldn't fail

Logs

final lines of user_project_wrapper/runs/user_project_wrapper/logs/finishing/26-user_project_wrapper.lef.log:

vssd1                                      |(no matching pin)                          
vssd1_uq0                                  |(no matching pin)                          
vssd1_uq1                                  |(no matching pin)                          
vssd1_uq2                                  |(no matching pin)                          
vssd1_uq3                                  |(no matching pin)

vijayank88 commented 2 years ago

@mattvenn I am trying to clone your repo with https://github.com/The-OpenROAD-Project/OpenLane-MPW-CI and running its got failed to find module wrapped_icestudio_test

[INFO]: Preparing LEF files for the max corner...
[INFO]: Looking for files defined in ::env(EXTRA_GDS_FILES)  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_function_generator.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_vga_clock.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_frequency_counter.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_rgb_mixer.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_hack_soc_dffram.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_teras.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_alu74181.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_vgademo_on_fpga.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_silife.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_acorn_prng.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_hsv_mixer.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_skullfet.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wb_bridge_2way.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wb_openram_wrapper.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/sky130_sram_1kbyte_1rw1r_32x256_8.gds  ...
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_function_generator.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_vga_clock.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_frequency_counter.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_rgb_mixer.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_hack_soc_dffram.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_teras.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_alu74181.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_vgademo_on_fpga.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_silife.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_acorn_prng.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_hsv_mixer.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_skullfet.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wb_bridge_2way.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wb_openram_wrapper.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/sky130_sram_1kbyte_1rw1r_32x256_8.gds exists.
[STEP 1]
[INFO]: Running Synthesis...
[STEP 2]
[INFO]: Running Single-Corner Static Timing Analysis...
[INFO]: Creating a netlist with power/ground pins.
[STEP 3]
[INFO]: Running Initial Floorplanning...
[ERROR]: Floorplanning failed
[ERROR]: module wrapped_icestudio_test not found in /openlane/designs/tapeout_100/runs/RUN_2022.04.30_09.48.18/tmp/merged.nom.lef
[ERROR]: Check whether EXTRA_LEFS is set appropriately
[INFO]: Generating final set of reports...
[INFO]: Created manufacturability report at 'runs/RUN_2022.04.30_09.48.18/reports/manufacturability.rpt'.
[INFO]: Created metrics report at 'runs/RUN_2022.04.30_09.48.18/reports/metrics.csv'.
[INFO]: Saving runtime environment...
[ERROR]: Flow failed.

    while executing
"flow_fail"
    (procedure "check_floorplan_missing_lef" line 11)
    invoked from within
"check_floorplan_missing_lef"
    (procedure "init_floorplan" line 12)
    invoked from within
"init_floorplan"
    (procedure "run_floorplan" line 7)
    invoked from within
"[lindex $step_exe 0] [lindex $step_exe 1] "
    (procedure "run_non_interactive_mode" line 55)
    invoked from within
"run_non_interactive_mode {*}$argv"
    invoked from within
"if { [info exists flags_map(-interactive)] || [info exists flags_map(-it)] } {
    if { [info exists arg_values(-file)] } {
        run_file [file normalize $a..."
    (file "./flow.tcl" line 427)
make: *** [quick_run] Error 1
make: Leaving directory `/home/vijayan/MPW_CI/OpenLane-MPW-CI/OpenLane'

Can you update required module and openlane/user_project_wrapper/config.tcl to run the flow to re-produce your issue at our end?

maliberty commented 2 years ago

Why does it being caravel make it hard to produce a tarball? @donn any thoughts?

donn commented 2 years ago

You can ask the caravel team.

mattvenn commented 2 years ago

@maliberty because the caravel_user_project is 8.2G. So I need to know what to .tar.gz in order to capture what you need for a test case

mattvenn commented 2 years ago

@vijayank88 thanks for taking a look at this. I've fixed the example repo, my bad I should have tested it locally.

antonblanchard commented 2 years ago

I'm not sure it's the best way, but I'll throw an exit 1 into the step that I want to debug, and Openlane should die and populate openroad_issue_reproducible. Assuming things start going wrong at the PDN stage, add it to scripts/openlane/pdn.tcl.

@donn or @maliberty might have a better idea.

mattvenn commented 2 years ago

actually I did know this trick but thought it would only work with openlane, rather than inside caravel_user_project. Thanks Anton!

vijayank88 commented 2 years ago

@mattvenn I cloned your repo and its fail with LVS mismach error.

Its similar to #1067

[STEP 29]
[INFO]: Running LEF LVS...
[ERROR]: There are LVS errors in the design: See 'runs/RUN_2022.05.02_12.28.56/logs/signoff/29-user_project_wrapper.lvs.lef.log' for details.
[INFO]: Generating final set of reports...
[INFO]: Created manufacturability report at 'runs/RUN_2022.05.02_12.28.56/reports/manufacturability.rpt'.
[INFO]: Created metrics report at 'runs/RUN_2022.05.02_12.28.56/reports/metrics.csv'.
[INFO]: Saving runtime environment...
[ERROR]: Flow failed.
[INFO]: The failure may have been because of the following warnings:
[WARNING]: Skipping Tap/Decap Insertion.
[WARNING]: Specifying a routing obstruction is now done using the coordinates
[WARNING]: of its bounding box instead of the now deprecated (x, y, size_x, size_y).

    while executing
"flow_fail"
    (procedure "quit_on_lvs_error" line 12)
    invoked from within
"quit_on_lvs_error -log $count_lvs_log"
    (procedure "run_lvs" line 79)
    invoked from within
"run_lvs"
    (procedure "run_lvs_step" line 11)
    invoked from within

vijayank88 commented 2 years ago

I'm not sure it's the best way, but I'll throw an exit 1 into the step that I want to debug, and Openlane should die and populate openroad_issue_reproducible. Assuming things start going wrong at the PDN stage, add it to scripts/openlane/pdn.tcl.

@donn or @maliberty might have a better idea.

This way its works and produce openroad_issue_reproducible, but user should remove exit 1 from both script.

vijayank88 commented 2 years ago

@maliberty

Fixing #1067 will resolve this error too.

mattvenn commented 2 years ago

I don't think so. I think this is valid that netgen is showing an error as there are many unconnected strips of metal. The problem is that the PDN is generating these hanging strips that don't connect to anything.

maliberty commented 2 years ago

I want to get OL to the new pdngen and see if that resolves the issue rather than fixing the old pdngen. I'm hoping it is close now.

maliberty commented 2 years ago

Can you try this with https://github.com/The-OpenROAD-Project/OpenLane/pull/1059 ?

vijayank88 commented 2 years ago

@antonblanchard Here @mattvenn re-used macros wrapped_icestudio_test 7 times.

Refer repo here: https://github.com/mattvenn/tapeout_100/blob/tapeout_100/openlane/user_project_wrapper/macro.cfg

How to define MACRO_HOOKS?

set ::env(FP_PDN_MACRO_HOOKS) "\
    mprj vccd1 vssd1"

antonblanchard commented 2 years ago

@vijayank88 @mattvenn I think you want:

set ::env(FP_PDN_MACRO_HOOKS) "\
    wrapped_icestudio_test_0 vccd1 vssd1, \
    wrapped_icestudio_test_1 vccd1 vssd1, \
    wrapped_icestudio_test_2 vccd1 vssd1, \
    wrapped_icestudio_test_3 vccd1 vssd1, \
    wrapped_icestudio_test_4 vccd1 vssd1, \
    wrapped_icestudio_test_5 vccd1 vssd1, \
    wrapped_icestudio_test_6 vccd1 vssd1, \
    wrapped_icestudio_test_7 vccd1 vssd1"

@maliberty I notice the Openlane documentation claims macros are connected to the first power domain if they are unspecified:

| `FP_PDN_MACRO_HOOKS` | Specifies explicit power connections of internal macros to the top level power grid. Comma separated list of macro instance names and power domain vdd and ground net names: `<instance_name> <vdd_net> <gnd_net>` <br> (Default: macros are connected to the first power domain

That's not correct is it? Should we remove that comment?

vijayank88 commented 2 years ago

@mattvenn Follow above FP_PDN_MACRO_HOOKS use set ::env(LVS_CONNECT_BY_LABEL) 1 Able to complete the flow successfully at my end. Please do confirm.

maliberty commented 2 years ago

@antonblanchard re "Should we remove that comment?" Yes. I'm not clear that was actually true before either but I haven't tested it. Arbitrarily connecting macros seems dangerous.

maliberty commented 2 years ago

@vijayank88 there should not be dangling metal. Connecting by label is a good workaround but it masks the issue. Have you verified if the problem still exists with the new pdngen without this workaround? If so please post an updated test case.

antonblanchard commented 2 years ago

@antonblanchard re "Should we remove that comment?" Yes. I'm not clear that was actually true before either but I haven't tested it. Arbitrarily connecting macros seems dangerous.

Created https://github.com/The-OpenROAD-Project/OpenLane/pull/1091

vijayank88 commented 2 years ago

@maliberty Latest PDN stage attached here: openroad_issue_reproducible.zip

maliberty commented 2 years ago

@arlpetergadfort there are dangling pieces of met4 between the macros: The right piece has a via but the left does not. The right belongs to the single met5 vdd strap while the left is part of vss and has nothing to connect to.

maliberty commented 2 years ago

mattvenn commented 2 years ago

@vijayank88 @mattvenn I think you want:

set ::env(FP_PDN_MACRO_HOOKS) "\
  wrapped_icestudio_test_0 vccd1 vssd1, \
  wrapped_icestudio_test_1 vccd1 vssd1, \
  wrapped_icestudio_test_2 vccd1 vssd1, \
  wrapped_icestudio_test_3 vccd1 vssd1, \
  wrapped_icestudio_test_4 vccd1 vssd1, \
  wrapped_icestudio_test_5 vccd1 vssd1, \
  wrapped_icestudio_test_6 vccd1 vssd1, \
  wrapped_icestudio_test_7 vccd1 vssd1"

@maliberty I notice the Openlane documentation claims macros are connected to the first power domain if they are unspecified:

| `FP_PDN_MACRO_HOOKS` | Specifies explicit power connections of internal macros to the top level power grid. Comma separated list of macro instance names and power domain vdd and ground net names: `<instance_name> <vdd_net> <gnd_net>` <br> (Default: macros are connected to the first power domain

That's not correct is it? Should we remove that comment?

I've found through experimentation that the FP_PDN_MACRO_HOOKS isn't required - macros get connected to power anyway. What does it do? I've also tried using it, and I still get warnings about macros not being powered.

mattvenn commented 2 years ago

@vijayank88 I tried your suggestions and got this new error:

[ERROR PDN-0165] Conflict found, instance wrapped_icestudio_test_0 is part of two grid definitions (CORE_macro_grid_2, CORE_macro_grid_1)

jurevreca12 commented 2 years ago

@mattvenn Hey matt, did you maybe manage to fix this issue? (Conflict found...) I am getting the same issue.

mattvenn commented 2 years ago

No, haven't found a fix yet.

jurevreca12 commented 2 years ago

@vijayank88 I noticed one thing after closing the issue. I commented out the macro set ::env(FP_PDN_MACRO_HOOKS) . If I don't do that i still get the same error as Matt. Is this a problem?

vijayank88 commented 2 years ago

@jurevreca12 Open a new issue with clear description and test case.

jurevreca12 commented 2 years ago

https://github.com/The-OpenROAD-Project/OpenLane/issues/1104

mattvenn commented 2 years ago

this issue has the 'waiting on OP' tag applied. But I'm still stuck on this issue as noted above on May 23.

With the config changes suggested by @vijayank88 I get the new error:

Conflict found, instance wrapped_icestudio_test_0 is part of two grid definitions (CORE_macro_grid_2, CORE_macro_grid_1).

mattvenn commented 2 years ago

I just checked @jurevreca12 's comment. By commenting out the FP_PDN_MACRO_HOOKS but leaving the LVS_CONNECT_BY_LABEL the flow completes.

d-m-bailey commented 2 years ago

@mattvenn LVS_CONNECT_BY_LABEL means that unconnected nets having the same label are treated as connected. This is not something I'd recommend on a top level block. For example, even if your macros all had vccd1 labels on the power supplies, you still want them to be physically connected. With this option, even if they are not connected LVS, will pass.

mattvenn commented 2 years ago

Thank you Mitch. Yes let's leave this open then as the LVS_CONNECT_BY_LABEL is definitely something I'll only use for prototyping

On Fri, 24 Jun 2022 at 17:54, Mitch Bailey @.***> wrote:

@mattvenn https://github.com/mattvenn LVS_CONNECT_BY_LABEL means that unconnected nets having the same label are treated as connected. This is not something I'd recommend on a top level block. For example, even if your macros all had vccd1 labels on the power supplies, you still want them to be physically connected. With this option, even if they are not connected LVS, will pass.

— Reply to this email directly, view it on GitHub https://github.com/The-OpenROAD-Project/OpenLane/issues/1076#issuecomment-1165707623, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAE223B4QVASNC6CANLVNNLVQXK2VANCNFSM5UWP4YLA . You are receiving this because you were mentioned.Message ID: @.***>

-- Matthew Venn web mattvenn.net twitter @matthewvenn https://twitter.com/matthewvenn

kareefardi commented 6 months ago

@mattvenn This is an old open issue. I see it was referenced in multiple other issues and PRs and might not be relevant anymore. Is there anything to look for here ?

The-OpenROAD-Project / OpenLane