The-OpenROAD-Project / OpenLane

OpenLane is an automated RTL to GDSII flow based on several components including OpenROAD, Yosys, Magic, Netgen and custom methodology scripts for design exploration and optimization.
https://openlane.readthedocs.io/
Apache License 2.0
1.35k stars 373 forks source link

Can not connect SRAM macros to the PDN #2019

Closed jacobhaehn closed 11 months ago

jacobhaehn commented 1 year ago

Description

I've followed the macro tutorials as closely as possible and have tried various different configurations, but OpenLane always seems to have issues connecting the SRAM22 macros (https://github.com/rahulk29/sram22_sky130_macros) to the PDN. I've been using the following settings (some details left out for brevity):

"EXTRA_LEFS": "...",
"EXTRA_GDS_FILES": "...",
"EXTRA_LIBS": "..."
"VERILOG_FILES_BLACKBOX": "..."
"MACRO_PLACEMENT_CFG": "dir::macro_placement.cfg"
"DESIGN_IS_CORE": true,
"FP_PDN_CORE_RING": true,
"SYNTH_USE_PG_PINS_DEFINES": "USE_POWER_PINS",
"VDD_NETS": "VPWR",
"GND_NETS": "VGND",
"FP_PDN_MACRO_HOOKS": "system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_0.cc_banks_0_ext.mem_0_0 VPWR VGND vdd vss ........."

Here the default VPWR and VGND nets are used and "hooked" to the vdd/vss pins on the SRAMS, but I get an error during PDN generation that all the SRAMs are not connected to the nets in the pdn.log. Same happens if I use vdd/vss as the net names.

I have also tried creating 2 power nets, with "VDD_NETS": "VPWR vdd", "GND_NETS": "VGND vss",, and this successfully creates two power nets and passes PDN, connecting the SKY130 cells to VPWR/VGND, and connecting the macros to vdd/vss, but then fails at DRC when vdd and vss are shorted together for some reason. Either way, there's no reason to have 2 power nets here.

Please let me know if you have any suggestions to try. Thanks!

Expected Behavior

OpenLane doesn't error out in PDN step and properly connects SRAM to the power grid.

Environment report

Kernel: Linux v6.2.0-33-generic
Distribution: ubuntu 22.04
Python: v3.10.12 (OK)
Container Engine: docker v24.0.6 (OK)
OpenLane Git Version: 5215ea7cc01e5311086e995905492f0e8fe1ceb6
pip: INSTALLED
python-venv: INSTALLED
---
PDK Version Verification Status: OK
---
Git Log (Last 3 Commits)

5215ea7c 2023-09-20T15:53:26+03:00 Update `open_pdks` (#1992) - Mohamed Gaber -  (HEAD -> master, origin/master, origin/HEAD)
7e5a2e9f 2023-09-10T19:06:55+03:00 fix synthesis `chparam` issue (#1981) - Kareem Farid -  (tag: 2023.09.11)
0e8827eb 2023-09-10T19:04:45+03:00 `run_sta`: fix `-blackbox_check` not running when `-multi_corner` is not set (#1980) - Kareem Farid -  ()
---
Git Remotes

origin  https://github.com/The-OpenROAD-Project/OpenLane (fetch)
origin  https://github.com/The-OpenROAD-Project/OpenLane (push)

Reproduction material

For reference, unpackage the files with the following command:

cat sram_pdn_issue_reproducible.* | tar -xzvf -

sram_pdn_issue_reproducible.partaa.tar.gz sram_pdn_issue_reproducible.partab.tar.gz sram_pdn_issue_reproducible.partac.tar.gz sram_pdn_issue_reproducible.partad.tar.gz

Relevant log output

7-pdn.log (truncated repeated warnings to first 16 of each new instance):

OpenROAD 0a6d0fd469bc674417036342994520ee2e0a2727 
This program is licensed under the BSD-3 license. See the LICENSE file for details.
Components of this program may be licensed under more restrictive licenses which must be honored.
[INFO]: Reading ODB at '/home/george/Documents/batfive/designs/batfive/openlane_src/runs/PDN_Issue/tmp/floorplan/6-tapcell.odb'…
define_corners Typical
read_liberty -corner Typical /home/george/.volare/sky130A/libs.ref/sky130_fd_sc_hd/lib/sky130_fd_sc_hd__tt_025C_1v80.lib
read_liberty -corner Typical /home/george/Documents/batfive/sram22_sky130_macros/sramgen_sram_4096x32m8w8_replica_v1/sramgen_sram_4096x32m8w8_replica_v1_tt_025C_1v80.lib
read_liberty -corner Typical /home/george/Documents/batfive/sram22_sky130_macros/sramgen_sram_4096x8m8w8_replica_v1/sramgen_sram_4096x8m8w8_replica_v1_tt_025C_1v80.lib
read_liberty -corner Typical /home/george/Documents/batfive/sram22_sky130_macros/sramgen_sram_1024x32m8w8_replica_v1/sramgen_sram_1024x32m8w8_replica_v1_tt_025C_1v80.lib
read_liberty -corner Typical /home/george/Documents/batfive/sram22_sky130_macros/sramgen_sram_512x32m4w8_replica_v1/sramgen_sram_512x32m4w8_replica_v1_tt_025C_1v80.lib
read_liberty -corner Typical /home/george/Documents/batfive/sram22_sky130_macros/sramgen_sram_64x32m4w32_replica_v1/sramgen_sram_64x32m4w32_replica_v1_tt_025C_1v80.lib
Using 1e-12 for capacitance...
Using 1e+03 for resistance...
Using 1e-09 for time...
Using 1e+00 for voltage...
Using 1e-03 for current...
Using 1e-09 for power...
Using 1e-06 for distance...
Reading design constraints file at '/openlane/scripts/base.sdc'…
[INFO]: Setting output delay to: 15.0
[INFO]: Setting input delay to: 15.0
[INFO]: Setting load to: 0.033442
[INFO]: Setting clock uncertainty to: 0.25
[INFO]: Setting clock transition to: 0.15
[INFO]: Setting timing derate to: 5.0 %
[INFO PDN-0001] Inserting grid: stdcell_grid
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_0.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_1.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_2.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_3.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_4.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_5.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_6.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_7.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_0.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_1.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_2.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_3.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_4.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_5.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_6.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_7.cc_dir_0_ext.mem_0_0

.......

[WARNING PDN-0110] No via inserted between met4 and met5 at (3324.5400, 2787.2700) - (3326.1400, 2788.2400) on VGND
[WARNING PDN-0110] No via inserted between met4 and met5 at (3821.3400, 2787.2700) - (3822.9400, 2788.2400) on VGND
[WARNING PDN-0110] No via inserted between met4 and met5 at (6677.9400, 2787.2700) - (6679.5400, 2788.2400) on VGND
[INFO PSM-0076] Setting metal node density to be standard cell height times 5.
[INFO PSM-0031] Number of PDN nodes on net VPWR = 922556.
[INFO PSM-0064] Number of voltage sources = 63929.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_0.cc_dir_0_ext.mem_0_0 is not connected to VPWR.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_1.cc_dir_0_ext.mem_0_0 is not connected to VPWR.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_2.cc_dir_0_ext.mem_0_0 is not connected to VPWR.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_3.cc_dir_0_ext.mem_0_0 is not connected to VPWR.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_4.cc_dir_0_ext.mem_0_0 is not connected to VPWR.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_5.cc_dir_0_ext.mem_0_0 is not connected to VPWR.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_6.cc_dir_0_ext.mem_0_0 is not connected to VPWR.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_7.cc_dir_0_ext.mem_0_0 is not connected to VPWR.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_0.cc_dir_0_ext.mem_0_0 is not connected to VPWR.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_1.cc_dir_0_ext.mem_0_0 is not connected to VPWR.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_2.cc_dir_0_ext.mem_0_0 is not connected to VPWR.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_3.cc_dir_0_ext.mem_0_0 is not connected to VPWR.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_4.cc_dir_0_ext.mem_0_0 is not connected to VPWR.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_5.cc_dir_0_ext.mem_0_0 is not connected to VPWR.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_6.cc_dir_0_ext.mem_0_0 is not connected to VPWR.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_7.cc_dir_0_ext.mem_0_0 is not connected to VPWR.

.........

[ERROR PSM-0069] Check connectivity failed.
Error: pdn.tcl, 39 PSM-0069
d-m-bailey commented 1 year ago

Some things you might want to check.

  1. Your FP_PDN_MACRO_HOOKS above appears truncated, so unfortunately I can not verify everything. Be sure to include a , after every block. eg.
    block1 VPWR vdd VGND vss, block2 VPWR vdd VGND vss, ...
  2. Here's a definition of the lef port syntax.
    
    PORT
    Starts a pin port statement that defines a collection of geometries that are electrically
    equivalent points (strongly connected). A pin can have multiple ports. Each PORT of the
    same PIN is considered weakly connected to the other PORTs, and should already be
    connected inside the MACRO (often through a resistive path).

Strongly connected shapes (that is, multiple shapes of one PORT) indicate that a signal router is allowed to connect to one shape of the PORT, and continue routing from another shape of the same PORT.

Weakly connected shapes (that is, separate PORTs of the same PIN) are assumed to be connected through resistive paths inside the MACRO that should not be used by routers. The signal router should connect to one or the other PORT, but not both.

Power routers should connect to every PORT statement, if there is more than one for a given PIN. For example, if a block has several PORTs on the boundary for the VSS PIN, each PORT should be connected by the power router.


So for your power nets, the router may be trying to connect each port of your power nets. From the current lef, this appears to be about a 1000 connects each for vdd and vss.
jacobhaehn commented 1 year ago

Hi Bailey, thanks for the response! A few comments:

  1. I truncated the "FP_PDN_MACRO_HOOKS" above for clarity since it's very dense. The full definition can be found in the issue_reproducible folder I included in any of the env. or run. files. I can paste if here if you'd like. But I do follow the proper syntax from the documentation "<instance_name> <vdd_net> <gnd_net> <vdd_pin> <gnd_pin>, ..."
  2. From the documentation, the "FP_PDN_MACRO_HOOKS" connects the nets to the pins, rather than the ports. Do we have a way to specify which port the router uses? Do you have any suggestions for how to check for or modify this behavior?
d-m-bailey commented 1 year ago

@jacobhaehn The lef documentation says that for power pins, all ports will be connected. I do not know if that is true for openlane.

kareefardi commented 1 year ago

@jacobhaehn Default configuration of OpenLane assumes that the macro power pins are on met4 while the sram macros has their power pins on met3. I believe that providing a custom FP_PDN_CFG (you can copy the one already in use by OpenLane) with the following extra statement would solve your issue.

add_pdn_connect \
    -grid macro \
    -layers "met3 met4"

I am still testing this the runtime of the design is quite high.

jacobhaehn commented 1 year ago

@kareefardi Thank you, this is very helpful and I didn't even think about this! I will try changing the PDN metal layer and get back to you. Yes, you are correct there about the large runtime haha

jacobhaehn commented 1 year ago

Thanks for the patience, it takes about a day to do each run and tried to do some of my own troubleshooting. The flow did fail again at PDN, but at least it did something a little different. Now the pdn.log shows warnings that there are no vias between met3 and met4 on vdd/vss, and that nodes on met4 were moved, and I can't figure out where the issue is. Any help would be appreciated. Here is the new issue_reproducible files (sorry its so large):

PDN_Issue_2.partaa.tar.gz PDN_Issue_2.partab.tar.gz PDN_Issue_2.partac.tar.gz PDN_Issue_2.partad.tar.gz PDN_Issue_2.partae.tar.gz PDN_Issue_2.partaf.tar.gz PDN_Issue_2.partag.tar.gz PDN_Issue_2.partah.tar.gz PDN_Issue_2.partai.tar.gz PDN_Issue_2.partaj.tar.gz PDN_Issue_2.partak.tar.gz PDN_Issue_2.partal.tar.gz PDN_Issue_2.partam.tar.gz PDN_Issue_2.partan.tar.gz PDN_Issue_2.partao.tar.gz PDN_Issue_2.partap.tar.gz PDN_Issue_2.partaq.tar.gz

These are the updated configuration variables:

set ::env(VDD_NETS) "vdd"
set ::env(GND_NETS) "vss"
set ::env(FP_PDN_CFG) "/home/george/Documents/batfive/designs/batfive/openlane_src/pdn_cfg.tcl"
set ::env(FP_PDN_MACRO_HOOKS) "system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_0.cc_banks_0_ext.mem_0_0 vdd vss vdd vss, ...

And with @kareefardi's suggestion, now this is at the bottom of pdn_cdg.tcl (full file included in the reproducible)

define_pdn_grid \
    -macro \
    -default \
    -name macro \
    -starts_with POWER \
    -halo "$::env(FP_PDN_HORIZONTAL_HALO) $::env(FP_PDN_VERTICAL_HALO)"

add_pdn_connect \
    -grid macro \
    -layers "$::env(FP_PDN_VERTICAL_LAYER) $::env(FP_PDN_HORIZONTAL_LAYER)"

#ADDED
add_pdn_connect \
    -grid macro \
    -layers "met3 met4"

Here is the truncated log printout (showing first 16 of each warning type):

OpenROAD 0a6d0fd469bc674417036342994520ee2e0a2727 
This program is licensed under the BSD-3 license. See the LICENSE file for details.
Components of this program may be licensed under more restrictive licenses which must be honored.
[INFO]: Reading ODB at '/home/george/Documents/batfive/designs/batfive/openlane_src/runs/PDN_Issue_2/tmp/floorplan/6-tapcell.odb'…
define_corners Typical
read_liberty -corner Typical /home/george/.volare/sky130A/libs.ref/sky130_fd_sc_hd/lib/sky130_fd_sc_hd__tt_025C_1v80.lib
read_liberty -corner Typical /home/george/Documents/batfive/sram22_sky130_macros/sramgen_sram_4096x32m8w8_replica_v1/sramgen_sram_4096x32m8w8_replica_v1_tt_025C_1v80.lib
read_liberty -corner Typical /home/george/Documents/batfive/sram22_sky130_macros/sramgen_sram_4096x8m8w8_replica_v1/sramgen_sram_4096x8m8w8_replica_v1_tt_025C_1v80.lib
read_liberty -corner Typical /home/george/Documents/batfive/sram22_sky130_macros/sramgen_sram_1024x32m8w8_replica_v1/sramgen_sram_1024x32m8w8_replica_v1_tt_025C_1v80.lib
read_liberty -corner Typical /home/george/Documents/batfive/sram22_sky130_macros/sramgen_sram_512x32m4w8_replica_v1/sramgen_sram_512x32m4w8_replica_v1_tt_025C_1v80.lib
read_liberty -corner Typical /home/george/Documents/batfive/sram22_sky130_macros/sramgen_sram_64x32m4w32_replica_v1/sramgen_sram_64x32m4w32_replica_v1_tt_025C_1v80.lib
Using 1e-12 for capacitance...
Using 1e+03 for resistance...
Using 1e-09 for time...
Using 1e+00 for voltage...
Using 1e-03 for current...
Using 1e-09 for power...
Using 1e-06 for distance...
Reading design constraints file at '/openlane/scripts/base.sdc'…
[INFO]: Setting output delay to: 15.0
[INFO]: Setting input delay to: 15.0
[INFO]: Setting load to: 0.033442
[INFO]: Setting clock uncertainty to: 0.25
[INFO]: Setting clock transition to: 0.15
[INFO]: Setting timing derate to: 5.0 %
[INFO PDN-0001] Inserting grid: stdcell_grid
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_0.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_1.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_2.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_3.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_4.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_5.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_6.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.directory.cc_dir_7.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_0.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_1.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_2.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_3.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_4.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_5.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_6.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_7.cc_dir_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_0.cc_banks_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_0.cc_banks_0_ext.mem_0_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_0.cc_banks_0_ext.mem_1_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_0.cc_banks_0_ext.mem_1_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_0.cc_banks_0_ext.mem_2_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_0.cc_banks_0_ext.mem_2_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_0.cc_banks_0_ext.mem_3_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_0.cc_banks_0_ext.mem_3_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_1.cc_banks_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_1.cc_banks_0_ext.mem_0_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_1.cc_banks_0_ext.mem_1_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_1.cc_banks_0_ext.mem_1_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_1.cc_banks_0_ext.mem_2_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_1.cc_banks_0_ext.mem_2_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_1.cc_banks_0_ext.mem_3_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_1.cc_banks_0_ext.mem_3_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_2.cc_banks_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_2.cc_banks_0_ext.mem_0_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_2.cc_banks_0_ext.mem_1_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_2.cc_banks_0_ext.mem_1_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_2.cc_banks_0_ext.mem_2_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_2.cc_banks_0_ext.mem_2_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_2.cc_banks_0_ext.mem_3_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_2.cc_banks_0_ext.mem_3_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_3.cc_banks_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_3.cc_banks_0_ext.mem_0_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_3.cc_banks_0_ext.mem_1_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_3.cc_banks_0_ext.mem_1_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_3.cc_banks_0_ext.mem_2_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_3.cc_banks_0_ext.mem_2_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_3.cc_banks_0_ext.mem_3_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_3.cc_banks_0_ext.mem_3_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_0.cc_banks_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_0.cc_banks_0_ext.mem_0_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_0.cc_banks_0_ext.mem_1_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_0.cc_banks_0_ext.mem_1_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_0.cc_banks_0_ext.mem_2_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_0.cc_banks_0_ext.mem_2_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_0.cc_banks_0_ext.mem_3_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_0.cc_banks_0_ext.mem_3_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_1.cc_banks_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_1.cc_banks_0_ext.mem_0_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_1.cc_banks_0_ext.mem_1_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_1.cc_banks_0_ext.mem_1_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_1.cc_banks_0_ext.mem_2_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_1.cc_banks_0_ext.mem_2_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_1.cc_banks_0_ext.mem_3_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_1.cc_banks_0_ext.mem_3_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_2.cc_banks_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_2.cc_banks_0_ext.mem_0_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_2.cc_banks_0_ext.mem_1_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_2.cc_banks_0_ext.mem_1_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_2.cc_banks_0_ext.mem_2_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_2.cc_banks_0_ext.mem_2_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_2.cc_banks_0_ext.mem_3_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_2.cc_banks_0_ext.mem_3_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_3.cc_banks_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_3.cc_banks_0_ext.mem_0_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_3.cc_banks_0_ext.mem_1_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_3.cc_banks_0_ext.mem_1_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_3.cc_banks_0_ext.mem_2_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_3.cc_banks_0_ext.mem_2_1
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_3.cc_banks_0_ext.mem_3_0
[INFO PDN-0001] Inserting grid: macro - system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_3.cc_banks_0_ext.mem_3_1
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_0.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_1.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_10.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_11.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_12.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_13.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_14.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_15.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_16.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_17.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_18.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_19.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_2.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_20.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_21.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_22.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_23.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_24.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_25.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_26.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_27.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_28.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_29.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_3.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_30.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_31.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_4.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_5.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_6.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_7.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_8.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.data.data_arrays_0_9.data_arrays_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.frontend.icache.data_arrays_0_0.data_arrays_0_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.frontend.icache.data_arrays_0_1.data_arrays_0_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.frontend.icache.data_arrays_0_2.data_arrays_0_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.frontend.icache.data_arrays_0_3.data_arrays_0_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.frontend.icache.data_arrays_1_0.data_arrays_0_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.frontend.icache.data_arrays_1_1.data_arrays_0_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.frontend.icache.data_arrays_1_2.data_arrays_0_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.frontend.icache.data_arrays_1_3.data_arrays_0_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.tag_array_0.tag_array_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.tag_array_1.tag_array_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.tag_array_2.tag_array_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.dcache.tag_array_3.tag_array_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.frontend.icache.tag_array_0.tag_array_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.frontend.icache.tag_array_1.tag_array_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.frontend.icache.tag_array_2.tag_array_0_0_ext.mem_0_0
[INFO PDN-0001] Inserting grid: macro - system.tile_prci_domain.tile_reset_domain_tile.frontend.icache.tag_array_3.tag_array_0_0_ext.mem_0_0
[WARNING PDN-0110] No via inserted between met4 and met5 at (3324.5400, 2787.2700) - (3326.1400, 2788.2400) on vss
[WARNING PDN-0110] No via inserted between met4 and met5 at (3821.3400, 2787.2700) - (3822.9400, 2788.2400) on vss
[WARNING PDN-0110] No via inserted between met4 and met5 at (6677.9400, 2787.2700) - (6679.5400, 2788.2400) on vss
[WARNING PDN-0110] No via inserted between met3 and met4 at (3554.8350, 7266.4900) - (3555.2350, 7274.9450) on vdd
[WARNING PDN-0110] No via inserted between met3 and met4 at (3557.2350, 7266.4900) - (3557.6350, 7273.4050) on vss
[WARNING PDN-0110] No via inserted between met3 and met4 at (3708.4350, 7266.4900) - (3708.8350, 7274.9450) on vdd
[WARNING PDN-0110] No via inserted between met3 and met4 at (3710.8350, 7266.4900) - (3711.2350, 7273.4050) on vss
[WARNING PDN-0110] No via inserted between met3 and met4 at (3862.0350, 7266.4900) - (3862.4350, 7274.9450) on vdd
[WARNING PDN-0110] No via inserted between met3 and met4 at (3864.4350, 7266.4900) - (3864.8350, 7273.4050) on vss
[WARNING PDN-0110] No via inserted between met3 and met4 at (4015.6350, 7266.4900) - (4016.0350, 7274.9450) on vdd
[WARNING PDN-0110] No via inserted between met3 and met4 at (4018.0350, 7266.4900) - (4018.4350, 7273.4050) on vss
[WARNING PDN-0110] No via inserted between met3 and met4 at (4169.2350, 6910.4000) - (4169.6350, 7274.9450) on vdd
[WARNING PDN-0110] No via inserted between met3 and met4 at (4171.6350, 6911.9400) - (4172.0350, 7273.4050) on vss
[WARNING PDN-0110] No via inserted between met3 and met4 at (3554.8350, 7781.4900) - (3555.2350, 7789.9450) on vdd
[WARNING PDN-0110] No via inserted between met3 and met4 at (3557.2350, 7781.4900) - (3557.6350, 7788.4050) on vss
[WARNING PDN-0110] No via inserted between met3 and met4 at (3708.4350, 7781.4900) - (3708.8350, 7789.9450) on vdd

.........

[WARNING PDN-0110] message limit reached, this message will no longer print
[INFO PSM-0076] Setting metal node density to be standard cell height times 5.
[INFO PSM-0031] Number of PDN nodes on net vdd = 929510.
[WARNING PSM-0033] Node at (7548335, 5025705) and layer met4 moved from (7548335, 5025705).
[WARNING PSM-0033] Node at (7548335, 5039305) and layer met4 moved from (7548335, 5039305).
[WARNING PSM-0033] Node at (7548335, 5052905) and layer met4 moved from (7548335, 5052905).
[WARNING PSM-0033] Node at (7548335, 5093705) and layer met4 moved from (7548335, 5093705).
[WARNING PSM-0033] Node at (7548335, 5107305) and layer met4 moved from (7548335, 5107305).
[WARNING PSM-0033] Node at (7548335, 5120905) and layer met4 moved from (7548335, 5120905).
[WARNING PSM-0033] Node at (7394735, 5025705) and layer met4 moved from (7394735, 5025705).
[WARNING PSM-0033] Node at (7394735, 5039305) and layer met4 moved from (7394735, 5039305).
[WARNING PSM-0033] Node at (7394735, 5052905) and layer met4 moved from (7394735, 5052905).
[WARNING PSM-0033] Node at (7394735, 5093705) and layer met4 moved from (7394735, 5093705).
[WARNING PSM-0033] Node at (7394735, 5107305) and layer met4 moved from (7394735, 5107305).
[WARNING PSM-0033] Node at (7394735, 5120905) and layer met4 moved from (7394735, 5120905).
[WARNING PSM-0033] Node at (7548335, 4827705) and layer met4 moved from (7548335, 4827705).
[WARNING PSM-0033] Node at (7548335, 4841305) and layer met4 moved from (7548335, 4841305).
[WARNING PSM-0033] Node at (7548335, 4854905) and layer met4 moved from (7548335, 4854905).
[WARNING PSM-0033] Node at (7548335, 4895705) and layer met4 moved from (7548335, 4895705).

.......

[WARNING PSM-0033] message limit reached, this message will no longer print
[INFO PSM-0064] Number of voltage sources = 63929.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_0.cc_banks_0_ext.mem_1_0 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_0.cc_banks_0_ext.mem_1_1 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_0.cc_banks_0_ext.mem_3_0 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_0.cc_banks_0_ext.mem_3_1 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_1.cc_banks_0_ext.mem_1_0 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_1.cc_banks_0_ext.mem_1_1 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_1.cc_banks_0_ext.mem_3_0 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_1.cc_banks_0_ext.mem_3_1 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_2.cc_banks_0_ext.mem_0_0 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_2.cc_banks_0_ext.mem_0_1 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_2.cc_banks_0_ext.mem_2_0 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_2.cc_banks_0_ext.mem_2_1 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_3.cc_banks_0_ext.mem_0_0 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_3.cc_banks_0_ext.mem_0_1 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_3.cc_banks_0_ext.mem_2_0 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_3.cc_banks_0_ext.mem_2_1 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_0.cc_banks_0_ext.mem_1_0 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_0.cc_banks_0_ext.mem_3_0 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_1.cc_banks_0_ext.mem_1_0 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_1.cc_banks_0_ext.mem_3_0 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_2.cc_banks_0_ext.mem_0_1 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_2.cc_banks_0_ext.mem_2_1 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_3.cc_banks_0_ext.mem_0_1 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.bankedStore.cc_banks_3.cc_banks_0_ext.mem_2_1 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_0.cc_dir_0_ext.mem_0_0 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_1.cc_dir_0_ext.mem_0_0 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_2.cc_dir_0_ext.mem_0_0 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_3.cc_dir_0_ext.mem_0_0 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_4.cc_dir_0_ext.mem_0_0 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_5.cc_dir_0_ext.mem_0_0 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_6.cc_dir_0_ext.mem_0_0 is not connected to vdd.
[WARNING PSM-0094] system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched_1.directory.cc_dir_7.cc_dir_0_ext.mem_0_0 is not connected to vdd.
[WARNING PSM-0094] system.tile_prci_domain.tile_reset_domain_tile.dcache.tag_array_0.tag_array_0_ext.mem_0_0 is not connected to vdd.
[WARNING PSM-0094] system.tile_prci_domain.tile_reset_domain_tile.dcache.tag_array_1.tag_array_0_ext.mem_0_0 is not connected to vdd.
[WARNING PSM-0094] system.tile_prci_domain.tile_reset_domain_tile.frontend.icache.tag_array_0.tag_array_0_0_ext.mem_0_0 is not connected to vdd.
[WARNING PSM-0094] system.tile_prci_domain.tile_reset_domain_tile.frontend.icache.tag_array_1.tag_array_0_0_ext.mem_0_0 is not connected to vdd.
[ERROR PSM-0069] Check connectivity failed.
Error: pdn.tcl, 39 PSM-0069

Thanks for any help you are able to provide!

kareefardi commented 1 year ago

@jacobhaehn So first, I disabled PDN_CHECK_NODES since I know that it is going to fail anyway. This reduced the runtime of the failing test case to about 5 mins. I was able to easily look at the design and spot the issue. image This is a sample from the top left side of the design. As you can see met4 is not passing through the macro. I am not entirely sure why yet however met5 is indeed passing through. So if we add another

add_pdn_connect \
    -grid macro \
    -layers "met3 met5"

The met5 stripes are connected to the met3 inside the macros. image

You would still see these kind of warnings:

[WARNING PDN-0110] No via inserted between met3 and met5 at (4248.4150, 1408.6500) - (4248.8150, 1410.1250) on VGND

I checked one of the locations mentioned in that warning. It is actually a place where met3 pins are cutoff at the end and the tool cannot actually establish the connection there with the right set of vias. Meaning that the warning in some cases can be ignored.

This fixes the issue where no connectivity is happening with the macros. However, I am not sure if there are other issues with the PDN itself. LVS would have the final say in this matter but this requires a complete flow. I still suggest that you look at the layout after the PDN is generated an visually inspect different aspects of the PDN.

Best of luck. This is a fairly large design and it would challenging to complete it through the flow.

jacobhaehn commented 1 year ago

Thanks for taking another look. Adding the connections between met3 and met5 allowed PDN to pass. It is certainly strange that met4 doesn't want to pass through the macro, considering there are no obstructions on met4 within the macro. Currently trying to push the design through the rest of the OpenLane flow. I will post an update once I can get to LVS and note if I have any errors.

jacobhaehn commented 1 year ago

After 7 days in routing to get to 64 max detailed routing iterations, there appear to be a large number of metal spacing and short DRC violations. There is no issue_reproducible, but I will attach the DEF and a few logs to view the routing and DRC issues (in OpenRoad viewer you can load in drt.drc from windows>DRC Viewer)

DRT_DRC_pseudo_issue_reproducible.partaa.tar.gz DRT_DRC_pseudo_issue_reproducible.partab.tar.gz DRT_DRC_pseudo_issue_reproducible.partac.tar.gz DRT_DRC_pseudo_issue_reproducible.partad.tar.gz DRT_DRC_pseudo_issue_reproducible.partae.tar.gz DRT_DRC_pseudo_issue_reproducible.partaf.tar.gz DRT_DRC_pseudo_issue_reproducible.partag.tar.gz

Looks like the issue is the PDN places hundreds of vias horizontally in each SRAM, since there are many vertical VDD/VSS straps within the macro. This effectively blocks any routing on met4.

image

Turning off met5 to show long line of met4 vias: image

Zooming in on DRC issue (metal spacing and shorting): image

Selecting one of these nets (highlighted in yellow), we can see that it's simply a wire trying to pass through the SRAM, and doesn't connect anywhere within. Why doesn't it just go around? image

This behavior can be seen on almost half the macros image

I'm also getting detailed routing warnings that the SRAM macros don't have pins on the routing grid, such as this for all macro pins, but it appears to be correctly connecting the IO pins:

[WARNING DRT-0418] Term system.subsystem_l2_wrapper.l2.inclusive_cache_bank_sched.bankedStore.cc_banks_0.cc_banks_0_ext.mem_0_0/addr[0] has no pins on routing grid

I presume this issue and the issue with the met4 PDN stripes not passing through the macro might be due to the variable "MACRO_BLOCKAGES_LAYER": "li1 met1 met2 met3 met4" which places blockages on met4, which I'm guessing is because OpenLane assumes there are pins on met4, however the SRAM macros I'm using have their pins on met3. I am currently trying a run with "MACRO_BLOCKAGES_LAYER": "li1 met1 met2 met3" to see if vertical met4 PDN stripes will pass through the macros, connecting to VDD/VSS vertically and hopefully prevent the hundred of horizontally placed met4 vias.

If this doesn't work, is there a way to change the vertical PDN layer to met5? I see the FP_PDN_HORIZONTAL/VERTICAL_LAYER variable in config.tcl, but don't see how the orientation is specified in pdn.cfg

d-m-bailey commented 1 year ago

@jacobhaehn are you using your own sram macros? You might want to take a look at the lef file. You can load it into klayout. The layout should be pretty simple with only the pins you want showing as details and the obstruction layers showing as large (mostly rectangular areas) covering most of the macro. If you have large lef files that contain unnecessary detail, this might be causing long run times.

jacobhaehn commented 1 year ago

@d-m-bailey No, these are not my own SRAM macros. These are the sky130 SRAM22 macros (https://github.com/rahulk29/sram22_sky130_macros) that are used in the Chipyard/Rocketchip vlsi flow. The main problem is that the LEF contains hundreds of VDD/VSS ports covering the entire surface (shown below: all the vertical lines are VDD/VSS), so OpenLane tries to connect to all of these whenever the PDN strap intersects.

image

You make a good point though, so the next plan is to delete all but a few of the VDD/VSS stripes around the edges of the macro from the LEF of each SRAM. This will prevent OpenLane from connecting the PDN to the macros hundreds of times, blocking met4 routing. I will update when this is complete.

P.S. changing "MACRO_BLOCKAGES_LAYER" variable didn't change anything and seems to be a dummy variable, as it's not referenced anywhere in OpenLane

maliberty commented 1 year ago

Good LEFs are an abstract of the block not just a direct copy of all the details.

d-m-bailey commented 1 year ago

@jacobhaehn This if from the LEF/DEF specification

PORT
Starts a pin port statement that defines a collection of geometries that are electrically
equivalent points (strongly connected). A pin can have multiple ports. Each PORT of the
same PIN is considered weakly connected to the other PORTs, and should already be
connected inside the MACRO (often through a resistive path).

Strongly connected shapes (that is, multiple shapes of one PORT) indicate that a signal
router is allowed to connect to one shape of the PORT, and continue routing from another
shape of the same PORT.

Weakly connected shapes (that is, separate PORTs of the same PIN) are assumed to be
connected through resistive paths inside the MACRO that should not be used by routers.
The signal router should connect to one or the other PORT, but not both.

Power routers should connect to every PORT statement, if there is more than one for a
given PIN. For example, if a block has several PORTs on the boundary for the VSS PIN,
each PORT should be connected by the power router. 

So the router is supposed to connect each power port within a pin definition. If you could combine the ports, the router will only need to connect to one shape within the port definition.

maliberty commented 1 year ago

True but it is still way too much detail for the goal. The router and pdn will be slowed down dealing with a lot of extraneous detail.

jacobhaehn commented 1 year ago

Got it, that's certainly good information to have about the routing rules for LEF/DEF. I have made the LEF modifications by deleting extra VDD/VSS ports and am waiting for the PDN step to complete. Still waiting to see if the unconnected met4 PDN straps will cause LVS issues.

The SRAM22 macros appear to still be under development and likely haven't been fully tapeout proven. The LEFs are created automatically by their generator, so this might be an enhancement suggestion on their part.

jacobhaehn commented 11 months ago

Update: Removing the excessive VDD/VSS ports from the LEF files allowed detailed routing to pass without any DRC errors and reduced routing congestion.

This is an example of the new LEF: image

Magic DRC: I got a child killed: kill signal at while Loading DRC CIF style which is normally an out of memory issue, but our machine now has 270 GB of ram...so I'm hoping this is some sort of issue with interactive mode. I'm re-running a full flow now to see if we can get through DRC/LVS.

jacobhaehn commented 11 months ago

Update: Magic DRC always gets a child killed and uses >215 GB of RAM over the course of multiple hours. Switching to Klayout DRC runs in about 10-15 minutes and only uses ~8 GB of RAM. Not sure why there's such a drastic difference, but it appears to work. I'm also getting errors with Klayout GDSII streaming (can't find gds for MOSSpace), so we are using Magic for that step.

The concern about LVS due to dangling PDN wires around the SRAM from @kareefardi doesn't seem to be an issue as it passes LVS with no problems. There do seem to be issues with IRDrop, as we are getting a IRDrop on the order of millions of volts, and it took 3 weeks to complete.

[INFO PSM-0022] Using 1.800V for vdd
[INFO PSM-0076] Setting metal node density to be standard cell height times 5.
[INFO PSM-0031] Number of PDN nodes on net vdd = 4930779.
[INFO PSM-0064] Number of voltage sources = 115585.
[INFO PSM-0040] All PDN stripes on net vdd are connected.
########## IR report #################
Corner: Typical
Worstcase voltage: -1.27e+08 V
Average IR drop  : 1.32e+04 V
Worstcase IR drop: 1.27e+08 V
######################################
[INFO PSM-0022] Using 0.000V for vss
[INFO PSM-0076] Setting metal node density to be standard cell height times 5.
[INFO PSM-0031] Number of PDN nodes on net vss = 4940722.
[INFO PSM-0064] Number of voltage sources = 115835.
[INFO PSM-0040] All PDN stripes on net vss are connected.
########## IR report #################
Corner: Typical
Worstcase voltage: 1.35e+08 V
Average IR drop  : 1.32e+04 V
Worstcase IR drop: 1.35e+08 V
######################################

I'm going to go ahead and close this issue as it seems resolved.