Rocket Mini to GDSII failed at step 15

[Ryabkomv]

Description

I have successfully generated GDSII file with open lane for picorv32 with configuration using docker container. However when I try to get GDSII for Rocket chip with Tiny configuration I got error at step 15.

[STEP 15]
[INFO]: Running Global Routing Resizer Timing Optimizations...
[ERROR]: during executing openroad script /openlane/scripts/openroad/resizer_routing_timing.tcl
[ERROR]: Exit code: 1
[ERROR]: full log: designs/ExampleRocketSystem/runs/RUN_2022.06.03_12.54.37/logs/routing/15-resizer.log
[ERROR]: Last 10 lines:
met2       Vertical      6182445       4937214          20.14%
met3       Horizontal    4121630       3293440          20.09%
met4       Vertical      2472978       1650582          33.26%
met5       Horizontal     824326        410880          50.16%
---------------------------------------------------------------

[INFO GRT-0101] Running extra iterations to remove overflow.
[ERROR GRT-0169] Net _0432988_: Invalid index for position (2590950, 3605250). Net degree: 6.
Error: resizer_routing_timing.tcl, 53 GRT-0169
child process exited abnormally

Googling I came to the following config.tcl

set ::env(DESIGN_NAME) ExampleRocketSystem
set ::env(FP_CORE_UTIL) 30

set ::env(CELL_PAD) 2

set ::env(PL_BASIC_PLACEMENT) 0
set ::env(PL_TARGET_DENSITY) 0.3

set ::env(GLB_RT_ADJUSTMENT) 0.2

set ::env(PL_ROUTABILITY_DRIVEN) 1

# Change if needed
set ::env(VERILOG_FILES) [glob $::env(DESIGN_DIR)/src/*.v]

# Fill this
set ::env(CLOCK_PERIOD) "10.0"
set ::env(CLOCK_PORT) "clk"

set filename $::env(DESIGN_DIR)/$::env(PDK)_$::env(STD_CELL_LIBRARY)_config.tcl
if { [file exists $filename] == 1} {
        source $filename
}

Environment

docker --version
Docker version 20.10.16, build aa7e414

Please run the following set of commands in the OpenLane folder:

python3 ./env.py issue-survey
Kernel: Linux v5.4.0-113-generic
Distribution: ubuntu 20.04
Python: v3.8.10 (OK)
Container Engine: UNKNOWN vUNKNOWN (UNSUPPORTED)
OpenLane Git Version: 2653e6e4d69c8a8270c494e2ce61cbfd151fae38
pip: INSTALLED
pip:venv: INSTALLED
---
PDK Version Verification Status: OK
---
Git Log (Last 3 Commits)

2653e6e 2022-05-25T19:27:29+02:00 Synthesis Parameters, More Report Metrics (#1105) - Arya Reais-Parsi -  (HEAD -> master, tag: 2022.05.26_01.40.55, origin/master, origin/HEAD)
4298bf2 2022-05-25T17:42:21+02:00 Remove `endcap_cpp` option from `tapcell` (#1085) - Anton Blanchard -  ()
601eede 2022-05-21T18:47:55+02:00 Support Multi-Arch Docker Builds (#1075) - Mohamed Gaber -  (tag: 2022.05.22_02.07.28)

Reproduction Material

• Upload a tarball containing the relevant design. the used command

./flow.tcl -design ExampleRocketSystem

Expected behavior

Successful GDSII generation

src.tar.gz

openroad_issue_reproducible

[vijayank88]

@Ryabkomv openroad_issue_reproducible attachment missing

Or share your repo link

[vijayank88]

Container Engine: UNKNOWN vUNKNOWN (UNSUPPORTED) Also update docker version to latest one

@Donn is possible to print issue-survey with existing container version followed by UNKNOWN message. So its easy to convey the message to user.

[Ryabkomv]

Added sources, and docker version

I'm not sure why I get Container Engine: UNKNOWN vUNKNOWN (UNSUPPORTED)

Docker version 20.10.16, build aa7e414 installed, also I'm able to run docker hello-word

sudo docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
2db29710123e: Pull complete 
Digest: sha256:80f31da1ac7b312ba29d65080fddf797dd76acfb870e677f390d5acba9741b17
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

[vijayank88]

@Ryabkomv Test case attached is not valid format. designs/ExampleRocketSystem/runs/RUN_2022.06.03_12.54.37/openroad_issue_reproducible exist or not?

If not exist Please package a standalone test case with https://github.com/The-OpenROAD-Project/OpenLane/blob/master/docs/source/using_or_issue.md

[Ryabkomv]

@Ryabkomv Test case attached is not valid format. designs/ExampleRocketSystem/runs/RUN_2022.06.03_12.54.37/openroad_issue_reproducible exist or not?

If not exist Please package a standalone test case with https://github.com/The-OpenROAD-Project/OpenLane/blob/master/docs/source/using_or_issue.md

Added corresponding folder as an archive at github. It pretty lengthy.

[vijayank88]

@Ryabkomv can you share me git repo link, where you found this project?

[Ryabkomv]

@Ryabkomv can you share me git repo link, where you found this project?

https://github.com/chipsalliance/rocket-chip I used TinyConfig configuration

[Ryabkomv]

If you're using default configuration try increase FP_CORE_UTIL to 30 and try

Yes, I copied my config and FP_CORE_UTIL is already set to 30

[vijayank88]

Can you share src/*.v files?

[Ryabkomv]

Can you share src/*.v files?

These files are already attached to the post, can you download them?

[vijayank88]

CLOCK_PORT is clock right?

[Ryabkomv]

set ::env(CLOCK_PORT) "clk"

Should be... So you mean I need to change set ::env(CLOCK_PORT) "clk" for set ::env(CLOCK_PORT) "clock" right?

[vijayank88]

yes. I am also running at my end with changes. Will update you the status.

[Ryabkomv]

oh, looks like a stupid mistake...I will rerun the flow with updated config file. By the way, Is there a way to run the flow step by step? Because the process is pretty time consuming, in case of failure at some stage I'm looking to make modifications in config.tcl and re-run just failed stage.

[vijayank88]

Use ./flow.tcl -interactive

[Ryabkomv]

Still have an error at the same stage

[STEP 15]
[INFO]: Running Global Routing Resizer Timing Optimizations...
[ERROR]: during executing openroad script /openlane/scripts/openroad/resizer_routing_timing.tcl
[ERROR]: Exit code: 1
[ERROR]: full log: designs/ExampleRocketSystem/runs/RUN_2022.06.06_12.17.57/logs/routing/15-resizer.log
[ERROR]: Last 10 lines:
[INFO GRT-0101] Running extra iterations to remove overflow.
[WARNING GRT-0227] Reached 20 congestion iterations with less than 15% of reduction between iterations.
[INFO GRT-0197] Via related to pin nodes: 3749275
[INFO GRT-0198] Via related Steiner nodes: 160597
[INFO GRT-0199] Via filling finished.
[INFO GRT-0111] Final number of vias: 6224460
[INFO GRT-0112] Final usage 3D: 33535666
[ERROR GRT-0118] Routing congestion too high.
Error: resizer_routing_timing.tcl, 53 GRT-0118
child process exited abnormally

[vijayank88]

@maliberty Can you please look into this?

[vijayank88]

@Ryabkomv Attach latest run test case

[Ryabkomv]

I've made some changes in configuration, so will upload after the run will be completed. Or hopefully it will be successful.

[Ryabkomv]

Unfortunately got the issue at step 15. updated test case at git repository

[vijayank88]

@maliberty its failing during global routing resizer timing.

[maliberty]

I am out this week, perhaps @luis201420 can help

[vijayank88]

@luis201420 do you had time to look into this issue?

[luis201420]

Yes, I got the following error:

[ERROR GRT-0169] Net _0428215_: Invalid index for position (2680650, 2176950). Net degree: 5.
[ERROR GUI-0070] Error: resizer_routing_timing.tcl, 54 GRT-0169

@Ryabkomv I suggest you use the set_routing_alpha 0 command before you call global route.

[vijayank88]

@donn Is there any configuration available to change set_routing_alpha 0 in openlane flow?

@luis201420 what is default setting for set_routing_alpha? In which scenario we need to pass this variable?

[maliberty]

The alpha=0 is a workaround to a router problem that we are fixing. You would need it when you hit this error.

[Ryabkomv]

Thanks for suggestion. I wonder if I need to set variable in config file? What is there name of the variable? Should it looks like: set ::env(set_routing_alpha) 0 ? I cannot see such variable in this list

[vijayank88]

@Ryabkomv Check for following file in your local path: https://github.com/The-OpenROAD-Project/OpenLane/blob/master/scripts/openroad/groute.tcl

In line 45 of groute.tcl add following lines and run the flow. set_routing_alpha 0

[Ryabkomv]

Thank, will do

[vijayank88]

@Ryabkomv updated command format

[Ryabkomv]

Got same error

[STEP 15] [INFO]: Running Global Routing Resizer Timing Optimizations... [ERROR]: during executing openroad script /openlane/scripts/openroad/resizer_routing_timing.tcl [ERROR]: Exit code: 1 [ERROR]: full log: designs/ExampleRocketSystem/runs/RUN_2022.06.16_06.21.38/logs/routing/15-resizer.log [ERROR]: Last 10 lines: met2 Vertical 6182445 4528494 26.75% met3 Horizontal 4121630 2882560 30.06% met4 Vertical 2472978 1614742 34.70% met5 Horizontal 824326 410880 50.16%

[INFO GRT-0101] Running extra iterations to remove overflow. [ERROR GRT-0169] Net 0430322: Invalid index for position (1983750, 3805350). Net degree: 6. Error: resizer_routing_timing.tcl, 53 GRT-0169 child process exited abnormally

part of groute.tcl with set set_routing_alpha

source $::env(SCRIPTS_DIR)/openroad/set_routing_layers.tcl

set_macro_extension $::env(GLB_RT_MACRO_EXTENSION)
**set ::env(set_routing_alpha) 0**
source $::env(SCRIPTS_DIR)/openroad/layer_adjustments.tcl

set arg_list [list]
lappend arg_list -congestion_iterations $::env(GLB_RT_OVERFLOW_ITERS)
lappend arg_list -verbose
if { $::env(GLB_RT_ALLOW_CONGESTION) == 1 } {
    lappend arg_list -allow_congestion
}

[vijayank88]

@Ryabkomv Its a mistake.

just use set_routing_alpha 0

[Ryabkomv]

oh, sorry

[Ryabkomv]

still failing at the same step with the same error

[vijayank88]

@luis201420 If I use set_routing_alpha 0 before global_route command openroad crashed.

[INFO GRT-0101] Running extra iterations to remove overflow.
[INFO GRT-0103] Extra Run for hard benchmark.
/opt/rh/devtoolset-8/root/usr/include/c++/8/bits/stl_vector.h:932: std::vector<_Tp, _Alloc>::reference std::vector<_Tp, _Alloc>::operator[](std::vector<_Tp, _Alloc>::size_type) [with _Tp = int; _Alloc = std::allocator<int>; std::vector<_Tp, _Alloc>::reference = int&; std::vector<_Tp, _Alloc>::size_type = long unsigned int]: Assertion '__builtin_expect(__n < this->size(), true)' failed.
Stack trace:
 0# 0x0000000000BF53D6 in /home/vijayan/OPENROAD_FLOW/OpenROAD_FLOW/OpenROAD/build/src/openroad
 1# 0x00007FD67ADD0400 in /lib64/libc.so.6
 2# gsignal in /lib64/libc.so.6
 3# abort in /lib64/libc.so.6
 4# boost::stacktrace::detail::this_thread_frames::collect(void const**, unsigned long, unsigned long) in /home/vijayan/OPENROAD_FLOW/OpenROAD_FLOW/OpenROAD/build/src/openroad
 5# 0x0000000002724FC2 in /home/vijayan/OPENROAD_FLOW/OpenROAD_FLOW/OpenROAD/build/src/openroad
 6# grt::FastRouteCore::updateRouteType1(int, grt::TreeNode const*, int, int, int, int, int, grt::TreeEdge*, int, int) in /home/vijayan/OPENROAD_FLOW/OpenROAD_FLOW/OpenROAD/build/src/openroad
 7# grt::FastRouteCore::mazeRouteMSMD(int, int, float, int, int, bool, int, float, int, int, int) in /home/vijayan/OPENROAD_FLOW/OpenROAD_FLOW/OpenROAD/build/src/openroad
 8# grt::FastRouteCore::run() in /home/vijayan/OPENROAD_FLOW/OpenROAD_FLOW/OpenROAD/build/src/openroad
 9# grt::GlobalRouter::findRouting(std::vector<grt::Net*, std::allocator<grt::Net*> >&, int, int) in /home/vijayan/OPENROAD_FLOW/OpenROAD_FLOW/OpenROAD/build/src/openroad
10# grt::GlobalRouter::globalRoute() in /home/vijayan/OPENROAD_FLOW/OpenROAD_FLOW/OpenROAD/build/src/openroad
11# 0x0000000001416B8F in /home/vijayan/OPENROAD_FLOW/OpenROAD_FLOW/OpenROAD/build/src/openroad

[luis201420]

@vijayank88 I cloned the github repository. I unzipped the file openroad_issue_reproducible.tar.gz and I ran openROAD on the directory "openroad_issue_reproducible/home/ryabko/OpenLane/designs/ExampleRocketSystem/runs/RUN_2022.06.07_10.44.44/openroad_issue_reproducible/": ./openroad -no_init run.tcl And I got the error:

[ERROR GRT-0169] Net _0428215_: Invalid index for position (2680650, 2176950). Net degree: 5.
[ERROR GUI-0070] Error: resizer_routing_timing.tcl, 54 GRT-0169

Then, I added on openlane/scripts/openroad/resizer_routing_timing.tcl file before the global_route command (Line 53-54) the set_routing_alpha 0 command to solve the invalid index error. With this command the design finishes the extra overflow iterations without problems but in the end the output gives Routing congestion too high.

[dralabeing]

@luis201420 What is the next step here?

[dralabeing]

@vijayank88 create a new GH with the ORFS crash you see on this test case. Please re-test and If there is no congestion issues close this one.

[Ryabkomv]

Dear all, I'm little bit lost. Is there a way to resolve the issue? Is so, can you kindly let me know the needed config file for that? In my case I still not able to pass stage 15. I tried default config for rocket chip and still have issue. The only successes was with picorv32 processor and simple components.

[vijayank88]

@Ryabkomv Do you have any idea how much clock frequency it can be operated?

[Ryabkomv]

This is good question. I used 10 MHz. I thought it is pretty reasonable. Never used less. Is it too fast?

[vijayank88]

I tried with 30ns = 33.33MHz. Its passing with ORFS flow. OpenLane Its crashed during global routing. Need to debug that.

[vijayank88]

Final routing view from ORFS flow.

[Ryabkomv]

By the way, can I obtain GDSII with ORFS flow (some step by step instruction)? I think it will be more involved, but if OpenLane can't help at the moment I will try.

[vijayank88]

https://github.com/The-OpenROAD-Project/OpenROAD-flow-scripts

[vijayank88]

@vijayank88 create a new GH with the ORFS crash you see on this test case. Please re-test and If there is no congestion issues close this one.

@dralabeing With ORFS flow mini rocket chip routing got completed. I think that crash due to memory issue. took 25GB RAM for post routing timing check and report.

[Ryabkomv]

By the way, in my case I have about 500Gb of RAM

[antonblanchard]

@Ryabkomv using either DFFRAM or OpenRAM for those arrays would likely help a lot.

[maliberty]

Your placement density is quite high:

You said you used PL_TARGET_DENSITY=0.3 & FP_CORE_UTIL=.3 but this is much denser than that. I only see ~8k of instances from optimization and 26k of CTS instances out of your 1.1M instances so it doesn't seem like OR added much.

I feel the problem is upstream of global routing as it is getting a very dense design. Would you provide the logs for all the steps of the flow?

[Ryabkomv]

Can you let me know which file you need? I think I provided the link link to the tarball with all possible logs along with config file containing all used parameters.