Design Always Fails Synthesis after Updating to Develop

[TsaiAnson]

I have a design that synthesizes properly on rc4 but fails when I updated OpenLane to develop (nothing else changed). The advice from Slack is to increase the clock period and double check the clock, but those did not resolve the issue. Below are more details:

Error:

puts "\[INFO\]: Setting load to: $cap_load"
[INFO]: Setting load to: 0.01765
set_load  $cap_load [all_outputs]
tns -3.94
wns -2.24
[ERROR]: during executing: "sta /openLANE_flow/scripts/sta.tcl |& tee >&@stdout /openLANE_flow/designs/mac_cluster/runs/17-11_00-16/logs/synthesis/opensta.log"
[ERROR]: Exit code: 1
[ERROR]: Last 10 lines:
child killed: segmentation violation

[ERROR]: Please check sta  log file
[ERROR]: Dumping to /openLANE_flow/designs/mac_cluster/runs/17-11_00-16/error.log

opensta.log (it's short so posting directly here)

OpenSTA 2.2.0 0d73b5b65a Copyright (c) 2019, Parallax Software, Inc.
License GPLv3: GNU GPL version 3 <http://gnu.org/licenses/gpl.html>

This is free software, and you are free to change and redistribute it
under certain conditions; type `show_copying' for details.
This program comes with ABSOLUTELY NO WARRANTY; for details type `show_warranty'.
Warning: /home/anson/cs250_pdk/sky130A/libs.ref/sky130_fd_sc_hd/lib/sky130_fd_sc_hd__ff_n40C_1v95.lib, line 31 default_operating_condition ff_n40C_1v95 not found.
Warning: /home/anson/cs250_pdk/sky130A/libs.ref/sky130_fd_sc_hd/lib/sky130_fd_sc_hd__ss_100C_1v60.lib, line 32 default_operating_condition ss_100C_1v60 not found.
create_clock [get_ports $::env(CLOCK_PORT)]  -name $::env(CLOCK_PORT)  -period $::env(CLOCK_PERIOD)
set input_delay_value [expr $::env(CLOCK_PERIOD) * $::env(IO_PCT)]
set output_delay_value [expr $::env(CLOCK_PERIOD) * $::env(IO_PCT)]
puts "\[INFO\]: Setting output delay to: $output_delay_value"
[INFO]: Setting output delay to: 40.0
puts "\[INFO\]: Setting input delay to: $input_delay_value"
[INFO]: Setting input delay to: 40.0
set clk_indx [lsearch [all_inputs] [get_port $::env(CLOCK_PORT)]]
#set rst_indx [lsearch [all_inputs] [get_port resetn]]
set all_inputs_wo_clk [lreplace [all_inputs] $clk_indx $clk_indx]
#set all_inputs_wo_clk_rst [lreplace $all_inputs_wo_clk $rst_indx $rst_indx]
set all_inputs_wo_clk_rst $all_inputs_wo_clk
# correct resetn
set_input_delay $input_delay_value  -clock [get_clocks $::env(CLOCK_PORT)] $all_inputs_wo_clk_rst
#set_input_delay 0.0 -clock [get_clocks $::env(CLOCK_PORT)] {resetn}
set_output_delay $output_delay_value  -clock [get_clocks $::env(CLOCK_PORT)] [all_outputs]
# TODO set this as parameter
set_driving_cell -lib_cell $::env(SYNTH_DRIVING_CELL) -pin $::env(SYNTH_DRIVING_CELL_PIN) [all_inputs]
set cap_load [expr $::env(SYNTH_CAP_LOAD) / 1000.0]
puts "\[INFO\]: Setting load to: $cap_load"
[INFO]: Setting load to: 0.01765
set_load  $cap_load [all_outputs]
tns -3.94
wns -2.24

Config:

# User config
set ::env(DESIGN_NAME) mac_cluster

# Change if needed
set ::env(VERILOG_FILES) [glob $::env(DESIGN_DIR)/src/*.v]
set ::env(SYNTH_READ_BLACKBOX_LIB) 1

# Fill this
set ::env(CLOCK_PERIOD) "200"
set ::env(CLOCK_PORT) "clk"

set ::env(SYNTH_MAX_FANOUT) 7
set ::env(PL_TARGET_DENSITY) 0.35

set filename $::env(DESIGN_DIR)/$::env(PDK)_$::env(STD_CELL_LIBRARY)_config.tcl
if { [file exists $filename] == 1} {
    source $filename
}

Note: I've also tried changing the parameters a bit to see if anything changes, but the error is consistent.

Openlane commit: (I tried 2, both fail: d03377b84212527a269dfc58bb637a9e921af150 (Nov 13), d9584fd1923a525de4a69b3da5a9d6f59fa25d9f (newest as of Nov 16))

Repo of OpenLane design: https://github.com/TsaiAnson/exampleDesignOpenLaneMACCluster ^ This design synthesizes fine on rc4 while fails on develop.

Slack Post (openlane channel): https://skywater-pdk.slack.com/archives/C016H8WJMBR/p1605399551389700

Could someone take a look for the issue? Thanks in advance.

[diecaptain]

Since it is at STA stage, your synthesis was successful. The violation might be when OpenSTA is writing check reports. See the reports/synthesis directory to see which of the opensta.timing.rpt or opensta.min_max.rpt or opensta.rpt are missing. You can get an idea where the flow stopped from that. Since you have negative slack, there are paths that should be reported here, but the flow might not be finding them resulting in this error. Are you providing a sky*config.tcl file and have you enabled simple CTS? I have come across this type of error when simple CTS is enabled on a structural top block like yours. I didnt solve it, but STA at this stage is not really necessary, so run interactively and replace run_synthesis with run_yosys and disable simple CTS. You can spend time worrying if this repeats during STA after CTS.

[agorararmard]

@TsaiAnson: I reran that out of the box and it worked normally up to placement (which is expected since we changed the way padding is done and PL_TARGET_DENSITY needs to be modified). However, I didn't face that segfault. So it might be an issue with your RAM, memory, CPU, or even permissions?

Maybe try running one of the test designs that we know pass successfully through the flow on develop? md5 maybe since it has a similar cell count?

Here is a log proof:

CELL_PIN) [all_inputs]                                                [162/2087]
set cap_load [expr $::env(SYNTH_CAP_LOAD) / 1000.0]
puts "\[INFO\]: Setting load to: $cap_load"
[INFO]: Setting load to: 0.01765
set_load  $cap_load [all_outputs]
tns -3.94
wns -2.24
[INFO]: Synthesis was successful
[INFO]: Running Floorplanning...
[INFO]: Running Initial Floorplanning...
OpenROAD 0.9.0 e582f2522b

P.S: once your machine issue is resolved try out these configs (to pass placement):

set ::env(FP_CORE_UTIL) 35
set ::env(PL_TARGET_DENSITY) 0.40

This has nothing to do with the segfault.

[TsaiAnson]

Thank you @diecaptain @agorararmard for the advice.

I've done some more tinkering and finally got it to work! However, it seems like it's an open_pdks/skywater-pdk issue now (at least on my machine). After I reverted my open_pdks and skywater-pdk commits to the ones used in rc4, I was able to get the global placement error you mentioned earlier @agorararmard. I was then able to get past that error with your suggested config for FP_CORE_UTIL and PL_TARGET_DENSITY.

To summarize, here are the commits of openlane, open_pdks, skywater-pdk that failed (weird seg violation) and passed (completed with no errors):

Failed setup: openlane: d9584fd1923a525de4a69b3da5a9d6f59fa25d9f (develop) open_pdks: 94513d439f76501eacb39701f6e98f3b4f07dcdf (automatic from Makefile) skywater-pdk: d8e2cf1ba006ed01468aa60e7f4e85a1ece74ca4 (automatic from Makefile)

Working setup: openlane: d9584fd1923a525de4a69b3da5a9d6f59fa25d9f (develop) open_pdks: 48db3e1a428ae16f5d4c86e0b7679656cf8afe3d (rc4) skywater-pdk: 5cd70ed19fee8ea37c4e8dbd5c5c3eaa9886dd23 (rc4)

@agorararmard If you want to try to recreate the weird seg violation error, I believe you can just run make in the openlane dir to automatically update and build the later pdks that caused the weird seg violation on my machine.

[agorararmard]

@TsaiAnson: Actually, I did use the commits from the Makefile on develop. Anyhow, since this is resolved, I'm closing this issue.

[TsaiAnson]

Ah that's strange. Thanks for the heads up though.

[baragona]

I run into the same issue using the same git commits checked out from @TsaiAnson under "Working Setup". It may still be an issue.

[agorararmard]

@ojotoxy: So, to clarify you're using openlane:develop but with the pdk commits in openlane:rc4 (master)?