Open odxa20 opened 2 years ago
Ah, the vlsi Makefile always expects an SRAM compiler to be run, so you will need to remove the sram_generator
step within these lines: https://github.com/ucb-bar/chipyard/blob/main/vlsi/Makefile#L80-L101 as well as the dependence on the SRAM compiler output here: https://github.com/ucb-bar/chipyard/blob/main/vlsi/Makefile#L224.
Also, this is not a bug - you're just doing something that is not normal for the VLSI flow.
Thank you very much for explaining what was wrong. I will try this out right away.
May I suggest changing the documentation chapter 5.6 to make sure other people don't run into this error ?
May I suggest changing the documentation chapter 5.6 to make sure other people don't run into this error ?
I'll figure out how to have the Makefile manage this case instead.
Oh that would be even better. Thank you
Hello. We migrated our EDA tools to a new machine with more RAM and CPU cores to be able to build larger designs. When trying to synthesize SmallBoomConfig using the synflops argument Genus uses more than 70GB of RAM which starts occupying SWAP space. When running the synthesis with SRAMs the memory utilization does not go over 12GB. Is this a normal occurrence ?
Yes, this is expected. Keep in mind that SRAMs are black box macros, while a memory generated as synflops is a massive sea of flip-flops. This makes it orders of magnitude harder for Genus to time and optimize all of the paths to/from each flip-flop instance.
Thank you very much for the reply. That’s what I thought I just wanted to be sure. Do you have any suggestions on how to approach floor planning the SRAMs if I choose to use the SRAM compiler ? Is there a way to automatically do this ? If not does the SRAM compiler output a file that specifies what macros correspond to the RTL memory instances so that I can use this information to more easily do the floor planning manually in the design.yaml file ?
You can first start by having Innovus auto-place for you (see discussion here: https://groups.google.com/g/chipyard/c/Xa6cSqKnKOM).
I'm going to just answer this question here: https://groups.google.com/g/chipyard/c/ZFm1rQU24Vs
I am trying to place and route SmallBoom using the automatic floorplanning mode. Unfortunately the process halts after a while and I get the following message
CCOpt::Phase::Construction...
Stage::Clustering...
Clustering...
Initialize for clustering...
Clock DAG stats before clustering:
cell counts : b=0, i=0, icg=4668, nicg=1, l=0, total=4669
misc counts : r=1, pp=0
cell areas : b=0.000um^2, i=0.000um^2, icg=23956.923um^2, nicg=2.799um^2, l=0.000um^2, total=23959.722um^2
hp wire lengths : top=0.000um, trunk=0.000um, leaf=158445.756um, total=158445.756um
Clock DAG library cell distribution before clustering {count}:
ICGs: ICGx5_ASAP7_75t_SL: 4668
NICGs: AND2x6_ASAP7_75t_SL: 1
Computing optimal clock node locations...
Computing optimal clock node locations done.
flow.cputime flow.realtime timing.setup.tns timing.setup.wns snapshot
UM:* before clustering
Computing max distances from locked parents...
Computing distance_from_locked_parent_restrictions for 0 nodes driven by 0 locked parents
Computing max distances from locked parents done.
Initialize for clustering done. (took cpu=0:00:01.7 real=0:00:00.6)
flow.cputime flow.realtime timing.setup.tns timing.setup.wns snapshot
UM:* Initialize for clustering
Bottom-up phase...
Clustering bottom-up starting from leaves...
Clock tree timing engine global stage delay update for PVT_0P63V_100C.setup_delay:setup.late...
Clock tree timing engine global stage delay update for PVT_0P63V_100C.setup_delay:setup.late done. (took cpu=0:00:00.2 real=0:00:00.0)
Clustering clock_tree clock_clock...
**DIAG[/icd/cm_t1nb_002/INNOVUS211/Rel/21.11/main/lnx86_64_opt/21.11-s130_1/fe/src/ccopt/clocktree/timing/stagedelays.cpp:2879:void Ccopt::ClockTree::Timing::NodeDelays::Internals::Update()]: NonFatalAssert Failed: When the allowQueryStageDelay flag is set to No we shouldn't be here in NodeDelays::Internals::Update.
**DIAG[/icd/cm_t1nb_002/INNOVUS211/Rel/21.11/main/lnx86_64_opt/21.11-s130_1/fe/src/ccopt/clocktree/timing/structure.cpp:2946:void Ccopt::ClockTree::Timing::Structure::Internals::Update()]: NonFatalAssert Failed: When the AllowQueryStructure flag is not set we shouldn't be here in Structure::Internals::Update.
Do you have any ideas on what is happening ? innovus.log
Hm, I have not seen that error before. It seems like an internal error with clock tree synthesis. Are any of your SRAM macros overlapping? Especially the clock pins?
Can I check this if par has not finished normally ? If it were done I could open the design in innovus with the generated script but since it isn't I can't do that
You can. The open_chip
script will always point to the latest completed checkpoint. Since it failed in clock tree synthesis, the floorplan step should already have been completed.
Got it. I will use the script and will check for overlaps. The dimensions I picked for SmallBoomConfig where 4000x4000 (meaning 1000x1000 in actual dimensions due to the ASAP7 scaling). Could this be too small and thus be causing these errors ?
I don't know - I've never tried P&R-ing a SmallBoomConfig.
Ah ok thank you either way. Before you answered I was testing with the previous version of Innovus (20.11) and it just finished clock tree synthesis without error. It now seems to be stuck optimizing the routing. In each optimization iteration the rate of violation elimination keeps decreasing so I don't think the run will ever finish. I will increase the chip size and rerun place and route. Once floorplanning is done I will use the script to open innovus and check for overlaps
Yeah, I have seen the routing violation issue too, and so has another user: https://groups.google.com/g/chipyard/c/-8JUSeCx5qs. I don't know what's causing it, but I think it is version-dependent, but I don't remember when it started happening.
I just opened the chip that was produced by version 21.11 (version which produces the assertion failed error) using the generated script. The floorplan view is the following
I don't see any overlapping but the SRAM cells seem to be outside of the main chip which is very weird. Is this normal ?
They would be placed outside the floorplan area if they haven't been placed. Are you sure that this database is one where plan_design
has already been executed? Furthermore, you may want to turn off the view for all the metal or look at amoeba view instead.
I used the script after the error on the cts run presented itself so it should be the right database
What does the amoeba view show?
This is the view in amoeba mode with all the metal layers disabled
This is a zip of the par-rundir if you want to open the chip in innovus yourself https://drive.google.com/file/d/14AylGeHKTM7OIS_ix8uzJkpG61lk2T2c/view?usp=sharing
I won't be able to read your database (it won't be able to find paths). But, your amoeba view shows all the placed SRAMs. The problem appears to be that they're all abutting rather than overlapping - therefore pin access is not possible.
You can use this rough placement to generate your own floorplan constraints.
Would increasing the chip size even more make the automatic floor planner leave more room ?
I don't know - I don't use the auto floorplanner. It looks like your density right now is around 70%? That's somewhat high, so give it a try?
I will definitely try. I will do some tests in the morning and post the results. Thank you very much for all the help
Hello. I have finally managed to place and route SmallBoomConfig using the Automatic Floorplan Mode of Innovus. The previous error I mentioned was a bug in Innovus 21.11. After installing Innovus 21.14 (there was a big delay as we only had access to older versions of the tools through a third-party and had to sign a new contract to be able to download tools directly from Cadence) the problem was resolved. In version 21.14 the issue mentioned here https://groups.google.com/g/chipyard/c/-8JUSeCx5qs is also resolved. Looking at the Innovus log afterwards I noticed two errors which do not seem to be fatal as the place and route process finishes succesfully.
**ERROR: (TECHLIB-1171): The attribute 'values' of group 'fall_constraint' has one or more values which are greater than 100sec. This may lead to unexpected analysis results. (File /home/odxa20/chipyard/vlsi/build/chipyard.TestHarness.SmallBoomConfig-ChipTop/tech-asap7-cache/LIB/NLDM/asap7sc7p5t_SEQ_RVT_SS_nldm_201020.lib.gz, Line 2057)
**ERROR: (IMPTR-2101): Layer M10: Pitch=11520x9 is still less than min width=32000 + min spacing=640.
Do you have any ideas on what the cause of these errors might be ? Could they be caused by some inconsistencies in the ASAP7 PDK ?
After completing P&R I moved on to running a post P&R power estimation. The process finishes successfully but there are some issues. Firstly rail analysis fails in the same way as described here. Aside from this in the folders containing the active power analysis results there is no .rpt report file like there is in the static power analysis folders as shown in the attached screenshot
Is this normal or is it some sort of bug ? If it is normal how can I see what the power consumption results are ?
I really appreciate all the help you've given me and wish you are well
I forgot to mention that I also made a hook to automatically convert vpd files to vcd since vpd files created by VCS 2020 cause errors in Voltus 21. Is there a place to post this hook so that other people can also benefit from it ?
Are you using the vpd2vcd utility? How does your hook work?
Yes I am calling the vpd2vcd utility for all waveforms files
Since you are using hammer-sim, you can probably add your hook as a new option in the appropriate hammer plugin. @harrisonliew probably knows where to do this.
Apologies for the late reply.
The lib that throws that error has constraint tables like this:
fall_constraint (mpw_constraint_template_7x7) {
index_1 ("5, 10, 20, 40, 80, 160, 320");
values ( \
"46.8442, 44.4245, 46.6537, 50.354, 80.5664, 161.133, 321.045" \
);
}
There are values that are greater than 100 but these seem OK given that this particular one is for a CLK pin of a flip-flop. I don't see why this is an error vs. a warning, but it's not something you need to worry about.
The M10 layer error doesn't quite make sense to me, since there are only 9 layers in ASAP7 + the the Pad layer. The tech LEF doesn't contain any information with those dimensions. You can probably also ignore this since M10 isn't used anywhere.
As for the issue with the rail analysis + active power analysis, are you still seeing the issue with the tech/stdcell libraries failing to be merged, and then the dynamic power analysis not recognizing the ptiavg files? If so, this still seems to be a tool bug/incompatibility issue, not an issue with the commands given to it. Can you attach your newest Voltus logs to verify this?
Yes, you can make a PR with the hook in https://github.com/ucb-bar/hammer-synopsys-plugins/blob/master/sim/vcs/__init__.py as a new method. I think the easiest way to have the user run this is to add it to the end of the steps list dependent on some key, such as sim.vcs.to_vcd
, and then in your wrappers set your output waveforms to the .vcd
files instead of .vpd
files.
Thank you very much for replying. I am glad that the P&R errors are not critical
Moving on to power analysis there are two issues
Rail analysis fails. This can be attributed to the two errors previously identified
[09/17 01:55:53 7421s] Started Rail Analysis at 01:55:51 09/17/2022
[09/17 01:55:53 7421s]
[09/17 01:55:54 7421s]
[09/17 01:55:54 7421s] Opening dproc sockets
[09/17 01:55:54 7421s] connection complete
[09/17 01:55:54 7421s]
[09/17 01:55:54 7421s] Multi-CPU Configuration:
[09/17 01:55:54 7421s] #CPU: 24
[09/17 01:55:54 7421s] Mode: local
[09/17 01:55:54 7421s] Host: edabox
[09/17 01:55:54 7421s]
[09/17 01:55:54 7421s]
[09/17 01:55:54 7421s] Start distributed processes ...
[09/17 01:55:54 7421s] INFO (PRL-36): The timeout for a remote job to respond is 3600 seconds.
[09/17 01:55:54 7421s]
[09/17 01:55:54 7421s] Submit command for task runs will be: local
[09/17 01:55:54 7421s] INFO (PRL-725): EDP: start server on edabox:46043
[09/17 01:55:54 7421s]
[09/17 01:55:54 7421s] INFO (PRL-819): Connected to edabox 44667 0 ( PID=346123 ). Wait time was 1 seconds
[09/17 01:55:54 7421s]
[09/17 01:55:54 7421s] INFO (PRL-819): Connected to edabox 47629 1 ( PID=346132 ). Wait time was 1 seconds
[09/17 01:55:54 7421s]
[09/17 01:55:54 7421s] INFO (PRL-742): EDP: all 2 edp slave(s) are launched
[09/17 01:55:54 7421s]
[09/17 01:55:54 7421s] Begin Processing Cell Libraries for Rail Analysis
[09/17 01:55:55 7421s]
[09/17 01:55:55 7421s] Merging libraries:
[09/17 01:55:55 7421s] /home/odxa20/chipyard/vlsi/build/chipyard.TestHarness.SmallBoomConfig-ChipTop/tech-asap7-cache/tech_pgv/PVT_0P63V_100C/techonly.cl
[09/17 01:55:55 7421s] /home/odxa20/chipyard/vlsi/build/chipyard.TestHarness.SmallBoomConfig-ChipTop/tech-asap7-cache/stdcell_pgv/PVT_0P63V_100C/stdcells.cl
[09/17 01:55:55 7421s]
[09/17 01:55:55 7421s] Output library: ./work/lib_merged.cl
[09/17 01:55:55 7421s]
[09/17 01:56:00 7421s] ** ERROR: (VOLTUS_RAIL-4063): Failed to merge the PGV libraries during rail analysis.
[09/17 01:56:00 7421s] Use the -check_compatibility option of the check_pg_library command to check compatibility between the specified libraries before rerunning rail analysis.
[09/17 01:56:00 7421s] The following errors occurred during the merge:
[09/17 01:56:00 7421s] Error : The first library specified is not a TECHNOLOGY library. Please generate TECHNOLOGY library using command "set_pg_library_mode -celltype techonly". Specify the TECHNOLOGY PGV generated as the first PGV in the list.
[09/17 01:56:00 7421s] Ended Processing Cell Libraries for Rail Analysis: (cpu=0:00:00, real=0:00:06, mem(process/total/peak)=38.96MB/14910.28MB/31201.21MB)
[09/17 01:56:00 7421s] ** ERROR: (VOLTUS_RAIL-1050): Failed to merge the PGV libraries during rail analysis.
[09/17 01:56:00 7421s] Use the -check_compatibility option of the check_pg_library command to check compatibility between the specified libraries before rerunning rail analysis.
[09/17 01:56:00 7421s] *** Error: In processing command script
[09/17 01:56:00 7421s] Library merging failed.Run OCI extraction flow failed.
[09/17 01:56:00 7421s]
[09/17 01:56:00 7421s] 1 error was found while processing the file /tmp/ssv_tmpdir_342080_WVeqCs/voltus_rail_342080.cmd
[09/17 01:56:02 7421s]
[09/17 01:56:02 7421s] Rail Analysis Statistics:
[09/17 01:56:02 7421s] Warning messages: 0
[09/17 01:56:02 7421s] Error messages: 2
[09/17 01:56:02 7421s]
[09/17 01:56:02 7421s] Rail Analysis is unsuccessful due to errors.
[09/17 01:56:02 7421s]
[09/17 01:56:02 7421s] Finished Rail Analysis at 01:56:02 09/17/2022 (cpu=0:00:00, real=0:00:11, peak mem=6527.91MB)
[09/17 01:56:02 7421s] Current Voltus IC Power Integrity Solution resource usage: (total cpu=1:53:31, real=0:17:46, mem=6384.16MB)
[09/17 01:56:02 7421s] voltus_rail exited unsuccessfully.
and
[09/17 01:56:31 7423s] @file 89: puts "set_rail_analysis_config -method dynamic -accuracy hd -process_techgen_em_rules true -em_peak_analysis true -enable_rlrp_analysis true -gif_resolution high -verbosity true -power_grid_libraries { /home/odxa20/chipyard/vlsi/build/chipyard.TestHarness.SmallBoomConfig-ChipTop/tech-asap7-cache/tech_pgv/PVT_0P63V_100C/techonly.cl /home/odxa20/chipyard/vlsi/build/chipyard.TestHarness.SmallBoomConfig-ChipTop/tech-asap7-cache/stdcell_pgv/PVT_0P63V_100C/stdcells.cl } -analysis_view PVT_0P63V_100C.setup_view -temperature 100.0"
[09/17 01:56:31 7423s] set_rail_analysis_config -method dynamic -accuracy hd -process_techgen_em_rules true -em_peak_analysis true -enable_rlrp_analysis true -gif_resolution high -verbosity true -power_grid_libraries { /home/odxa20/chipyard/vlsi/build/chipyard.TestHarness.SmallBoomConfig-ChipTop/tech-asap7-cache/tech_pgv/PVT_0P63V_100C/techonly.cl /home/odxa20/chipyard/vlsi/build/chipyard.TestHarness.SmallBoomConfig-ChipTop/tech-asap7-cache/stdcell_pgv/PVT_0P63V_100C/stdcells.cl } -analysis_view PVT_0P63V_100C.setup_view -temperature 100.0
[09/17 01:56:31 7423s] @@file 90: set_rail_analysis_config -method dynamic -accuracy hd -process_techgen_em_rules true -em_peak_analysis true -enable_rlrp_analysis true -gif_resolution high -verbosity true -power_grid_libraries { /home/odxa20/chipyard/vlsi/build/chipyard.TestHarness.SmallBoomConfig-ChipTop/tech-asap7-cache/tech_pgv/PVT_0P63V_100C/techonly.cl /home/odxa20/chipyard/vlsi/build/chipyard.TestHarness.SmallBoomConfig-ChipTop/tech-asap7-cache/stdcell_pgv/PVT_0P63V_100C/stdcells.cl } -analysis_view PVT_0P63V_100C.setup_view -temperature 100.0
[09/17 01:56:31 7423s] **WARN: (VOLTUS-1179): Settings of all PG nets have been reset to default due to change in the analysis view using set_rail_analysis_mode. Re-run set_pg_nets to set threshold values based on design requirements.
[09/17 01:56:31 7423s] #set_pg_nets -net VDD -voltage 0.63 -threshold 0.567
[09/17 01:56:31 7423s] #set_pg_nets -net VSS -voltage 0.0 -threshold 0.063
[09/17 01:56:31 7423s] #set_rail_analysis_domain -name AO -pwrnets VDD -gndnets VSS -threshold 0.10
[09/17 01:56:31 7423s] #set_rail_analysis_domain -name ALL -pwrnets VDD -gndnets VSS -threshold 0.10
[09/17 01:56:31 7423s] @file 91: puts "set_power_data -reset"
[09/17 01:56:31 7423s] set_power_data -reset
[09/17 01:56:31 7423s] @@file 92: set_power_data -reset
[09/17 01:56:31 7423s] @file 93: puts "set_power_data -format current { activePower.rv32ui-p-simple.vcd.PVT_0P63V_100C.setup_view/dynamic_VDD.ptiavg activePower.rv32ui-p-simple.vcd.PVT_0P63V_100C.setup_view/dynamic_VSS.ptiavg }"
[09/17 01:56:31 7423s] set_power_data -format current { activePower.rv32ui-p-simple.vcd.PVT_0P63V_100C.setup_view/dynamic_VDD.ptiavg activePower.rv32ui-p-simple.vcd.PVT_0P63V_100C.setup_view/dynamic_VSS.ptiavg }
[09/17 01:56:31 7423s] @@file 94: set_power_data -format current { activePower.rv32ui-p-simple.vcd.PVT_0P63V_100C.setup_view/dynamic_VDD.ptiavg activePower.rv32ui-p-simple.vcd.PVT_0P63V_100C.setup_view/dynamic_VSS.ptiavg }
[09/17 01:56:31 7423s] @file 95: puts "report_rail -output_dir activeRailReports.rv32ui-p-simple.vcd -type domain ALL"
[09/17 01:56:31 7423s] report_rail -output_dir activeRailReports.rv32ui-p-simple.vcd -type domain ALL
[09/17 01:56:31 7423s] @@file 96: report_rail -output_dir activeRailReports.rv32ui-p-simple.vcd -type domain ALL
[09/17 01:56:31 7423s] **ERROR: (VOLTUS-1322): Detect missing files for rail analysis. Please check ./rail_missing_files for more details.
The ptiavg files that are reported missing do exist but may be corrupted. I am attaching the voltus log file and ptiavg files in question voltus.zip
Active power analysis does not output a report file Active power analysis seems to finish without errors but a report file is not generated. Maybe this is related to the following message I found in the log file
[09/17 01:41:57 1040s] ** WARN: (VOLTUS_POWR-2031): The report_power command in the dynamic mode is not incremental.
[09/17 01:41:57 1040s] To generate a text-based power reports from the dynamic run, user must specify report_power along with relevant
[09/17 01:41:57 1040s] parameters using the set_power_include_file command.
I really appreciate the support. I will attempt to make a PR with the hook once we clear up these issues
Unfortunately, I don't think I can help you out with the issue with the PGV libraries and the ptiavg files. These are files internal to your version of Voltus and your run environment. I'd suggest opening a ticket in the Cadence Learning and Support Portal detailing your issue. In your ticket, you should also include the commands used to create the PGV libraries and the power analysis (i.e. the Tcl files generated by Hammer) - we've already looked at these in detail and found nothing wrong with how the rail analysis is commanded.
I see. You can add the -out_file <file>
option wherever report_power
is called. The first instance is here: https://github.com/ucb-bar/hammer-cadence-plugins/blob/730f87bc8dd54cc558dc48bcd4b4bf7fa4312e68/power/voltus/__init__.py#L491 and there are a few more places in the rest of the file. We'd appreciate a PR if you get it working.
P.S. until the rail analysis issues are resolved, for your class, I'd suggest just adding removal hooks for static_rail
and active_rail
steps for now.
Thank you very much for your input. I will try and contact Cadence support regarding the library merging issue. I have been trying to make the report_power command output a report file but I've had no luck. Dynamic power analysis after appending the -out_file <file>
option just outputs a blank power.rpt file. Do you have any ideas on why this could be happening ?
No, I don't. Does dynamic power analysis seem to be running without errors? Does static power analysis successfully write a report file? Might there be something wrong with your input waveform?
I just checked. There are no errors reported during dynamic power analysis. Static power analysis also outputs a report file. This warning
[09/17 01:41:57 1040s] ** WARN: (VOLTUS_POWR-2031): The report_power command in the dynamic mode is not incremental.
[09/17 01:41:57 1040s] To generate a text-based power reports from the dynamic run, user must specify report_power along with relevant
[09/17 01:41:57 1040s] parameters using the set_power_include_file command.
indicates that in order to get a text-based report one needs to also call the set_power_include_file
command with an appropriate input file. Having searched the documentation though I couldn't find what said file should contain in order to make Voltus produce the desired power report
According to the Voltus TCR, set_power_include_file
:
Specifies a power analysis include file. You can use the set_power_include_file command to specify certain commands of the dynamic engine which do not have a Voltus equivalent. For more information on these commands, refer to the Power Analysis Options chapter. The set_power_include_file command is required only if the power_static_netlist attribute is set to def. Note: The set_power_include_file command is used to pass the legacy power analysis commands and report_power command options only in the dynamic mode (vectorbased and vectorless). If the power_method attribute is set to static, the set_power_include_file command is ignored.
I don't know what legacy commands this could be (I guess Cadence acquired some other company's tool?), but there's an example on that page: report_power -cell {*INV} -outpower Invpower.rep
that I guess you can emulate. Weird that the static & dynamic analysis engines are different.
Background Work
Chipyard Version and Hash
Release: 1.7.0 Hash: 4a11896
OS Setup
Result of
uname -a
Linux edabox 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 x86_64 x86_64 x86_64 GNU/Linux
Result of
lsb_release -a
Other Setup
No response
Current Behavior
I am trying to synthesize TinyRocketConfig using the ASAP7 PDK with the macrocompiler flag
--mode synflops
. I am running the following command firstmake buildfile MACROCOMPILER_MODE='--mode synflops' CONFIG=TinyRocketConfig
and am getting the following errorI am attaching my .yml configuration files config_yml.zip
Expected Behavior
Should finish normally
Other Information
No response