search

YosysHQ / yosys

Yosys Open SYnthesis Suite
https://yosyshq.net/yosys/
ISC License
3.37k stars 871 forks source link

`autoname` memory usage very high with CPU generated from `ghdl-yosys-plugin`. #2816

Open cr1901 opened 3 years ago

cr1901 commented 3 years ago

Steps to reproduce the issue

When specifically trying to synthesize a Microwatt CPU SoC using yosys, I've noticed that the autoname pass takes an unusual amount of memory that I can't seem to duplicate with other designs. Microwatt is a large CPU, and while I can't easily synthesize another SoC quite as big as that one at present, I've noticed that a SoC about half the size uses 1/4 of the memory, and autoname is not where the most time is spent.

Here is an example run of yosys trying to synthesize the Microwatt CPU using ghdl-yosys-plugin:

6.48. Executing AUTONAME pass.
Renamed 2309229 objects in module gsd_orangecrab (484 iterations).
<suppressed ~63121 debug messages>

6.49. Executing HIERARCHY pass (managing design hierarchy).

6.49.1. Analyzing design hierarchy..
Top module:  \gsd_orangecrab

6.49.2. Analyzing design hierarchy..
Top module:  \gsd_orangecrab
Removed 0 unused modules.

6.50. Printing statistics.

=== gsd_orangecrab ===

   Number of wires:              30195
   Number of wire bits:         176365
   Number of public wires:       30195
   Number of public wire bits:  176365
   Number of memories:               0
   Number of memory bits:            0
   Number of processes:              0
   Number of cells:              41188
     CCU2C                        1409
     CLKDIVF                         1
     DDRDLLA                         1
     DELAYG                         43
     DP16KD                         38
     DQSBUFM                         2
     ECLKBRIDGECS                    1
     ECLKSYNCB                       1
     EHXPLLL                         2
     IDDRX2DQA                      16
     L6MUX21                      1765
     LUT4                        23896
     MULT18X18D                     16
     ODDRX2DQA                      18
     ODDRX2DQSB                      2
     ODDRX2F                        27
     PDPW16KD                       16
     PFUMX                        5921
     TRELLIS_DPR16X4               331
     TRELLIS_FF                   7644
     TRELLIS_IO                     20
     TSHX2DQA                       16
     TSHX2DQSA                       2

6.51. Executing CHECK pass (checking for obvious problems).
Checking module gsd_orangecrab...
Found and reported 0 problems.

6.52. Executing JSON backend.

Warnings: 4 unique messages, 4 total
End of script. Logfile hash: 9510fc3eb5, CPU: user 202.60s system 2.28s, MEM: 4975.58 MB peak
Yosys 0.9+4081 (git sha1 82f5829ab, sccache gcc 9.3.0-17ubuntu1~20.04 -fPIC -Os)
Time spent: 32% 1x autoname (72 sec), 13% 44x opt_clean (31 sec), ...

The bulk of the time is spent in autoname, and yosys takes nearly 5GB to run to completion.

By comparison, a Linux RISCV SoC about half the size in LUT usage takes between 1/4-1/3 of the memory, and autoname is not in the top two passes where time was spent (so less than 20 seconds):

5.48. Executing AUTONAME pass.
Renamed 727614 objects in module gsd_orangecrab (221 iterations).
<suppressed ~31569 debug messages>

5.49. Executing HIERARCHY pass (managing design hierarchy).

5.49.1. Analyzing design hierarchy..
Top module:  \gsd_orangecrab

5.49.2. Analyzing design hierarchy..
Top module:  \gsd_orangecrab
Removed 0 unused modules.

5.50. Printing statistics.

=== gsd_orangecrab ===

   Number of wires:              15703
   Number of wire bits:          68569
   Number of public wires:       15703
   Number of public wire bits:   68569
   Number of memories:               0
   Number of memory bits:            0
   Number of processes:              0
   Number of cells:              21936
     CCU2C                         522
     CLKDIVF                         1
     DDRDLLA                         1
     DELAYG                         43
     DP16KD                          8
     DQSBUFM                         2
     ECLKBRIDGECS                    1
     ECLKSYNCB                       1
     EHXPLLL                         2
     IDDRX2DQA                      16
     L6MUX21                       432
     LUT4                        12021
     MULT18X18D                      4
     ODDRX2DQA                      18
     ODDRX2DQSB                      2
     ODDRX2F                        27
     PDPW16KD                       29
     PFUMX                        2066
     TRELLIS_DPR16X4               163
     TRELLIS_FF                   6539
     TRELLIS_IO                     20
     TSHX2DQA                       16
     TSHX2DQSA                       2

5.51. Executing CHECK pass (checking for obvious problems).
Checking module gsd_orangecrab...
Found and reported 0 problems.

5.52. Executing JSON backend.

Warnings: 2 unique messages, 2 total
End of script. Logfile hash: 29c8f2c7d5, CPU: user 104.14s system 0.63s, MEM: 1339.80 MB peak
Yosys 0.9+4081 (git sha1 82f5829ab, sccache gcc 9.3.0-17ubuntu1~20.04 -fPIC -Os)
Time spent: 22% 54x opt_clean (25 sec), 17% 57x opt_expr (20 sec), ...

To duplicate, I've included an "M"VCE of the RTLIL of the Microwatt SoC; run yosys autoname.il to duplicate: autoname.zip

Expected behavior

I would not expect a design twice as big to take nearly 4 times as much memory for the autoname pass to run, nor be where the bulk of time is spent in a design.

Actual behavior

autoname takes a long time to run and a large amount of memory. I'm opening this issue because I want to know if autoname memory usage is a quirk of ghdl-yosys-plugin that can be avoided/fixed, and/or autoname is working as intended. I noticed that that the Linux RISCV SoC has approximately 1/3 of the "objects" to autoname as the Microwatt SoC, so maybe autoname's memory usage is linear w/ the number of objects?

anuejn commented 3 years ago

I am hitting something similiar with a nmigen design that produces ~200k lines of rtlil. Maybe this helps? mipi_demo.zip

zachjs commented 3 years ago

I have a some simple optimizations which look promising. I'll have them pushed soon.

nakengelhardt commented 3 years ago

Thanks Zach! I'm sure there is lots of room for improvement... For context, autoname was something Claire threw together quickly to help me debug something where I needed to trace where signals came from rather late in the flow (I don't really remember the details anymore). It was immediately obvious that it didn't scale well, but it did seem useful enough to have around. That's why it was only put in synth_ice40 and none of the other flows; we figured ice40 designs would be small enough not to run into performance problems (it seems we were wrong on that guess!)

anuejn commented 3 years ago

That's why it was only put in synth_ice40 and none of the other flows

hm... it seems to be enabled for ecp5 too nowdays? Is that intentional?

@zachjs thanks a lot :):)

cr1901 commented 3 years ago 
william@xubuntu-dtrain:~/Projects/FPGA/litex/litex-boards/autoname$ yosys autoname.ys

 /----------------------------------------------------------------------------\
 |                                                                            |
 |  yosys -- Yosys Open SYnthesis Suite                                       |
 |                                                                            |
 |  Copyright (C) 2012 - 2020  Claire Wolf <claire@symbioticeda.com>          |
 |                                                                            |
 |  Permission to use, copy, modify, and/or distribute this software for any  |
 |  purpose with or without fee is hereby granted, provided that the above    |
 |  copyright notice and this permission notice appear in all copies.         |
 |                                                                            |
 |  THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES  |
 |  WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF          |
 |  MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR   |
 |  ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES    |
 |  WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN     |
 |  ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF   |
 |  OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.            |
 |                                                                            |
 \----------------------------------------------------------------------------/

 Yosys 0.9+4081 (git sha1 2e697f565, sccache gcc 9.3.0-17ubuntu1~20.04 -fPIC -Os)

-- Executing script file `autoname.ys' --

1. Executing RTLIL frontend.
Input filename: ./autoname.il

2. Executing AUTONAME pass.
Renamed 2406431 objects in module gsd_orangecrab (490 iterations).
<suppressed ~61189 debug messages>

End of script. Logfile hash: 018655bf6d, CPU: user 22.74s system 0.37s, MEM: 716.26 MB peak
Yosys 0.9+4081 (git sha1 2e697f565, sccache gcc 9.3.0-17ubuntu1~20.04 -fPIC -Os)
Time spent: 93% 1x autoname (21 sec), 6% 2x read_ilang (1 sec)

Even these few changes are much better, thanks @zachjs! And this is just the low-hanging fruit?