Run out of RAM when compiling with 4Gbyte

[darkstar007]

Hi,

When I try to compile nextpnr (configured with cmake . -DARCH=all -DTRELLIS_ROOT=/home/matt/prjtrellis), it runs out of RAM (I have 4GB, no swap, no graphics) with the following error:

[  2%] Generating ecp5/chipdbs/chipdb-85k.bba
Traceback (most recent call last):
  File "/home/matt/nextpnr/ecp5/trellis_import.py", line 381, in <module>
    main()
  File "/home/matt/nextpnr/ecp5/trellis_import.py", line 370, in main
    ddrg = pytrellis.make_dedup_chipdb(chip)
MemoryError
make[2]: *** [CMakeFiles/ecp5_chipdb.dir/build.make:90: ecp5/chipdbs/chipdb-85k.bba] Error 1
make[1]: *** [CMakeFiles/Makefile2:72: CMakeFiles/ecp5_chipdb.dir/all] Error 2
make: *** [Makefile:130: all] Error 2

Can anything be done to reduce its RAM usage?

Thanks,

Matt

[eddiehung]

@darkstar007 It is unfortunate that building the DB currently requires so much memory. I believe the ECP-85k is probably the biggest device semi-officially supported by nextpnr. If you can't build the DB then you may well struggle to run any designs through nextpnr that utilise that capacity.

If you can must use ECP5 device, then consider a smaller device: https://github.com/YosysHQ/nextpnr/blob/3b2b15dc4a6cdf9cadab96b1db5483d4f7082dff/ecp5/arch.cc#L66 contains a define that can be set to use only a single, smaller device, though you may have to adapt this: https://github.com/YosysHQ/nextpnr/blob/3b2b15dc4a6cdf9cadab96b1db5483d4f7082dff/ice40/family.cmake#L1 to get only that DB to build.

Otherwise if you just want to play with nextpnr, you can try the smaller ice40 architecture.

Hope that helps!

[daveshah1]

I think the long term solution to this problem is to distribute the (plaintext, so not strictly a binary blob) .bba files which are expensive to build but <10MB in a tar.xz file, every other part of the build should be considerably less memory-intensive. Infrastructure would need to be set up to handle building and distributing these though.

[darkstar007]

@eddiehung OK, thanks. I'll see what I can do.

What would be the recommended minimum amount of ram for a 85k device? (If project x-ray eventually runs on Kintex-7 devices, how much ram would be required then?)

I'll leave this bug open so that other people can be aware of this issue.

Thanks,

Matt

[daveshah1]

nextpnr uses a deduplicated database approach, which means that it's memory usage while running is quite low (but our current approach to build the database is quite expensive). For a medium-size design on the 85k, I am seeing a memory usage of around 250MB (design size should not greatly affect this).

This deduplicated approach could also be applied to Xilinx devices in the future, and I would expect their memory usage to be on a similar order of magnitude as a result.

[darkstar007]

@daveshah1 OK, thanks for the numbers. So it seems like when it is packaged, by say Debian, it will be fine to run (as the database will have been created by them) and memory usage when running is fine. As there currently (and understandably) no packages, I suspect I could probably just copy the files from my desktop (which has much more ram and builds it fine).

[daveshah1]

Yes, once packaged it should be fine (although right now I do not consider Trellis to be stable enough to package).

I think in the future there should also be a build option to pass to CMake that will tell it to fetch the databases prebuilt from a build server, but build everything else locally.

[darkstar007]

OK, thanks for all your help.

[q3k]

FWIW, trellis_import.py seems to currently peak at 3814M RES.

[gojimmypi]

I can confirm this is still an issue:

[ 38%] Generating ecp5/chipdbs/chipdb-25k.bba
[ 40%] Generating ecp5/chipdbs/chipdb-45k.bba
[ 41%] Generating ecp5/chipdbs/chipdb-85k.bba
Child killed
make[2]: *** [ecp5/chipdbs/chipdb-85k.bba] Error 1
CMakeFiles/ecp5_chipdb.dir/build.make:97: recipe for target 'ecp5/chipdbs/chipdb-85k.bba' failed
make[1]: *** [CMakeFiles/ecp5_chipdb.dir/all] Error 2
CMakeFiles/Makefile2:143: recipe for target 'CMakeFiles/ecp5_chipdb.dir/all' failed
Makefile:129: recipe for target 'all' failed
make: *** [all] Error 2

As seen with dmesg:

gojimmypi@ubuntu:~/workspace/ulx3s_fpga_toolchain$ dmesg | tail -n 10
[58444.221717] [   9048]  1000  9048    10490        1   122880      161             0 make
[58444.221718] [   9049]  1000  9049     3645        0    69632       24             0 tee
[58444.221718] [   9053]  1000  9053    10490        1   110592      146             0 make
[58444.221719] [   9285]  1000  9285    10490        1   114688      148             0 make
[58444.221720] [   9302]  1000  9302     1156        0    53248       23             0 sh
[58444.221721] [   9303]  1000  9303    43774        1   307200      385             0 cmake
[58444.221722] [   9304]  1000  9304   870340   816858  6991872    37214             0 python3
[58444.221723] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=python3,pid=9304,uid=1000
[58444.221728] Out of memory: Killed process 9304 (python3) total-vm:3481360kB, anon-rss:3267432kB, file-rss:0kB, shmem-rss:0kB
[58444.452731] oom_reaper: reaped process 9304 (python3), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
gojimmypi@ubuntu:~/workspace/ulx3s_fpga_toolchain$ 

It appears that 5GB RAM is the minimum to make nextpnr-ecp5 happy at this time.

I'm using this script to build the toolchain in an Ubuntu 18.04 LTS VM with the latest everything as of today.

See also 4GB minimum needed for prjtrellis confirming what @q3k saw a couple of years ago.