HLS cosim error when using too many ports

[RobertGlein]

This issue has been reported in a forum post but there are no replies to that: https://forums.xilinx.com/t5/High-Level-Synthesis-HLS/Cosim-fails-to-open-file-limit-on-the-number-of-ports-in-Vivado/td-p/1130888

Summary: For HLS designs with a large number of ports csim and csynth pass but cosim reports an error. This was not the case for Vivado HLS 2019.1 and earlier. We see this error in Vivado/Vitis HLS 2019.2 and later. https://github.com/cms-tracklet/firmware-hls/actions/runs/236781634 https://gitlab.cern.ch/rglein/firmware-hls_actions/-/jobs/9850280#L676

We tried it within a Docker container (not supported by Xilinx) hosted in MacOS X and it happened to work. We tried it on different systems using different operating systems (mostly supported ones) and there was no issue with Vivado HLS 2019.1 and earlier as well as in the Docker container for later versions.

Error message:

## run all
WARNING: file ../tv/cdatafile/c.testFunction.autotvin_array_508.dat could not be opened
Failed to open file "../tv/cdatafile/c.testFunction.autotvin_array_508.dat"!
ERROR: Simulation using HLS TB failed.
$finish called at time : 110 ns : File "/home/gitlab-runner/builds/biTwR5NQ/1/rglein/firmware-hls_actions/ulimit_test/test/solution1/sim/verilog/testFunction.autotb.v" Line 63135
## quit

[RobertGlein]

It is solved by increasing the limit for the maximum number of files:

ulimit -a # Reports the limit
ulimit -S -n 2048 # Sets the soft limit to 2048
ulimit -a # Reports the limit

If the soft limit exceeds the hard this has to increased as well (sudo needed): ulimit -H -n 2048

It is fixed here: https://github.com/cms-tracklet/firmware-hls/commit/eb1afe2ba1e4b24fdc58c2e6f1e73aec8302834a https://github.com/cms-tracklet/firmware-hls/actions/runs/236838216 https://gitlab.cern.ch/rglein/firmware-hls_actions/-/jobs/9850487#L707

Further Mei-Li tried it with ModelSim and there the error of the file limit is reported correctly.

Here are the comments from the Xilinx development team as the answer to our service request:

• I tried the provided testcase both in 2019.1 and 2019.2/2020.1 on xsjapps58. It passed few times with 2019.1 but it also failed under certain circumstances, especially when I re-ran XSim compilation (this is even with 2019.1).
• I think with 2019.1 total number of files opened by the flow is just touching max limit allowed i.e. 1024. This includes 1016 (i.e. 508*2) files opened by generated Verilog testbench, then all support files, log files etc. opened by tools (including XSim).
• In 2019.1, this number is right around 1024 (mostly under).
• In 2020.1, it may be the case that some parts of the flow have changed or some tool has changed such that we are slightly exceeding that number of 1024 for this test and failing there.
• But again if the testbench opened 1024 files instead of 1016 (i.e have 512 streams instead of 508), we will hit this limit regardless of the build.
• My point is this test opens 1016 files which stretch file i/o limit and leaves a very little margin to accommodate file i/o needed by tools for their operation (such as opening temporary files, log files etc.), and the behavior will change depending on how many such extra files tools are opening.
• This is my observation so far.
• In my opinion, this test case has shown us that if HLS generates this kind of HDL that has over 1000 file open operations, we need to make users aware that O.S. setting may not allow this and there is a workaround of modifying this using ulimit command.