DSLX: Order of expressions breaks tests; RAM hangs

Introduction

Observed unexpected behavior whilst developing new module and tests:

test_proc or proc (procA) can affect execution of another test_proc (procB)
order of spawn expressions in test_proc affects operation

I am not sure if these are related to each other, but I was unable to reproduce the issue on a smaller test case. Faulty code is on branch bug-procs-order, rebased on commit 609a7ab89d96d2a7396d2418d00d30e4cb57b119

I executed the tests with a prepared rule:

bazel run //xls/modules/axi4/dma:test_fifo

Previously, I had opened an issue related to order of statements #1191, which may be related. This issue is a blocker for #1208 .

My design

There are:

2 test_proc, which interact with top-level entity FIFO
FIFO spawns FIFORAM, and 2 processes to manage input and output: Reader, Writer
Process named fifo_synth is a wrapper to set parametrics to constant values and use it to generate verilog

Side-effect on another `test_proc`

Issue:

If only test_fifo is present in the file, it ends successfully, so I assume the test is OK.
Then I added a second test test_double_fifo_gpf, which cause the the original proc test_fifo to never end and as a result be killed by the runner (internal error: DEADLINE_EXCEEDED: Exceeded limit of 100000 proc ticks before terminating)
The same exact thing happens if test_fifo is present and fifo_synth is uncommented.

Link to code

Order of `spawn` expressions

In test_double_fifo_gpf I spawn 2 FIFOs to test a more complicated scenario and it turns out that the order of spawn expressions has effect on the execution.

Link to code

Order #1

spawn FIFO<TEST_1_DATA_W, TEST_1_DATA_W_DIV8, TEST_1_ID_W, TEST_1_DEST_W, TEST_1_ADDR_W, TEST_1_FIFO_L>
(ch_fifo1_read_s, ch_fifo1_write_r);
spawn FIFO<TEST_1_DATA_W, TEST_1_DATA_W_DIV8, TEST_1_ID_W, TEST_1_DEST_W, TEST_1_ADDR_W, TEST_1_FIFO_L>
(ch_fifo0_read_s, ch_fifo0_write_r);

I trace the state of 2 RAMs in 2 FIFOs

1st FIFO:

RAM State = [10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 0, 0, 0, 0, 0, 0]

2nd FIFO:

RAM State = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

This is almost what I expected whilst developing. There are writes to first FIFO and I expect that later data will go through the GPF and into the 2nd FIFO. Thanks to the trace_channels option, I can confirm that a valid request is sent to RAM and my processes hang on awaiting a response from RAM.

Why would either 1 or 2 RAMs hang indefinitely?

Order #2 (reverse of #1)

spawn FIFO<TEST_1_DATA_W, TEST_1_DATA_W_DIV8, TEST_1_ID_W, TEST_1_DEST_W, TEST_1_ADDR_W, TEST_1_FIFO_L>
(ch_fifo0_read_s, ch_fifo0_write_r);

spawn FIFO<TEST_1_DATA_W, TEST_1_DATA_W_DIV8, TEST_1_ID_W, TEST_1_DEST_W, TEST_1_ADDR_W, TEST_1_FIFO_L>
(ch_fifo1_read_s, ch_fifo1_write_r);

I trace the state of 2 RAMs in 2 FIFOs

1st FIFO:

RAM State = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

2nd FIFO:

RAM State = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

Both are always empty, not even the first write occurs anymore.

Can you please help me with understanding/debugging why this occurs?

google / xls