openhwgroup / cva6

The CORE-V CVA6 is an Application class 6-stage RISC-V CPU capable of booting Linux
https://docs.openhwgroup.org/projects/cva6-user-manual/
Other
2.27k stars 686 forks source link

AXI-4 Protocol for AMO #2525

Open Arhum-Ahmed opened 1 month ago

Arhum-Ahmed commented 1 month ago

Hi,

I was testing the AMO instructions. I observed that in the axi adapter, communication on AW and W channels are being carried out simultaneously (in the same cycle). I also checked the TB of pulp repo and found the same interface there. But, according to the AXI-4 protocol specification, first the data should be written on AW channel and in the next cycle in the W channel. So, when I changed the interface, according to the specification, the tests started to hang. I wanted to know that is this interface implementation specific to this AMO only, or am I missing something?

niwis commented 1 month ago

Hi @Arhum-Ahmed, the AW and W channels are decoupled, and there are no requirements for ordering in AXI. See A3.4 of the AXI spec (issue K):

The protocol does not define any other relationship between the channels.

The lack of relationship means, for example, that the write data can appear at an interface before the write request for the transaction. This can occur if the write request channel contains more register stages than the write data channel. Similarly, the write data might appear in the same cycle as the request.

Where do you see the requirement for sequencing AW and W?

Arhum-Ahmed commented 1 month ago

Actually the problem occurs when I change the AMO's TB transaction behavior to sequenced communication (first using the AW channel, then the W channel). After making this change, I ran a test which had load-reserved/store-conditional (lr/sc) AMO instruction in it. On the sc-instruction, this change caused incorrect response from the B channel. The B.VALID arrived along with the AW.READY, which is against the spec (according to spec there should be a delay of at least one clock cycle).

The way I'm sending request:

  1. Setting AW.Valid signal. Sending AW channel's data (atop, addr, etc). For sc (store conditional) setting AW.Lock to 1 otherwise 0.
  2. Waits for AW.Ready signal from AMO. After receiving this signal setting AW.Valid to 0.
  3. Setting W.Valid and W.Last to 1. Sending W channel's data (strb, data, etc).
  4. Waits for W.Ready signal, after that sets W.Valid to 0.
  5. Waits for B.Valid, response from AMO. (This is the part where the TB expects the B.Valid response to arrive after W.Ready, but it gets asserted along with W.Ready)

What might be the issue?

CVA-6 AMO's TB Code for Reference:

          // Send AW and W request
            ax_beat.ax_id    = id;
            ax_beat.ax_user  = user;
            ax_beat.ax_addr  = address;
            ax_beat.ax_size  = size;
            ax_beat.ax_burst = axi_pkg::BURST_INCR;
            w_beat.w_data    = axi_data;
            w_beat.w_strb    = strb;
            w_beat.w_last    = 1'b1;
            if (atop == ATOP_LRSC) begin
                // LRSC pair
                axi_read(address, result, size, id, user, 1'b1);
                rand_delay(0,10*RAND_DELAY);
                ax_beat.ax_atop = '0;
                ax_beat.ax_lock = 1'b1;
            end else begin
                ax_beat.ax_atop = atop;
                ax_beat.ax_lock = 1'b0;
            end
            fork
                // AW
                begin
                    rand_delay(0,RAND_DELAY);
                    send_aw(ax_beat);
                end
                // W
                begin
                    rand_delay(0,RAND_DELAY);
                    send_w(w_beat);
                end
            join
            // Wait for B response
            fork
                // B
                begin
                    rand_delay(0,RAND_DELAY);
                    recv_b(b_beat);
                    b_resp = b_beat.b_resp;
                    if (b_beat.b_id != id) begin
                        $display("%0t: %d: AXI WRITE: b_id (0x%x) did not match aw_id (0x%x)", $time, obj_id, b_beat.b_id, id);
                    end
                end
                // R response if atop
                begin
                    if (atop[axi_pkg::ATOP_R_RESP]) begin // Atomic operations with read response
                        rand_delay(0,RAND_DELAY);
                        recv_r(r_beat);
                        result = r_beat.r_data;
                        if (r_beat.r_id != id) begin
                            $display("%0t: %d: AXI WRITE: r_id (0x%x) did not match aw_id (0x%x)", $time, obj_id, r_beat.r_id, id);
                        end
                    end
                end
            join
github-actions[bot] commented 18 hours ago

👋 Hi there!

This issue seems inactive. Need more help? Feel free to update us. If there are no updates within the next few days, we'll go ahead and close this issue. 😊