Wrong slave port is selected

[SebastianZa]

Hi all,

I have a problem with the axi_xbar: I use a configuration with 4 masters and 2 slaves. The second master writes valid data to the interconnect and the data is confirmed by the slave, but by the wrong slave.

I write to address h'0080_0000 and expect the first slave to be connected but what I see is that data are going to slave 0 and in slave 0 the data are consumed, of course in a wrong way, but the bus transfer is correct.

I post my address map and the instantiation of the xbar, hopefully anybody has an idea:

    localparam int unsigned NoMasters   = 4; // How many Axi Masters there are
    localparam int unsigned NoSlaves    = 2; // How many Axi Slaves  there are

    // axi configuration
    localparam int unsigned AxiIdWidthMasters =  4;
    localparam int unsigned AxiIdUsed         =  4; // Has to be <= AxiIdWidthMasters
    localparam int unsigned AxiIdWidthSlaves  =  AxiIdWidthMasters + $clog2(NoMasters);
    localparam int unsigned AxiAddrWidth      =  32; // Axi Address Width
    localparam int unsigned AxiDataWidth      =  32; // Axi Data Width
    localparam int unsigned AxiStrbWidth      =  AxiDataWidth / 8;
    localparam int unsigned AxiUserWidth      =  5;
    // in the bench can change this variables which are set here freely
    localparam axi_pkg::xbar_cfg_t xbar_cfg = '{
    NoSlvPorts:         NoMasters,
    NoMstPorts:         NoSlaves,
    MaxMstTrans:        10,
    MaxSlvTrans:        6,
    FallThrough:        1'b0,
    LatencyMode:        axi_pkg::CUT_ALL_AX,
    AxiIdWidthSlvPorts: AxiIdWidthMasters,
    AxiIdUsedSlvPorts:  AxiIdUsed,
    AxiAddrWidth:       AxiAddrWidth,
    AxiDataWidth:       AxiDataWidth,
    // number of rules = number of slaves (one rule for each slave)
    NoAddrRules:        NoSlaves
    };

    // start_addr <= addr < end_addr => idx is the index of the slave to which the data goes
    localparam rule_t [xbar_cfg.NoAddrRules-1:0] AddrMap = '{
    '{idx: 32'd1, start_addr: 32'h0080_0000, end_addr: 32'h0080_1000},
    '{idx: 32'd0, start_addr: 32'h0000_0000, end_addr: 32'h0080_0000}
    };

The instantiation of the xbar is the following:

    // -------------------------------------------------------------------
    // -- AXI Intercon                                                  --
    // -------------------------------------------------------------------

    axi_xbar #(
        .Cfg          ( xbar_cfg ),
        .slv_aw_chan_t( aw_chan_mst_t ),
        .mst_aw_chan_t( aw_chan_slv_t ),
        .w_chan_t     (  w_chan_t     ),
        .slv_b_chan_t (  b_chan_mst_t ),
        .mst_b_chan_t (  b_chan_slv_t ),
        .slv_ar_chan_t( ar_chan_mst_t ),
        .mst_ar_chan_t( ar_chan_slv_t ),
        .slv_r_chan_t (  r_chan_mst_t ),
        .mst_r_chan_t (  r_chan_slv_t ),
        .slv_req_t    ( mst_req_t     ),
        .slv_resp_t   ( mst_resp_t    ),
        .mst_req_t    ( slv_req_t     ),
        .mst_resp_t   ( slv_resp_t    ),
        .rule_t       (rule_t         )
    ) i_xbar (
        .clk_i      ( sys_clk  ),
        .rst_ni     ( rst_n    ),
        .test_i     ( 1'b0     ),
        .slv_ports_req_i  ( masters_req  ),
        .slv_ports_resp_o ( masters_resp ),
        .mst_ports_req_o  ( slaves_req   ),
        .mst_ports_resp_i ( slaves_resp  ),
        .addr_map_i       ( AddrMap      ),
        .en_default_mst_port_i ( '0      ),
        .default_mst_port_i    ( '0      )
    );

In submodule i_xbar/gen_slv_port_demux[1]/i_axi_aw_decode/addr_map_i I can see that slave_aw_select is changing to one so I would expect that the correct master port of the intercon (corresponds to slave port in my design) is selected.

Thanks for helping! Kind regards Sebastian

[andreaskurth]

Hello @SebastianZa,

Does the transaction to 0x80_0000 go to mst_ports_req_o[1] or [0]?

• If [1], that is correct and I suggest you check whether slaves_req is defined as "downto" ([1:0]) and whether both slave modules are connected at the correct index of that signal.
• If [0], that is wrong and I suggest you change AddrMap from a localparam to a signal with an invariant assign statement. addr_map_i is a port and not a parameter of axi_xbar, so if you assign a localparam there that could cause problems.

[SebastianZa]

Hi Andreas (@accuminium )

thanks for your fast response. I had a look at the signal struct mst_ports_req_o, coming out of the xbar. Unfortunately the data comes out at Index 0 and not at index 1, what we both expected. I changed the localparam to a variable but with no effect. I don't know what's going wrong here.

I will have a further look and debug the whole thing. In addition I will try to give a detailed report with more source code and wave screenshots so it gets easier to debug the problem.

Kind regards Sebastian

[SebastianZanker]

On the bottom I attached a screenshot of the relevant signals. The address map can be seen and AddrMap[1] contains the rule which I expect to be relevant for the address. On the right you can see the mst_ports_req_o, I expanded index [0]. On the right side of the image address 0x0080_0008 is transmitted over index [0] and I need this data to be transmitted over mst_ports_req_o[1].

According to the address map it should be transmitted over the first master output, shouldn't it? I don't get the error.

Kind regards Sebastian

[SebastianZa]

What I will do next is to set all currently unused signals to zero so that they are not 'x' anymore. Might be that it is a simulation issue?

[WRoenninger]

Hello Sebastian,

Could you also provide a waveform with the request and response structs of the salve ports? A complete waveform of the address decoder on the slave port issuing the transaction into the crossbar would also be useful. (Module: axi_xbar/gen_slv_port_demux[<slv_port_idx>]/i_axi_aw_decode). There could be an issue with the handshaking signals as the B ready signal is X in the waveform . Make sure that after the reset, all handshaking signals of all ports going into the crossbar are well defined 1 or 0.

[SebastianZa]

Hi Wolfgang (@WRoenninger),

it seems that we had the same idea. I will set all signals to a well-defined value. Thanks for help!

Kind regards Sebastian

[SebastianZanker]

Hi Andreas and Wolfgang (@accuminium , @WRoenninger )

this morning I initialized all signals, i.e. that all signals in all Master Interfaces are initialized now from the beginning. Unfortunately with no effect. I attached a screenshot of decode[1] because the Master IF writing to the Intercon is Master[1], i.e. Slave 1 seen from the Intercon.

I expanded all relevant signals as the address map. I think that this seems beeing correct as the first rule is chosen as matching rool. Nontheless data is transmitted over Intercon-Master-IF[0].

Kind regards Sebastian

[SebastianZa]

Hi @WRoenninger

with the help of a colleague I was able to find the error. It is in the spill register: i_xbar_dut/gen_slv_port_demux[1]/i_axi_demux/gen_demux/i_aw_spill_reg

If I set the Bypass to 1 (in instantiation of the spill register in module _xbar_dut/gen_slv_port_demux[1]/i_axi_demux/gen_demux) , then it is working.

If we leave the code as is -- Bypasse ( ~SpillAw) -- then we see that this signal is inverted: i_xbar_dut/gen_slv_port_demux[1]/i_axi_demux/gen_demux/i_aw_spill_reg/data_o.aw_select as can be seen in the screenshot.

I'd like to understand the problem in detail and I hope it is easier to understand now what's going on. :)

When the Spill-Register is enabled I see waveforms which I wouldn't expect at all. a_data_q changes at the same time as data_i. With the register enabled I would expect a delay of one clock cycle. Even more curious is that the variable aw_chan in this struct is assigned while aw_select is not assigned.

Kind regards Sebastian

[SebastianZa]

I wrote a little testbench for the spill register with an VHDL generated clock, because I use the Pulp AXI in a mixed VHDL/SV environment with a clock generated by a VHDL source.

In this small simulation I see that the spill register is working as expected with the VHDL generated clock. So I cannot explain why in the overall testbench (with all AXI master and slave modules and the Pulp AXI) with all connected modules it is not working.

I would be glad if you had some ideas what could yo wrong here. I think it is not an error in your sources but might be a simulation mismatch.

It is working when I do the following:

    localparam axi_pkg::xbar_cfg_t xbar_cfg = '{
    NoSlvPorts:         NoMasters,
    NoMstPorts:         NoSlaves,
    MaxMstTrans:        10,
    MaxSlvTrans:        6,
    FallThrough:        1'b0,
    LatencyMode:        axi_pkg::NO_LATENCY,         <<== disable all registers
    AxiIdWidthSlvPorts: AxiIdWidthMasters,
    AxiIdUsedSlvPorts:  AxiIdUsed,
    AxiAddrWidth:       AxiAddrWidth,
    AxiDataWidth:       AxiDataWidth,
    // number of rules = number of slaves (one rule for each slave)
    NoAddrRules:        NoSlaves
    };

This is not surprising because the spill registers are then all bypassed.

Kind regards and thanks for helping! Sebastian

[WRoenninger]

Hi @SebastianZa, How are you generating the stimuli, as you said, the spill register should always have exactly one cycle delay and acts like a FIFO with two stages. In the screenshots it seems that the application of the data happens exactly on the @(posedge clk). There could be some sort of race condition happening with the different always_ff blocks. Can you try to delay the application of payload data on the AW channel and application of the valid so that it does not happen on the clock edge?

[SebastianZa]

Hi Wolfgang ( @WRoenninger ),

thanks for your reply and help! I changed my sources in a way that SystemVerilog generates the clock now and this clock is fed into Pulp and our VHDL sources and not the other way round (VHDL generated clock => SV) and then it works as expected, so I will close this issues.

I try to get help from a MentorGraphics FAE how to do proper clock synchronization between VHDL and SystemVerilog in a way that we can use our VHDL sources as is in simulation and the clock is correctly transferred into the SV-domain.

If you have some experience with that kind of clock synchronization in simulation don't hesitate to write me. :)

Kind regards Sebastian