About the doubt of the "rmbuf" and "wmbuf" of "struct dpdk_private_context"

[ghost]

Hi,

I'm looking at the piece of code about "dpdk_recv_pkts()" and "dpdk_send_pkts()", I noticed that "rmbuf" pointed to the data buffer received from "rte_eth_rx_burst", like this:

`dpc = (struct dpdk_private_context *) ctxt->io_private_context;

int portid = CONFIG.eths[ifidx].ifindex; ret = rte_eth_rx_burst((uint8_t)portid, ctxt->cpu, dpc->pkts_burst, MAX_PKT_BURST);`

And I also noticed that when sending packets, "dpdk_send_pkts" get mbuf from "wmbuf", and send it by "rte_eth_tx_burst", like this:

`struct dpdk_private_context dpc; dpc = (struct dpdk_private_context )ctxt->io_private_context;

struct rte_mbuf **pkts; int cnt = dpc->wmbufs[ifidx].len; pkts = dpc->wmbufs[ifidx].m_table;

do { / tx cnt # of packets / ret = rte_eth_tx_burst(portid, ctxt->cpu, pkts, cnt); pkts += ret; cnt -= ret; / if not all pkts were sent... then repeat the cycle / } while (cnt > 0);`

This is right, but the point is I don't see the logic for the conversion from "rmbuf" to "wmbuf" when I looked through the entire project code. Can you tell me something about this, it's been confusing to me for a long time.

I hope you can help me with this questions. Thank you in advance.

[eunyoung14]

Hi,

mTCP does not relay the packets from rmbuf to wmbuf directly. It processes the input packets, stores the results to the TCP buffer, and generates output packets depending on the state of the stack. In core.c, you will see that we call ProcessPacket after reading packets through mtcp->iom->get_rptr. For the output, we use EthernetOutput to buffer packets to wmbuf and send them through mtcp->iom->send_pkts in core.c.

[ghost]

Hi,

Thank your reply, I see what you mean.

But I don't see any operations about buffering packets to wmbuf in EthernetOutput as you said, the code of EthernetOutput as follows:

/*----------------------------------------------------------------------------*/
uint8_t *
EthernetOutput(struct mtcp_manager *mtcp, uint16_t h_proto, 
        int nif, unsigned char* dst_haddr, uint16_t iplen)
{
    uint8_t *buf;
    struct ethhdr *ethh;
    int i, eidx;

    /* 
     * -sanity check- 
     * return early if no interface is set (if routing entry does not exist)
     */
    if (nif < 0) {
        TRACE_INFO("No interface set!\n");
        return NULL;
    }

    eidx = CONFIG.nif_to_eidx[nif];
    if (eidx < 0) {
        TRACE_INFO("No interface selected!\n");
        return NULL;
    }

    buf = mtcp->iom->get_wptr(mtcp->ctx, eidx, iplen + ETHERNET_HEADER_LEN);
    if (!buf) {
        //TRACE_DBG("Failed to get available write buffer\n");
        return NULL;
    }
    //memset(buf, 0, ETHERNET_HEADER_LEN + iplen);

#if 0
    TRACE_DBG("dst_hwaddr: %02X:%02X:%02X:%02X:%02X:%02X\n",
                dst_haddr[0], dst_haddr[1], 
                dst_haddr[2], dst_haddr[3], 
                dst_haddr[4], dst_haddr[5]);
#endif

    ethh = (struct ethhdr *)buf;
    for (i = 0; i < ETH_ALEN; i++) {
        ethh->h_source[i] = CONFIG.eths[eidx].haddr[i];
        ethh->h_dest[i] = dst_haddr[i];
    }
    ethh->h_proto = htons(h_proto);

    return (uint8_t *)(ethh + 1);
}

As shown above, it get the wmbuf by mtcp->iom->get_wptr which doesn't have the operation of buffering.

Forgive me for being stupid, hoping you can give me a little more advice.

Best wishes.

[ajamshed]

EthernetOutput() enqueues one packet (at a time) in the wmbufs buffer. If you notice in dpdk_module.c file, the dpc->wmbufs[ifidx].len variable gets incremented by 1 every time get_wptr() is called. In core.c (line 839), mtcp->iom->send_pkts(ctx, tx_inf); flushes out the entire batch of outgoing packets out of the NIC.

So, in summary, you need to backtrack from where EthernetOutput() is being called (ip_out.c --> tcp_out.c and so on...).

[ghost]

Oh, I see it, thank you very much again!