Problem about RDMA Fetch&Add operation

[Eternity-Wang]

Currently I'm able to get the RDMA Write-Only operation to work, but not the RDMA Fetch&Add operation. I have used ibv_modify_qp() on the server to modify the access properties of the queue pairs to support atomic operations. And have used ibv_post_send on the client to construct and send Fetch&Add operations, but only receive NAK messages from the server.Here is some code I wrote, hope that you can help me find some problems.

This is the C++ code of executing Fetch&Add operation in client:

static int perform_rdma_fetch_and_add(struct ibv_qp *qp, struct ibv_mr *mr, uint64_t compare_add_value, uint64_t remote_addr, uint32_t rkey) {

    struct ibv_send_wr send_wr = {};
    struct ibv_send_wr *bad_wr;
    struct ibv_sge atomic_sge = {};

    // (Not use in fetch&add operation) The address of the buffer to read from or write to
    atomic_sge.addr = (uint64_t)mr->addr;   
    // (Atomic operations operate on 8-byte values) The length of the buffer in bytes
    atomic_sge.length = sizeof(uint64_t);
    // (Not use in fetch&add operation) The Local key of the Memory Region that this memory buffer was registered with
    atomic_sge.lkey = mr->lkey;  

    memset(&send_wr, 0, sizeof(send_wr));
    // send_wr.wr_id = 0;   // Optional, can be used for tracking purposes
    send_wr.num_sge = 1;
    send_wr.sg_list = &atomic_sge;
    send_wr.opcode = IBV_WR_ATOMIC_FETCH_AND_ADD;
    send_wr.send_flags = IBV_SEND_SIGNALED; // Optionally, set the signaled flag to receive a completion notification
    send_wr.wr.atomic.remote_addr = remote_addr;
    send_wr.wr.atomic.rkey = rkey;
    send_wr.wr.atomic.compare_add = compare_add_value;

    int ret = ibv_post_send(qp, &send_wr, &bad_wr);
    if (ret) {
        perror("ibv_post_send for RDMA Fetch&Add");
        return -1;
    }

    return 0;
}

This is the capture of the Fetch&Add packet from the client

This is the capture of the NAK packet from the server