Error: Error(s) during Write: * At index 0: INTERNAL, Error adding table entry with table_name

[RogueCHU]

Hey, guys! I tried to learn how to use add_on_miss, the new feature of PNA, and errors occurred when I used p4rt-ctl to add an entry.

root@IPDK:~/examples/l3_pna#p4rt-ctl add-entry br0 MainControlImpl.ipv4_host 
 "hdr.ipv4.dst_addr=1.1.1.1,action=MainControlImpl.send_hit(0)"
Error: Error(s) during Write:
    * At index 0: INTERNAL, 'Error adding table entry with table_name: pipe.MainControlImpl.ipv4_host, table_id: 47903772, table_type: 2048, tdi_table_key { hdr.ipv4.dst_addr { field_id: 1 key_type: 0 field_size: 32 value: 0x01010101 } }, tdi_table_data { action_id: 29066479 port { field_id: 1 data_type: 3 field_size: 32 value: 0x00000000 is_active: 1 } }'

My p4 program l3_pna.p4 is based upon the simple_l3.p4 and l2_pna.p4.
I wanted to figure out whether the L3 switch can update the forward-rules by itself. So I tried add_on_miss capability.

First, I setup two taps/VMs.

ip netns add VM0
ip netns add VM1

ip link set TAP0 netns VM0
ip link set TAP1 netns VM1

ip netns exec VM0 ip link set TAP0 up
ip netns exec VM1 ip link set TAP1 up

ip netns exec VM0 ip addr add 1.1.1.1/24 dev TAP0
ip netns exec VM1 ip addr add 2.2.2.2/24 dev TAP1

ip netns exec VM0 ip neigh add 2.2.2.2 dev TAP0 lladdr "$(ip netns exec VM1 ip -o link show TAP1 | awk -F" " '{print $17}')"
ip netns exec VM1 ip neigh add 1.1.1.1 dev TAP1 lladdr "$(ip netns exec VM0 ip -o link show TAP0 | awk -F" " '{print $17}')"

ip netns exec VM0 ip route add 2.2.2.0/24 via 1.1.1.1 dev TAP0
ip netns exec VM1 ip route add 1.1.1.0/24 via 2.2.2.2 dev TAP1

The network topology as follow.

VM0---port0  IPDK container  port1---VM1

And I wrote the l3_pna.p4.

#include <core.p4>
#include "../pna.p4"

typedef bit<48>  EthernetAddress;
const bit<16> ETHERTYPE_TPID = 0x8100;
const bit<16> ETHERTYPE_IPV4 = 0x0800;
const int IPV4_HOST_SIZE = 65536;

header ethernet_t {
    EthernetAddress dstAddr;
    EthernetAddress srcAddr;
    bit<16>         ethertype;
}

header ipv4_h_t {
    bit<8>       version_ihl;
    bit<8>       diffserv;
    bit<16>      total_len;
    bit<16>      identification;
    bit<16>      flags_frag_offset;
    bit<8>       ttl;
    bit<8>       protocol;
    bit<16>      hdr_checksum;
    bit<32>      src_addr;
    bit<32>      dst_addr;
}

struct metadata_t {
}

struct headers_t {
    ethernet_t ethernet;
    ipv4_h_t ipv4;
}

parser MainParserImpl(
    packet_in pkt,
    out   headers_t  hdr,
    inout metadata_t meta,
    in    pna_main_parser_input_metadata_t istd)
{
    state start {
        transition parse_ethernet;
    }

    state parse_ethernet {
        pkt.extract(hdr.ethernet);
        transition select(hdr.ethernet.ethertype){
            ETHERTYPE_IPV4: parse_ipv4;
            default: accept;
        }
    }

    state parse_ipv4 {
        pkt.extract(hdr.ipv4);
        transition accept;
    }
}

control PreControlImpl(
    in headers_t hdr,
    inout metadata_t meta,
    in pna_pre_input_metadata_t istd,
    inout pna_pre_output_metadata_t ostd
){
    apply{}
}

control MainControlImpl(
    inout headers_t  hdr,
    inout metadata_t meta,
    in    pna_main_input_metadata_t  istd,
    inout pna_main_output_metadata_t ostd)
{

    ExpireTimeProfileId_t new_expire_time_profile_id = (ExpireTimeProfileId_t)3;
    bool do_add_on_miss;

    action send_hit(PortId_t port){
        send_to_port(port);
    }

    action send_miss(){
        PortId_t port = (PortId_t)1;
        add_entry(
            action_name = "send_hit",
            action_params = {port},
            expire_time_profile_id = new_expire_time_profile_id
        );
    }
//add_on_miss test
    table ipv4_host{
        key = {hdr.ipv4.dst_addr : exact;}
        actions = {
            @tableonly send_hit;
            @defaultonly send_miss;
        }
        add_on_miss = true;

        idle_timeout_with_auto_delete = true;

        const default_action = send_miss;
        size = IPV4_HOST_SIZE;
    }

    apply{
        ipv4_host.apply();
    }
}

control MainDeparserImpl(
    packet_out pkt,
    in    headers_t hdr,
    in    metadata_t meta,
    in    pna_main_output_metadata_t ostd)
{
    apply {
        pkt.emit(hdr);
    }
}

PNA_NIC(
    MainParserImpl(),
    PreControlImpl(),
    MainControlImpl(),
    MainDeparserImpl()
    ) main;

The idea of my code is, if dst_ip misses, the table will trigger the send_miss action, send_miss action will add port=1 to send_hit. So, I added an entry from control plane firstly.

//VM1(2.2.2.2)-->VM0(1.1.1.1)
p4rt-ctl add-entry br0 MainControlImpl.ipv4_host  "hdr.ipv4.dst_addr=1.1.1.1,action=MainControlImpl.send_hit(0)"

Unfortunately, the error occured.

Theoretically, if the program works, switch will add an VM0-->VM1 rule to the send_hit, and then VM0 ping VM1 successfuly.

So, my question is :

1. There must be some problems in my code, such as incorrect use of add_on_miss.
2. Mybe we can't add entry to the hit action from control plane so far

Any ideas? Thanks a lot!

[RogueCHU]

update: Emmm it seems that switch adds entry from data plane successfully. I use tcpdump catch VM0-->VM1 packets. ICMP request and reply communication has occurred in VM1.

root@IPDK:~/examples/l3_pna#ip netns exec VM0 ping 2.2.2.2 -c 5
root@IPDK:~/examples/l3_pna#ip netns exec VM1 tcpdump -i TAP1 -nne 
07:12:55.859587 72:e6:6a:53:a7:99 > 5e:d3:da:ec:5f:5d, ethertype IPv4 (0x0800), length 98: 1.1.1.1 > 2.2.2.2: ICMP echo request, id 3211, seq 1, length 64
07:12:55.859621 5e:d3:da:ec:5f:5d > 72:e6:6a:53:a7:99, ethertype IPv4 (0x0800), length 98: 2.2.2.2 > 1.1.1.1: ICMP echo reply, id 3211, seq 1, length 64
07:12:55.859681 5e:d3:da:ec:5f:5d > 72:e6:6a:53:a7:99, ethertype IPv4 (0x0800), length 98: 2.2.2.2 > 1.1.1.1: ICMP echo reply, id 3211, seq 1, length 64
07:12:56.861508 72:e6:6a:53:a7:99 > 5e:d3:da:ec:5f:5d, ethertype IPv4 (0x0800), length 98: 1.1.1.1 > 2.2.2.2: ICMP echo request, id 3211, seq 2, length 64
07:12:56.861533 5e:d3:da:ec:5f:5d > 72:e6:6a:53:a7:99, ethertype IPv4 (0x0800), length 98: 2.2.2.2 > 1.1.1.1: ICMP echo reply, id 3211, seq 2, length 64
07:12:56.861572 5e:d3:da:ec:5f:5d > 72:e6:6a:53:a7:99, ethertype IPv4 (0x0800), length 98: 2.2.2.2 > 1.1.1.1: ICMP echo reply, id 3211, seq 2, length 64
07:12:57.889230 72:e6:6a:53:a7:99 > 5e:d3:da:ec:5f:5d, ethertype IPv4 (0x0800), length 98: 1.1.1.1 > 2.2.2.2: ICMP echo request, id 3211, seq 3, length 64
07:12:57.889262 5e:d3:da:ec:5f:5d > 72:e6:6a:53:a7:99, ethertype IPv4 (0x0800), length 98: 2.2.2.2 > 1.1.1.1: ICMP echo reply, id 3211, seq 3, length 64
07:12:57.889301 5e:d3:da:ec:5f:5d > 72:e6:6a:53:a7:99, ethertype IPv4 (0x0800), length 98: 2.2.2.2 > 1.1.1.1: ICMP echo reply, id 3211, seq 3, length 64
07:12:58.909512 72:e6:6a:53:a7:99 > 5e:d3:da:ec:5f:5d, ethertype IPv4 (0x0800), length 98: 1.1.1.1 > 2.2.2.2: ICMP echo request, id 3211, seq 4, length 64
07:12:58.909538 5e:d3:da:ec:5f:5d > 72:e6:6a:53:a7:99, ethertype IPv4 (0x0800), length 98: 2.2.2.2 > 1.1.1.1: ICMP echo reply, id 3211, seq 4, length 64
07:12:58.909596 5e:d3:da:ec:5f:5d > 72:e6:6a:53:a7:99, ethertype IPv4 (0x0800), length 98: 2.2.2.2 > 1.1.1.1: ICMP echo reply, id 3211, seq 4, length 64
07:12:59.933219 72:e6:6a:53:a7:99 > 5e:d3:da:ec:5f:5d, ethertype IPv4 (0x0800), length 98: 1.1.1.1 > 2.2.2.2: ICMP echo request, id 3211, seq 5, length 64
07:12:59.933248 5e:d3:da:ec:5f:5d > 72:e6:6a:53:a7:99, ethertype IPv4 (0x0800), length 98: 2.2.2.2 > 1.1.1.1: ICMP echo reply, id 3211, seq 5, length 64
07:12:59.933289 5e:d3:da:ec:5f:5d > 72:e6:6a:53:a7:99, ethertype IPv4 (0x0800), length 98: 2.2.2.2 > 1.1.1.1: ICMP echo reply, id 3211, seq 5, length 64

However, I can't add VM1-->VM0 entry from control plane. So... VM0 can not receive ICMP reply.

root@IPDK:~/examples/l3_pna#ip netns exec VM0 ping 2.2.2.2 -c 5
root@IPDK:~/examples/l3_pna#ip netns exec VM0 tcpdump -i TAP0 -nne 
07:11:49.227862 72:e6:6a:53:a7:99 > 5e:d3:da:ec:5f:5d, ethertype IPv4 (0x0800), length 98: 1.1.1.1 > 2.2.2.2: ICMP echo request, id 3208, seq 1, length 64
07:11:50.237212 72:e6:6a:53:a7:99 > 5e:d3:da:ec:5f:5d, ethertype IPv4 (0x0800), length 98: 1.1.1.1 > 2.2.2.2: ICMP echo request, id 3208, seq 2, length 64
07:11:51.261527 72:e6:6a:53:a7:99 > 5e:d3:da:ec:5f:5d, ethertype IPv4 (0x0800), length 98: 1.1.1.1 > 2.2.2.2: ICMP echo request, id 3208, seq 3, length 64
07:11:52.285189 72:e6:6a:53:a7:99 > 5e:d3:da:ec:5f:5d, ethertype IPv4 (0x0800), length 98: 1.1.1.1 > 2.2.2.2: ICMP echo request, id 3208, seq 4, length 64
07:11:53.309438 72:e6:6a:53:a7:99 > 5e:d3:da:ec:5f:5d, ethertype IPv4 (0x0800), length 98: 1.1.1.1 > 2.2.2.2: ICMP echo request, id 3208, seq 5, length 64

[jafingerhut]

I believe the DPDK software switch does not currently support adding entries to add-on-miss tables from the control plane, only via add_entry calls in the P4 program.

A potential workaround for now is to modify the P4 program so it does a lookup in two similar tables, one add-on-miss, the other one not add-on-miss. Make control plane additions to the not-add-on-miss table. There are multiple ways to combine the results from the two lookups, but one straightforward way would be to give a hit in one strict priority over the other.

[RogueCHU]

I believe the DPDK software switch does not currently support adding entries to add-on-miss tables from the control plane, only via add_entry calls in the P4 program.

A potential workaround for now is to modify the P4 program so it does a lookup in two similar tables, one add-on-miss, the other one not add-on-miss. Make control plane additions to the not-add-on-miss table. There are multiple ways to combine the results from the two lookups, but one straightforward way would be to give a hit in one strict priority over the other.

Thanks for your reply! Do you think this feature will be supported in the future?

[jafingerhut]

I would like it if it were, but I have no idea if/when anyone with the time and knowledge to do so will make such an enhancement.

[RogueCHU]

I would like it if it were, but I have no idea if/when anyone with the time and knowledge to do so will make such an enhancement. Thanks.