search

multipath-tcp / mptcp

⚠️⚠️⚠️ Deprecated 🚫 Out-of-tree Linux Kernel implementation of MultiPath TCP. 👉 Use https://github.com/multipath-tcp/mptcp_net-next repo instead ⚠️⚠️⚠️
https://github.com/multipath-tcp/mptcp_net-next
Other
888 stars 335 forks source link

Experimental Scheduler based on TCP Header Fields #400

Open merterdemir opened 4 years ago

merterdemir commented 4 years ago

Hi all,

I would like to write an experimental scheduler that decides on which subflow to sent the current segment according to “some”, currently undecided, fields on transport layer header (it can be TCP Options) by parsing the TCP Header in the scheduler. To do so, I planned to parse TCP Header with tcp_hdr(skb) in the default scheduler. Then, I should be able to read those “some” fields in the TCP header and decide on one of the paths in the scheduler. Finally, choosing the path should be trivial by iterating over mpcb and looking for the correct path_index.

However, I cannot see the correct Source, Destination addresses and ports. Both IPv4 and TCP headers doesn’t seem correct on printk output even though endiannes is changed from network to host. What could be the main problem with that? Here is the debugging code I have in get_available_subflow function in default scheduler, (right after the first variable declarations). I will also append a part of the kernel log for a full load of http://multipath-tcp.org webpage at the end of this comment.

Let me know if you need any more information.

My system specs: 

struct sock *get_available_subflow(struct sock *meta_sk, struct sk_buff *skb,
                   bool zero_wnd_test)
{
    struct mptcp_cb *mpcb = tcp_sk(meta_sk)->mpcb;
    struct sock *sk;
    bool looping = false, force;

    /***********************************************************************/
    /*                          CUSTOM CODE BLOCK                          */
    /***********************************************************************/
    struct iphdr *iph;          /* IPv4 header */
    struct tcphdr *tcph;        /* TCP header */
    u16 sport, dport;           /* Source and destination ports */
    u32 saddr, daddr;           /* Source and destination addresses */
    unsigned char *user_data;   /* TCP data begin pointer */
    unsigned char *tail;        /* TCP data end pointer */
    unsigned char *it;          /* TCP data iterator */

    iph  = ip_hdr(skb);         /* get IP header */
    tcph = tcp_hdr(skb);        /* get TCP header */

    /* Convert network endianness to host endiannes */
    saddr = ntohl(iph->saddr);
    daddr = ntohl(iph->daddr);
    sport = ntohs(tcph->source);
    dport = ntohs(tcph->dest);

    /* Calculate pointers for begin and end of TCP packet data */
    user_data = (unsigned char *)((unsigned char *)tcph + (tcph->doff * 4));
    tail = skb_tail_pointer(skb);

    /* ----- Print all needed information from received TCP packet ------ */
    /* Print packet route */
    printk(KERN_DEBUG "---------------------------------------------------------------------");
    printk(KERN_DEBUG "Source -> Destination: %pI4h:%d -> %pI4h:%d\n", &saddr, sport, &daddr, dport);
    printk(KERN_DEBUG "TCP source: %hu, TCP dest: %hu\n", ntohs(tcph->source), ntohs(tcph->dest));
    printk(KERN_DEBUG "TCP seq: %u, TCP ack_seq: %u\n", ntohl(tcph->seq), ntohl(tcph->ack_seq));
    printk(KERN_DEBUG "TCP doff: %hu, TCP window: %hu\n", ntohs(tcph->doff), ntohs(tcph->window));
    printk(KERN_DEBUG "TCP check: 0x%hx, TCP urg_ptr: %hu\n", ntohs(tcph->check), ntohs(tcph->urg_ptr));
    printk(KERN_DEBUG "FLAGS=%c%c%c%c%c%c\n\n",
            tcph->urg ? 'U' : '-',
            tcph->ack ? 'A' : '-',
            tcph->psh ? 'P' : '-',
            tcph->rst ? 'R' : '-',
            tcph->syn ? 'S' : '-',
            tcph->fin ? 'F' : '-');
    /* Print TCP packet data (payload) */
    printk(KERN_DEBUG "TCP Data: \n");
    for (it = user_data; it != tail; ++it) {
        char c = *(char *)it;
        if (c == '\0')
            break;
        printk(KERN_DEBUG "%c", c);
    }
    printk(KERN_DEBUG "\n");
    printk(KERN_DEBUG "---------------------------------------------------------------------");
    /***********************************************************************/

    /* Answer data_fin on same subflow!!! */
    if (meta_sk->sk_shutdown & RCV_SHUTDOWN &&
...
...
}

Here is the kernel log (none of the ports, addresses or field values seems correct to me, except the payload (weird)):

Sep 13 18:32:09 mert-target kernel: [  550.002728] ---------------------------------------------------------------------
Sep 13 18:32:09 mert-target kernel: [  550.002731] Source -> Destination: 1.0.0.0:0 -> 0.0.0.0:2291
Sep 13 18:32:09 mert-target kernel: [  550.002732] TCP source: 0, TCP dest: 2291
Sep 13 18:32:09 mert-target kernel: [  550.002733] TCP seq: 3739127551, TCP ack_seq: 4278190080
Sep 13 18:32:09 mert-target kernel: [  550.002733] TCP doff: 0, TCP window: 0
Sep 13 18:32:09 mert-target kernel: [  550.002734] TCP check: 0x10, TCP urg_ptr: 4332
Sep 13 18:32:09 mert-target kernel: [  550.002734] FLAGS=------
Sep 13 18:32:09 mert-target kernel: [  550.002734] 
Sep 13 18:32:09 mert-target kernel: [  550.002735] TCP Data: 
Sep 13 18:32:09 mert-target kernel: [  550.002735] 
Sep 13 18:32:09 mert-target kernel: [  550.002735] ---------------------------------------------------------------------
Sep 13 18:32:09 mert-target kernel: [  550.003048] ---------------------------------------------------------------------
Sep 13 18:32:09 mert-target kernel: [  550.003050] Source -> Destination: 1.0.0.0:32871 -> 0.0.0.0:59440
Sep 13 18:32:09 mert-target kernel: [  550.003050] TCP source: 32871, TCP dest: 59440
Sep 13 18:32:09 mert-target kernel: [  550.003051] TCP seq: 1392508928, TCP ack_seq: 2154277029
Sep 13 18:32:09 mert-target kernel: [  550.003051] TCP doff: 1280, TCP window: 0
Sep 13 18:32:09 mert-target kernel: [  550.003051] TCP check: 0x8067, TCP urg_ptr: 18651
Sep 13 18:32:09 mert-target kernel: [  550.003052] FLAGS=------
Sep 13 18:32:09 mert-target kernel: [  550.003052] 
Sep 13 18:32:09 mert-target kernel: [  550.003052] TCP Data: 
Sep 13 18:32:09 mert-target kernel: [  550.003052] W
Sep 13 18:32:09 mert-target kernel: [  550.003053] 
Sep 13 18:32:12 mert-target kernel: [  550.003053] ---------------------------------------------------------------------
Sep 13 18:32:12 mert-target kernel: [  553.317886] ---------------------------------------------------------------------
Sep 13 18:32:12 mert-target kernel: [  553.317890] Source -> Destination: 0.0.0.0:0 -> 0.0.0.0:0
Sep 13 18:32:12 mert-target kernel: [  553.317890] TCP source: 0, TCP dest: 0
Sep 13 18:32:12 mert-target kernel: [  553.317891] TCP seq: 0, TCP ack_seq: 10519128
Sep 13 18:32:12 mert-target kernel: [  553.317892] TCP doff: 1792, TCP window: 36607
Sep 13 18:32:12 mert-target kernel: [  553.317892] TCP check: 0xff00, TCP urg_ptr: 0
Sep 13 18:32:12 mert-target kernel: [  553.317893] FLAGS=-APRSF
Sep 13 18:32:12 mert-target kernel: [  553.317893] 
Sep 13 18:32:12 mert-target kernel: [  553.317893] TCP Data: 
Sep 13 18:32:12 mert-target kernel: [  553.317893] 
Sep 13 18:32:12 mert-target kernel: [  553.317893] ---------------------------------------------------------------------
Sep 13 18:32:12 mert-target kernel: [  553.317927] ---------------------------------------------------------------------
Sep 13 18:32:12 mert-target kernel: [  553.317928] Source -> Destination: 227.127.0.0:0 -> 240.58.50.49:0
Sep 13 18:32:12 mert-target kernel: [  553.317928] TCP source: 0, TCP dest: 0
Sep 13 18:32:12 mert-target kernel: [  553.317929] TCP seq: 0, TCP ack_seq: 0
Sep 13 18:32:12 mert-target kernel: [  553.317929] TCP doff: 0, TCP window: 0
Sep 13 18:32:12 mert-target kernel: [  553.317930] TCP check: 0x0, TCP urg_ptr: 0
Sep 13 18:32:12 mert-target kernel: [  553.317930] FLAGS=------
Sep 13 18:32:12 mert-target kernel: [  553.317930] 
Sep 13 18:32:12 mert-target kernel: [  553.317930] TCP Data: 
Sep 13 18:32:12 mert-target kernel: [  553.317931] 
Sep 13 18:32:12 mert-target kernel: [  553.317931] ---------------------------------------------------------------------
Sep 13 18:32:12 mert-target kernel: [  553.317996] ---------------------------------------------------------------------
Sep 13 18:32:12 mert-target kernel: [  553.317998] Source -> Destination: 109.252.255.255:0 -> 64.15.136.65:0
Sep 13 18:32:12 mert-target kernel: [  553.317998] TCP source: 0, TCP dest: 0
Sep 13 18:32:12 mert-target kernel: [  553.317999] TCP seq: 0, TCP ack_seq: 0
Sep 13 18:32:12 mert-target kernel: [  553.318017] TCP doff: 0, TCP window: 0
Sep 13 18:32:12 mert-target kernel: [  553.318048] TCP check: 0x0, TCP urg_ptr: 0
Sep 13 18:32:12 mert-target kernel: [  553.318049] FLAGS=------
Sep 13 18:32:12 mert-target kernel: [  553.318049] 
Sep 13 18:32:12 mert-target kernel: [  553.318049] TCP Data: 
Sep 13 18:32:12 mert-target kernel: [  553.318049] 
Sep 13 18:32:12 mert-target kernel: [  553.318049] ---------------------------------------------------------------------
Sep 13 18:32:12 mert-target kernel: [  553.318113] ---------------------------------------------------------------------
Sep 13 18:32:12 mert-target kernel: [  553.318114] Source -> Destination: 109.252.255.255:1 -> 128.207.40.64:0
Sep 13 18:32:12 mert-target kernel: [  553.318115] TCP source: 1, TCP dest: 0
Sep 13 18:32:12 mert-target kernel: [  553.318115] TCP seq: 0, TCP ack_seq: 13168367
Sep 13 18:32:12 mert-target kernel: [  553.318116] TCP doff: 3328, TCP window: 0
Sep 13 18:32:12 mert-target kernel: [  553.318116] TCP check: 0x68, TCP urg_ptr: 31910
Sep 13 18:32:12 mert-target kernel: [  553.318117] FLAGS=------
Sep 13 18:32:12 mert-target kernel: [  553.318117] 
Sep 13 18:32:12 mert-target kernel: [  553.318117] TCP Data: 
Sep 13 18:32:12 mert-target kernel: [  553.318117] |
Sep 13 18:32:12 mert-target kernel: [  553.318117] \DF
Sep 13 18:32:12 mert-target kernel: [  553.318118] \8E
Sep 13 18:32:12 mert-target kernel: [  553.318118] \FF
Sep 13 18:32:12 mert-target kernel: [  553.318118] \FF
Sep 13 18:32:12 mert-target kernel: [  553.318119] 
Sep 13 18:32:12 mert-target kernel: [  553.318119] ---------------------------------------------------------------------
Sep 13 18:32:12 mert-target kernel: [  553.318135] ---------------------------------------------------------------------
Sep 13 18:32:12 mert-target kernel: [  553.318135] Source -> Destination: 109.252.255.255:0 -> 128.115.156.65:0
Sep 13 18:32:12 mert-target kernel: [  553.318136] TCP source: 0, TCP dest: 0
Sep 13 18:32:12 mert-target kernel: [  553.318136] TCP seq: 0, TCP ack_seq: 0
Sep 13 18:32:12 mert-target kernel: [  553.318137] TCP doff: 0, TCP window: 0
Sep 13 18:32:12 mert-target kernel: [  553.318144] TCP check: 0x0, TCP urg_ptr: 0
Sep 13 18:32:12 mert-target kernel: [  553.318145] FLAGS=------
Sep 13 18:32:12 mert-target kernel: [  553.318145] 
Sep 13 18:32:12 mert-target kernel: [  553.318145] TCP Data: 
Sep 13 18:32:12 mert-target kernel: [  553.318145] 
Sep 13 18:32:52 mert-target kernel: [  553.318145] ---------------------------------------------------------------------
Sep 13 18:32:52 mert-target kernel: [  593.533126] ---------------------------------------------------------------------
Sep 13 18:32:52 mert-target kernel: [  593.533129] Source -> Destination: 32.47.115.121:0 -> 115.32.114.119:0
Sep 13 18:32:52 mert-target kernel: [  593.533130] TCP source: 0, TCP dest: 0
Sep 13 18:32:52 mert-target kernel: [  593.533131] TCP seq: 0, TCP ack_seq: 0
Sep 13 18:32:52 mert-target kernel: [  593.533131] TCP doff: 0, TCP window: 0
Sep 13 18:32:52 mert-target kernel: [  593.533132] TCP check: 0x0, TCP urg_ptr: 0
Sep 13 18:32:52 mert-target kernel: [  593.533132] FLAGS=------
Sep 13 18:32:52 mert-target kernel: [  593.533132] 
Sep 13 18:32:52 mert-target kernel: [  593.533133] TCP Data: 
…
… (TRIMMED)
…
Sep 13 18:32:54 mert-target kernel: [  595.658521] s
Sep 13 18:32:54 mert-target kernel: [  595.658522] y
Sep 13 18:32:54 mert-target kernel: [  595.658522] m
Sep 13 18:32:54 mert-target kernel: [  595.658522] b
Sep 13 18:32:54 mert-target kernel: [  595.658523] o
Sep 13 18:32:54 mert-target kernel: [  595.658523] l
Sep 13 18:32:54 mert-target kernel: [  595.658523] =
Sep 13 18:32:54 mert-target kernel: [  595.658523] F
Sep 13 18:32:54 mert-target kernel: [  595.658524] a
Sep 13 18:32:54 mert-target kernel: [  595.658524] l
Sep 13 18:32:54 mert-target kernel: [  595.658524] s
Sep 13 18:32:54 mert-target kernel: [  595.658525] e
Sep 13 18:32:54 mert-target kernel: [  595.658525] \BF
Sep 13 18:32:54 mert-target kernel: [  595.658525] 
Sep 13 18:32:58 mert-target kernel: [  595.658526] ---------------------------------------------------------------------
Sep 13 18:32:58 mert-target kernel: [  599.594524] ---------------------------------------------------------------------
Sep 13 18:32:58 mert-target kernel: [  599.594529] Source -> Destination: 1.0.0.0:0 -> 0.0.0.0:0
Sep 13 18:32:58 mert-target kernel: [  599.594530] TCP source: 0, TCP dest: 0
Sep 13 18:32:58 mert-target kernel: [  599.594530] TCP seq: 0, TCP ack_seq: 0
Sep 13 18:32:58 mert-target kernel: [  599.594531] TCP doff: 0, TCP window: 0
Sep 13 18:32:58 mert-target kernel: [  599.594532] TCP check: 0x0, TCP urg_ptr: 0
Sep 13 18:32:58 mert-target kernel: [  599.594532] FLAGS=------
Sep 13 18:32:58 mert-target kernel: [  599.594532] 
Sep 13 18:32:58 mert-target kernel: [  599.594533] TCP Data: 
Sep 13 18:32:58 mert-target kernel: [  599.594533] 
Sep 13 18:32:58 mert-target kernel: [  599.594534] ---------------------------------------------------------------------
Sep 13 18:32:58 mert-target kernel: [  599.629156] ---------------------------------------------------------------------
Sep 13 18:32:58 mert-target kernel: [  599.629162] Source -> Destination: 52.53.51.51:0 -> 93.32.10.67:0
Sep 13 18:32:58 mert-target kernel: [  599.629162] TCP source: 0, TCP dest: 0
Sep 13 18:32:58 mert-target kernel: [  599.629163] TCP seq: 0, TCP ack_seq: 0
Sep 13 18:32:58 mert-target kernel: [  599.629164] TCP doff: 0, TCP window: 0
Sep 13 18:32:58 mert-target kernel: [  599.629164] TCP check: 0x0, TCP urg_ptr: 0
Sep 13 18:32:58 mert-target kernel: [  599.629166] FLAGS=------
Sep 13 18:32:58 mert-target kernel: [  599.629166] 
Sep 13 18:32:58 mert-target kernel: [  599.629166] TCP Data: 
Sep 13 18:32:58 mert-target kernel: [  599.629166] 
Sep 13 18:32:58 mert-target kernel: [  599.629167] ---------------------------------------------------------------------
Sep 13 18:32:58 mert-target kernel: [  599.662625] ---------------------------------------------------------------------
Sep 13 18:32:58 mert-target kernel: [  599.662630] Source -> Destination: 57.49.54.54:0 -> 93.32.10.67:0
Sep 13 18:32:58 mert-target kernel: [  599.662630] TCP source: 0, TCP dest: 0
Sep 13 18:32:58 mert-target kernel: [  599.662631] TCP seq: 0, TCP ack_seq: 0
Sep 13 18:32:58 mert-target kernel: [  599.662632] TCP doff: 0, TCP window: 0
Sep 13 18:32:58 mert-target kernel: [  599.662632] TCP check: 0x0, TCP urg_ptr: 0
Sep 13 18:32:58 mert-target kernel: [  599.662633] FLAGS=------
Sep 13 18:32:58 mert-target kernel: [  599.662633] 
Sep 13 18:32:58 mert-target kernel: [  599.662634] TCP Data: 
Sep 13 18:32:58 mert-target kernel: [  599.662634] 
Sep 13 18:32:59 mert-target kernel: [  599.662635] ---------------------------------------------------------------------
Sep 13 18:32:59 mert-target kernel: [  600.106641] ---------------------------------------------------------------------
Sep 13 18:32:59 mert-target kernel: [  600.106645] Source -> Destination: 164.0.0.0:0 -> 7.1.115.0:0
Sep 13 18:32:59 mert-target kernel: [  600.106646] TCP source: 0, TCP dest: 0
Sep 13 18:32:59 mert-target kernel: [  600.106647] TCP seq: 0, TCP ack_seq: 0
Sep 13 18:32:59 mert-target kernel: [  600.106648] TCP doff: 0, TCP window: 0
Sep 13 18:32:59 mert-target kernel: [  600.106648] TCP check: 0x0, TCP urg_ptr: 0
Sep 13 18:32:59 mert-target kernel: [  600.106649] FLAGS=------
Sep 13 18:32:59 mert-target kernel: [  600.106649] 
Sep 13 18:32:59 mert-target kernel: [  600.106650] TCP Data: 
Sep 13 18:32:59 mert-target kernel: [  600.106650] 
Sep 13 18:32:59 mert-target kernel: [  600.106651] ---------------------------------------------------------------------
Sep 13 18:32:59 mert-target kernel: [  600.158191] ---------------------------------------------------------------------
Sep 13 18:32:59 mert-target kernel: [  600.158196] Source -> Destination: 54.54.53.48:0 -> 93.32.10.67:0
Sep 13 18:32:59 mert-target kernel: [  600.158197] TCP source: 0, TCP dest: 0
Sep 13 18:32:59 mert-target kernel: [  600.158198] TCP seq: 0, TCP ack_seq: 0
Sep 13 18:32:59 mert-target kernel: [  600.158198] TCP doff: 0, TCP window: 0
Sep 13 18:32:59 mert-target kernel: [  600.158199] TCP check: 0x0, TCP urg_ptr: 0
Sep 13 18:32:59 mert-target kernel: [  600.158200] FLAGS=------
Sep 13 18:32:59 mert-target kernel: [  600.158200] 
Sep 13 18:32:59 mert-target kernel: [  600.158200] TCP Data: 
Sep 13 18:32:59 mert-target kernel: [  600.158201] 
Sep 13 18:32:59 mert-target kernel: [  600.158201] ---------------------------------------------------------------------
Sep 13 18:32:59 mert-target kernel: [  600.823863] ---------------------------------------------------------------------
Sep 13 18:32:59 mert-target kernel: [  600.823868] Source -> Destination: 147.0.0.0:37 -> 7.1.115.0:41182
Sep 13 18:32:59 mert-target kernel: [  600.823869] TCP source: 37, TCP dest: 41182
Sep 13 18:32:59 mert-target kernel: [  600.823870] TCP seq: 486539264, TCP ack_seq: 2429151
Sep 13 18:32:59 mert-target kernel: [  600.823870] TCP doff: 256, TCP window: 0
Sep 13 18:32:59 mert-target kernel: [  600.823871] TCP check: 0x25, TCP urg_ptr: 28808
Sep 13 18:32:59 mert-target kernel: [  600.823873] FLAGS=------
Sep 13 18:32:59 mert-target kernel: [  600.823873] 
Sep 13 18:32:59 mert-target kernel: [  600.823873] TCP Data: 
Sep 13 18:32:59 mert-target kernel: [  600.823873] 
Sep 13 18:32:59 mert-target kernel: [  600.823874]
cpaasch commented 4 years ago

Hello,

at this stage in the output path the TCP/IP-headers haven't been written yet, because the scheduler needs to make the decision on which subflow to choose before the TCP/IP headers can be written.

What is your intention of using the TCP-header fields? Is it more like the receiver in the TCP-ACK sends those bits to the sender so that the sender uses these bits to steer the traffic?

merterdemir commented 4 years ago

Assuming that we can generate the traces manually in a scenario that we have oracle access to the sizes of a set of full webpage loads, I would like to send the packets to the predecided/precomputed paths in the scheduler. To do so, I though embedding that information in the packets and reading it in the scheduler makes the implementation in the kernel-side a lot easier. Generating the traces and computing which packets go where can be done in application layer in this experimental setup.

cpaasch commented 4 years ago

But how would the application pass this information down to the kernel? The application can't write things into a TCP/IP-header.

merterdemir commented 4 years ago

Embedding such an information to payload as a part of the data and parse & filter in the scheduler is the only way of transferring it to MPTCP scheduler in such an environmental setup, then? I am just trying to make it easier on the kernel-side. I think there is no other way of precomputing traces for a set of websites in MPTCP kernel. Do you have any suggestions about how to construct this workflow? I just want to say which path to choose to MPTCP scheduler from application layer.