Include load balancing across core switches

[TribuneX]

Implement a load balancing scheme to load balance traffic from one edge switch to another across multiple core switches.

Decide on the granularity of the load balancing

• e.g. per flow

Decide where to include the flow rules to store the actions for load balancing

• extra table
• include in iSDX outbound table

[mcanini]

@TribuneX please come up with a proposal on how we can do this. We shall discuss this next week.

[ederlf]

It seems it started and there is a branch on iSDX with this work ongoing. https://github.com/h2020-endeavour/iSDX/tree/load_balancing

@TribuneX I have a suggestion for the metadata value to match https://github.com/h2020-endeavour/iSDX/blob/load_balancing/flanc/lib.py#L378

Can you write the core datapath_id as the metadata? Both are 64 bits identifiers, so writing should not be a problem. I ask this because i will need to map the edge port to the core port. With the dpid it is easier to retrieve that mapping from the configuration. Also, it is easier to create the match, because the config file maps the cores to dpids and not another type of index.

[ederlf]

Ah, there is one side effect of calculating the umbrella encoding based on the datapath_id (metadata)

We will need one flow per link connected to the core. e.g, if we needed 2 encoding flow per participant (ARP and normal traffic), it will need 8 flows in a topology with 4 cores switches.

[TribuneX]

@ederlf Can we make the flows which encode the paths somehow participant independent? It should only concern umbrella where a packet must be forwarded, not from which participant? I fear that this might be in conflict with policy handling, but do you see a way to achieve this?

[ederlf]

@TribuneX The encoding is only about where the packet should be forwarded, but it is based on the participant. So you need to match them in order to encode its path. Right now I do not see a way to do that.

Actually, thinking about the scenario we discussed this week (cores might have different port numbers leading to an edge), umbrella will need different flows for the encoding of different cores, anyway.

My suggestion is to keep it like that, let's live with this flow increase, evaluate the impact and the we think about optimizations. (Always good to remember that early optmization is the root of all evil :-) ).

For now I would just use the dp_id in the metadata field.

[TribuneX]

My suggestion is to keep it like that, let's live with this flow increase, evaluate the impact and the we think about optimizations. (Always good to remember that early optmization is the root of all evil :-) ).

Agree! Let's see how it goes.

For now I would just use the dp_id in the metadata field.

Done! See: https://github.com/h2020-endeavour/iSDX/blob/load_balancing/flanc/lib.py#L395

As a next step I will evaluate which kind of matching works for balancing the load across the links.

[ederlf]

@TribuneX Cool! It looks promising :-)

I have another suggestion. I would put the load balancing methods in another file, and import it to the lib.py. Isolating the logic on its own module makes it easier to navigate, debug, review, extend later, etc. Also, it is better in case we want to turn it into a configuraable option.

For dependencies, e.g, datapaths, tables, or any other configuration needed, you can pass them as arguments to the load balancer methods.

[TribuneX]

@marchiesa Can you include some pointers to the code where we will integrate iSDX to make use of the load balancing code.

[marchiesa]

@TribuneX @ederlf Sure.

As far as I understand, we need to first modify 'gss.py' and replace all the forwarding actions through a physical port with GoToTable actions. The output of the iSDX is a packet that enters the Umbrella tables with its egress port identifier written in the VMAC. The Umbrella tables are then responsible for load-balancing the traffic within the IXP network. Therefore, I think that the load-balancer should configure the Umbrella tables in such a way that the mapping egress-port -> path is the optimal one in terms of link utilization. This is outside of the iSDX code.

In a first phase, I would assume that all packets are iSDX packets and then consider also legacy packets, which should not be processed by the iSDX pipeline.

Let me know if I can be of further help or something is not clear.

[ederlf]

@TribuneX @marchiesa I am writing a GSS module for the multi-hop topology already considering legacy packets. (e.g BGP and ARP). I expect to have something by the end of the week.

[ederlf]

@TribuneX , will we send control traffic to the load balancing table? Do you think they should be distributed among the cores too?

[marchiesa]

@ederlf thanks Eder! As for the control traffic, in my opinion, I think that it is not important to load balance it (but you can do it if it is easier).

[ederlf]

@TribuneX @marchiesa Just pushed the GSS module. Basically tables from the main-out direct to the load balancing table. https://github.com/h2020-endeavour/iSDX/commit/21bbb6d246dca60883a1736c6fd10658e9cd7e25

Now I need to make changes in the file where the flows are generated in the refmon.

[TribuneX]

@ederlf thanks Eder! As for the control traffic, in my opinion, I think that it is not important to load balance it (but you can do it if it is easier).

@marchiesa I would also direct the control traffic to the load balancing table. In my opinion it makes it easier if we threat all traffic the same in this respect. Also keeps enables backup paths in case of a core failure.

[TribuneX]

@ederlf Do you want the load balancing rules to be installed via the refrom.py since you included a type for it here: https://github.com/h2020-endeavour/iSDX/blob/mh-ctrlr/flanc/ofp13.py#L12

[ederlf]

@TribuneX not necessarily. But it is included because I am sending a flow from the xctrl (the ixp controller module that sends flows to the ixp Openflow, refmon.py, controller). In the ofp13.py, more than create the flows, it validates them. So I need to include the load-balancing on the existent types.

If we want to modify the load balance dynamically, the xctrl (or another external app) is right place to have it, as the refmon.py only install default flows. Unless we want to include more responsibilities for the OpenFlow controller. It is a design choice.

[TribuneX]

@TribuneX not necessarily. But it is included because I am sending a flow from the xctrl (the ixp controller module that sends flows to the ixp Openflow, refmon.py, controller). In the ofp13.py, more than create the flows, it validates them. So I need to include the load-balancing on the existent types.

@ederlf But we had initially discussed that the validation is not a must for operator specific rules such as the load balancers. I agree that we need to have it for rules which are set by participants. Its just a choice we need to make.

If we want to modify the load balance dynamically, the xctrl (or another external app) is right place to have it, as the refmon.py only install default flows. Unless we want to include more responsibilities for the OpenFlow controller. It is a design choice.

@ederlf We will look into that, but this would then mean that all load balancing specific flow rules are send via the refmon client to the server right?

[ederlf]

As i said, it is a design choice :-) And validation is not only about allowing a participant to insert a flow. For example, also corrects missing fields ( for example, you add a tcp flow, but not insert the eth_type and the ip_proto fields).

Actually it would be the xctrl client. If you think it is easier to follow the current approach (doing in the refmon), I am fine with that.

[ederlf]

@TribuneX

I was taking a look in the last commit of the branch you are integrating with the mh-ctrl and I have a comment: On this line, we do not need a new field in the message. We use "fwd:". If forward has numbers, the action is output. If there is a name, a table name, it translates into the table name. https://github.com/h2020-endeavour/iSDX/blob/mh-ctrlr_load_balancing/flanc/ofp13.py#L156

That said, here: https://github.com/h2020-endeavour/endeavour/blob/load_balancing/uctrl/umbrella.py#L168

instructions = {"meta": metadata, "goto": 'umbrella-edge'}

Becomes:

instructions = {"meta": metadata, "fwd": ["umbrella-edge]}

[Floknation]

@ederlf

Thank's for this comment. You're right, I think it's a design choice :) We should discuss this on Monday? I think it's a better idea to implement this goto argument, because the seperate exception handling if the table name is not valid.

[ederlf]

@Floknation I like the idea to have an explicit command for a goto instruction. However, the iSDX approach of a single form (fwd ) is because the message abstracts any underlying hardware. For example, the meaning of a string can also means to send the packet to another switch instead of a table. (OpenFlow 1.0 needs this aproach and it is also used for the ARP switch on the multi table version).

I would keep the former semantic for now. Otherwise we might have ending with two different styles for the packet destination in a message (table, port, switch...) or having to make a good amount of changes to have it fully covered by goto. It implies in more time on tests to check if we did not break the old code.

[Floknation]

I am currently working at the branch load_balancing#14 and implement an advanced IP Load-Balancer. The most code updates are in the load_balancer.py. Few updates are in the umbrella.py. First Step, choose the match-bytes and generate all combinations of matches. match_byte2 = [0, 0, 0, "01000000"] match_byte3 = [0, 0, 0, "00100000"] id_matcher2, id_matcher3 = self.lbal.init_multi_match(match_byte2, match_byte3) Second Step, match combinations assigned to the core ids. self.lbal.set_core_multi_match(self.config.cores, [id_matcher2, id_matcher3]) Last Step, create matches for one core with core id and the match fields. match, metadata = self.lbal.get_ip_multi_match(core_id, ['ipv4_src','ipv4_dst'])

This ist the dump for the Example: cookie=0x49, duration=14.145s, table=4, n_packets=0, n_bytes=0, priority=10, ip, nw_src=0.0.0.0/0.0.0.64, nw_dst=0.0.0.0/0.0.0.32 actions=write_metadata:0x30/0xffffffff cookie=0x4a, duration=14.145s, table=4, n_packets=0, n_bytes=0, priority=10, ip, nw_src=0.0.0.64/0.0.0.64, nw_dst=0.0.0.32/0.0.0.32 actions=write_metadata:0x40/0xffffffff cookie=0x48, duration=14.148s, table=4, n_packets=0, n_bytes=0, priority=10, ip, nw_src=0.0.0.0/0.0.0.64, nw_dst=0.0.0.32/0.0.0.32 actions=write_metadata:0x20/0xffffffff cookie=0x47, duration=14.148s, table=4, n_packets=0, n_bytes=0, priority=10 ,ip ,nw_src=0.0.0.64/0.0.0.64, nw_dst=0.0.0.0/0.0.0.32 actions=write_metadata:0x10/0xffffffff

Is this okay for our test scenario? Currently the code is not generic, you can only use the fourth byte for the match and you can only use two match fields.