feat(OpenStack): Network picking from subset/all possible networks

[Tiboris]

With this patch OpenStack is able to pick Network from all of networks based on network load and requirement got from the metadata. Each of required host gets its own position, aka weigh in interval <0, 1> based on the order (index) in metadata. Before that all of the networks are 'normalized' in a way that the random selection from subset of networks (or all of network if number of hosts > number of networks) is divided to effective range (multiple intervals <0, 1>) for the network based on the relative network weight compared to full capacity of considered networks.

An example:

• considering 5 host request
• picked 5 networks where availability is > 5%

Host weights:

• host 1 - 0/5 = 0
• host 2 - 1/5 = 0.2
• host 3 - 2/5 = 0.4
• host 4 - 3/5 = 0.6
• host 5 - 4/5 = 0.8

Network availability:

• net 1 - 20 addresses
• net 2 - 100 addresses
• net 3 - 130 addresses
• net 4 - 145 addresses
• net 5 - 105 addresses

Full capacity of these 5 nets: 500

Normalized network range: net 4 - <0, 0.29> net 3 - (0,29, 0.55> net 5 - (0.55, 0.76> net 2 - (0.76, 0.96> net 1 - (0.96, 1>

Which will divide networks for hosts: host 1 => net 4 (0 falls into net 4 interval) host 2 => net 4 (0.2 falls into net 4 interval) host 3 => net 3 (0.4 falls into net 3 interval) host 4 => net 5 (0.6 falls into net 5 interval) host 5 => net 2 (0.8 falls into net 2 interval)

net 1 will be unused as bigger networks are preferred this way.

[Tiboris]

Adding basic tests and will add more, just creating skeleton

[Tiboris]

I have added

async def test_network_picking_validation(self, x):

which should test new way of picking networks. This MR from my POV can be reviewed now.

[Tiboris]

I'm missing in the patch the part which we discuss in person - keep the original behavior (all hosts has the same network picked) for the usual cases.

Hello @pvoborni I find this part very sad because:

• I have run several regression runs and some runs with big requests and the picking algorithm worked as designed without any issues.
• If there is a problem with network connectivity between multiple network, we can not fix this here and it will be problem of infrastructure.
• If problem like above happens and we are not able to connect to host it will be destroyed and provisioning for failed hosts will start over.
• This feature will have no value if i put an if there and 99% use-cases would use the old way - one network for all hosts from requests.

I would really like to convince you that this is nothing bad from the perspective of picking the networks. So far there are no issues and I would rather to create such patch as a quick fix if there is something wrong with this approach (even if you consider this unsafe development path). Would you give it a try?

I'd also consider if we should not create a new module for openstack network switching

If I understood correctly you mean something like we have with provisioning_base and strategies.

[Tiboris]

I have updated docstrings for the methods which are network_pick related. I have extracted the text from commit message and added few examples.

[Tiboris]

I have added tests for 0 hosts available to test corner-case with no IPs left.

[Tiboris]

regression runs passing, merging