Elektrobit / flake-pilot

Registration/Control utility for applications launched through a runtime-engine, e.g containers
MIT License
9 stars 5 forks source link

Refactor port allocation for UDS socket #154

Closed schaefi closed 1 year ago

schaefi commented 1 year ago

When using the firecracker-pilot in a resume type flake, the pilot creates a UDS socket for listening on data. Each of this socket connections requires a free port. The former implementation selects the socket by a random number within a given range. However, this is not a stable solution and the risk to select an already occupied port is high. This commit refactors the port allocation to a simply calculate a new port from FIRECRACKER_VSOCK_PORT_START + the PID of the pilot process.

Further conversations on this topic brought up some ideas how to select a system wide free port by e.g a kernel module but as a first improvement to the former implementation this commit should serve the purpose.

How to reproduce

flake-ctl firecracker pull --name leap --kis-image https://download.opensuse.org/repositories/home:/marcus.schaefer:/delta_containers/images_15.4/firecracker-basesystem.$(uname -m).tar.xz

flake-ctl firecracker register --vm leap --app $HOME/mybash --target /bin/bash --overlay-size 20GiB --resume

for i in 1 2 3 4 5;do ~/mybash --version;done

This Fixes #114

schaefi commented 1 year ago

I think there might be even a simpler solution... testing it

Ichmed commented 1 year ago

Looks good apart from my comment, definitely more robust than random ports and also way faster then looping through filenames.

schaefi commented 1 year ago

Looks good apart from my comment, definitely more robust than random ports and also way faster then looping through filenames.

Thanks, so if you are fine with it I'll wait for approval prior merge :)

I would also be interested what happens behind the scene, as you mentioned a kernel module for getting a system wide free port through a semaphore I was curious to see if there is some code, poc or something available. You mentioned @m-kat is working on that but it seems there is not much sharing on this activity on the opensource side of things. So if there is anything you can share I would be interested. Thanks

isbm commented 1 year ago

I would be voting against this solution. Current random solution is bad enough to be considered improved, but it still works by far anyhow. There is no need to replace one bad solution with another bad solution, creating more work and ripples around. 😉

The way it would be proper is to allocate systen-wide port from a range, check it of course, and then store its number in either a file somewhere in /run/... or (for RO systems) take it already from a /proc/... which is exposed and allocated already by own module.

I still don't get why not make a first half of it: just allocate a port system-wide, check if it is free and store in a file. Then next time increment it with a next process. Since that file must be stored in a volatile storage (always), the next reboot will remove it.