oxidecomputer / omicron

Omicron: Oxide control plane
Mozilla Public License 2.0
240 stars 36 forks source link

Only allocate SNAT IP when there is no ephemeral or floating IP #4317

Open bnaecker opened 10 months ago

bnaecker commented 10 months ago

In the instance-creation saga, we unconditionally allocate an SNAT IP for every instance. This is regardless of whether the request explicitly asked for ephemeral addresses. In that case, OPTE always uses the ephemeral address anyway, and no network traffic will ever flow through the SNAT IP. We ought to make such allocation optional, and only provide one if no ephemeral addresses were requested.

bnaecker commented 10 months ago

Related to https://github.com/oxidecomputer/omicron/issues/2715. We should ideally also move the SNAT address allocation and release to the instance start saga and (maybe a new?) stop saga.

david-crespo commented 4 months ago

Wonder if the fact that you can now detach and attach ephemeral IPs changes things here. If you start with an ephemeral IP and no SNAT IP, then you detach the ephemeral IP, you have nothing.

david-crespo commented 4 months ago

Based on the discussion in #5043, we should:

None of that sounds too hard. One wrinkle is that we'd be introducing a new way that detaching a floating or ephemeral IP can fail, because if there is no default pool or the default pool is full, then allocating the SNAT IP at detach time will fail and we probably shouldn't allow that. Fortunately the lack of a default pool means it's very hard to do anything, so we should consider that just a Bad Configuration among many possible bad configurations operators must endeavor to avoid. Default pool being full is also generally a bad situation — you can't create new instances either — so it's not that weird if a few other operations are hamstrung by that.

david-crespo commented 2 months ago

We're not getting this into v9.

askfongjojo commented 1 week ago

Would this still be required given the internet gateway feature? (https://github.com/oxidecomputer/omicron/issues/2154)