Closed leftwo closed 1 month ago
I think this is actually working as intended, even though it's weird. Lemme justify why, and tell me if you think I'm still wrong.
oxp_a527b2b2-e35c-44e1-9eac-c1aeef3a1c9e
is one such zpool with no corresponding Crucible zone.So, here's the order of events I think is happening:
As a result, that zpool is initialized, but no crucible service is ever provisioned to it.
Right now, only RSS is responsible for the provisioning of Crucible datasets. However, in the future, Nexus should maintain this responsibility, as documented by https://github.com/oxidecomputer/omicron/issues/732 .
So, right now, if it's not part of the original RSS plan, it never gets provisioned. Later, we'd like Nexus to notice these late-addition zpools, and provision Crucible datasets to them (among other services).
So, in summary, this is not a bug, but a not yet finished part of Nexus. As long as that issue is tracked else where, we can probably close this.
Is there anything we could do on the dogfood system to sandbag RSS starting, to give the pools time to show up?
I believe this issue is fixed by the reconfigurator, which provisions crucible zones to all active physical disks in the control plane
On the dogfood system (BRM42220070, gimlet in cubby 26) I've installed Omicron.
I have 10 U.2 SSDs and 2 M.2s.
I can see sled-agent has created 13 pools for me. One for the root pool, two on the M.2s, and 10 on the U.2s:
However, sled-agent itself only has 8 crucible zones:
Two of these are on the M.2 devices, so we really have six crucible zones out of 10 expected. For some reason the additional zones are not created.
I've attached the sled agent log. sled-agent.log