search

metal3-io / ironic-image

Container image to run OpenStack Ironic as part of Metal³
Apache License 2.0
57 stars 117 forks source link

Allow overriding/specifying IRONIC_IP/IRONIC_URL_HOST #468

Closed lentzi90 closed 6 months ago

lentzi90 commented 10 months ago

It is currently possible to set either the PROVISIONING_INTERFACE or PROVISIONING_IP. If PROVISIONING_IP is set, this IP is assumed to be associated with some interface and Ironic will wait until it can see it there. If PROVISIONING_INTERFACE is set, we check what IP is associated with it and assume that this is the IRONIC_IP and IRONIC_URL_HOST.

Ref: https://github.com/metal3-io/ironic-image/blob/d62a9b7cab9e854df833f441bace4852e0cb57b9/scripts/ironic-common.sh#L43-L72

I would like to expose Ironic through a Service of type LoadBalancer instead of using host network. When doing this, the load balancer IP will not be directly associated with any interface in the container. This means that I cannot set the PROVISIONING_IP since Ironic would then wait indefinitely to see this IP on some interface (which will never happen). Instead I set the PROVISIONING_INTERFACE. This works great and I can reach both Ironic and Inspector when curling the load balancer IP (e.g. 192.168.222.200).

However, due to the snippet above, Ironic and Inspector will be configured to try to reach each other using the cluster network IP of the Pod (e.g. 10.244.0.13). This IP is volatile and not something that would be in the certificate when using TLS, so communication breaks down.

What can we do about this? Is it something that would go away together with the Inspector anyway? Maybe not worth doing anything at this point then. Or should we make it possible to override the IRONIC_IP and/or IRONIC_URL_HOST?

If this sounds interesting and you would like to play with it, try this:

  1. Clone https://github.com/lentzi90/playground/tree/ironic-loadbalancer#metal3 (use the branch ironic-loadbalancer)
  2. Run ./Metal3/dev-setup.sh
  3. Wait for all pods to be up
  4. Curl the APIs:
    1. curl https://192.168.222.200:5050 -k
    2. curl https://192.168.222.200:6385 -k
  5. Try creating a BMH to see the inspection error: NUM_BMH=1 ./Metal3/create-bmhs.sh
dtantsur commented 10 months ago

Is it something that would go away together with the Inspector anyway?

Yes, but we may hit similar issues with JSON RPC. Which is, whoever, its own ball of wax.

lentzi90 commented 10 months ago

Is it something that would go away together with the Inspector anyway?

Yes, but we may hit similar issues with JSON RPC. Which is, whoever, its own ball of wax.

Could you elaborate? I'm afraid I'm lost :sweat_smile:

dtantsur commented 9 months ago

For JSON RPC to work, Ironic instances have to be able to talk to each other. That requires every Ironic to know its own IP or host name.

Rozzii commented 9 months ago

/triage accepted /help

metal3-io-bot commented 9 months ago

@Rozzii: This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed by commenting with the /remove-help command.

In response to [this](https://github.com/metal3-io/ironic-image/issues/468): >/triage accepted >/help Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
lentzi90 commented 9 months ago

Based on the discussion in the last community meeting, I'm adding some notes here so I don't forget.

metal3-io-bot commented 6 months ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues will close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle stale

lentzi90 commented 6 months ago

I think we can close this as fixed in https://github.com/metal3-io/ironic-image/pull/476