Closed zarenner closed 1 year ago
/kind support
Turns out manually creating/connecting an additional shared network is NOT an option, due to the container addresses should have 2 values, got 3 values
check: https://github.com/kubernetes/minikube/blob/879c592e02668cd85334fce9b28c6ac6b6fb8780/pkg/drivers/kic/oci/network.go#L227-L244
That seems to leave using the host to proxy traffic between nodes as perhaps the only working method to communicate across clusters?
zarenner I think you might have found a case that we dont have support for it,
by design separate profiles (clusters) are totally separate cluster, do you mind sharing why you would want separate clusters to share the same networking ?
this might be an easy fix if you or anyone wants to contribute to minikube.
the logic of creating networks for docker driver can be found in this package minikube/pkg/drivers/kic/oci/network_create.go https://github.com/medyagh/minikube/blob/879c592e02668cd85334fce9b28c6ac6b6fb8780/pkg/drivers/kic/oci/network_create.go#L17
@medyagh We were implementing a gateway / proxying service that fronts multiple clusters. We wanted our minikube-based dev environment to roughly match production in this manner, hence bridging/sharing their networks in some way.
In the end we decided instead to use separate ingress gateways, namespaces, etc to mimic multiple clusters within a single minikube cluster. Less resource intensive anyways and still sufficient so far for our needs.
As such, at least in the short term I think it's unlikely that I'll get around to adding support for this - feel free to close this issue if desired.
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.
This bot triages issues and PRs according to the following rules:
lifecycle/stale
is appliedlifecycle/stale
was applied, lifecycle/rotten
is appliedlifecycle/rotten
was applied, the issue is closedYou can:
/remove-lifecycle stale
/lifecycle rotten
/close
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.
This bot triages issues and PRs according to the following rules:
lifecycle/stale
is appliedlifecycle/stale
was applied, lifecycle/rotten
is appliedlifecycle/rotten
was applied, the issue is closedYou can:
/remove-lifecycle rotten
/close
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle rotten
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.
This bot triages issues according to the following rules:
lifecycle/stale
is appliedlifecycle/stale
was applied, lifecycle/rotten
is appliedlifecycle/rotten
was applied, the issue is closedYou can:
/reopen
/remove-lifecycle rotten
Please send feedback to sig-contributor-experience at kubernetes/community.
/close not-planned
@k8s-triage-robot: Closing this issue, marking it as "Not Planned".
What Happened?
While
--network
allows selecting a specific docker network, attempting to attach multiple clusters to that network fails.Repro steps:
minikube start -p test1 --driver docker --network sharednetwork
test1
profile is created successfully.docker network list
shows newsharednetwork
network.minikube start -p test2 --driver docker --network sharednetwork -v=1 --alsologtostderr
(see logs below for details)
Looking at https://github.com/kubernetes/minikube/blob/879c592e02668cd85334fce9b28c6ac6b6fb8780/pkg/drivers/kic/kic.go#L98-L107 and https://github.com/kubernetes/minikube/blob/879c592e02668cd85334fce9b28c6ac6b6fb8780/pkg/minikube/driver/driver.go#L371-L384 it seems the logic to determine the IP address is based on the node index and doesn't consider multiple profiles.
It seems like the logic should be profile-aware, or otherwise allow a different IP to be chosen that doesn't match the node index (e.g. exclude IPs already used in the network and take next available). Or perhaps there's a better alternative to communicate across clusters / this scenario isn't supported and a better error message should be emitted.
Workarounds I've found to communicate across clusters:
Use(see comment)docker network create
to create a new network, then e.g.docker network connect --alias test1-cluster <network> <test1_kic_container_id>
on both kic containers to network the nodes/clusters together.host.minikube.internal
to proxy traffic between clusters through the host.Attach the log file
Operating System
Ubuntu
Driver
Docker