Closed mkruliv closed 3 years ago
Hello @mkruliv
Thanks for reporting! We have not done any tests on dual-stack ipv4/ipv6 Kubernetes, but I would like to definitely try.
Hi @rodrigovalin! Thanks for quick response
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.0", GitCommit:"9e991415386e4cf155a24b1da15becaa390438d8", GitTreeState:"clean", BuildDate:"2020-03-25T14:50:46Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}
---
apiVersion: mongodbcommunity.mongodb.com/v1
kind: MongoDBCommunity
metadata:
name: tst-mongo-service
namespace: default
spec:
members: 3
type: ReplicaSet
version: "4.4.8"
security:
authentication:
modes: ["SCRAM-SHA-1"]
users:
- name: useradmin
db: admin
passwordSecretRef:
name: tst-useradmin-password
roles:
- name: userAdmin
db: admin
scramCredentialsSecretName: tst-mongo-service
additionalMongodConfig:
net.ipv6: true
net.bindIpAll: true
storage.wiredTiger.engineConfig.cacheSizeGB: 1.2
If we don't set ipv6 parameter cluster starts successfully
Hello @mkruliv
I managed to reproduce your configuration using kind with the following configuration:
# kind_config.yaml
---
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
ipFamily: dual
And starting a cluster with:
kind create cluster --config kind_config.yaml
I've tested your configuration and I can confirm that the Operator does not work on dual-stack Kubernetes with ipv6 parameter. I've been investigating this a bit and the agent won't get to running state as you mention.
The problem seems to be the type of Service
the operator creates:
$ kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
tst-mongo-service-svc ClusterIP None <none> 27017/TCP 1s
I managed to make the MongoDB resource to get to a running state by making the following changes on the Service (kubectl edit service/tst-mongo-service-svc
). You have to add an entry to ipFamilies
and change ipFamilyPolicy
to PreferDualStack
:
apiVersion: v1
kind: Service
metadata:
name: tst-mongo-service-svc
namespace: mongodb
spec:
clusterIP: None
clusterIPs:
- None
ipFamilies:
- IPv6 # <-- add this one here
- IPv4
ipFamilyPolicy: PreferDualStack # <-- and change this one here
ports:
- name: mongodb
port: 27017
protocol: TCP
targetPort: 27017
publishNotReadyAddresses: true
selector:
app: tst-mongo-service-svc
sessionAffinity: None
type: ClusterIP
After a few minutes the MongoDB resource will reach "Running" state.
I will discuss with the team about supporting this particular IPv6 scenario, or maybe only document the process (manual configuration) for running the Operator in dual stack Kubernetes clusters.
Hey @mkruliv
I have edited this comment to only include relevant information
Unfortunatelly, the Operator, if running with net.ipv6: true
, will always bind to ::
no matter what you have in bindIp
.
So the following configuration:
net.ipv6: true
net.bindIpAll: true
and
net.ipv6: true
net.bindIp: "::,0.0.0.0"
Will result in mongod
binding to ::
only. This is the behaviour of our automation agent, and nothing we can do about for now.
This means that if ipv6
is enabled, mongod
will only bind to ipv6
addresses.
I can't find the MongoDB automation agent source code. Is it free or proprietary?
The agent's code is not open source unfortunately.
I understand you might have a very good reason to enable ipv6 on a SingleStack Kube cluster, but for now, and to make your resource to work, it has to be disabled.
@rodrigovalin thanks for the explanation
I'm going to close the ticket as I'm able to fix that issue by editing the mongo Kubernetes operator Go code (related to container run command) and building a new operator docker image and it works as expected (I need a few tests to confirm it). Not sure that I need to create PR with my changes here because it looks like a workaround. If someone needs the same please ping me. Thanks.
I think I'm interested in seeing your solution; if you don't mind, create that PR and we can discuss about that being a solution for this particular use case and how we can include it in the future.
We need to create MongoDB clusters for Kubernetes that will work on IPv4 and IPv6 platforms, but when we set
the cluster doesn't start on IPv4 platform, for IPv6 only platform it works well Is it possible to have one config for both platforms?
What did you expect? We expect that the Kubernetes operator with that parameters should work on both ipv4 only and ipv6 only Kubernetes cluster
What happened instead? MongoDB Kubernetes operator with IP6 enabled doesn't start on IPv4 k8s cluster
Operator Information
Kubernetes Cluster Information
Image Registry location (quay, or an internal registry): internal
mongod container logs:
mongo agent container log: