search

tigera / operator

Kubernetes operator for installing Calico and Calico Enterprise
Apache License 2.0
185 stars 140 forks source link

metrics port hardcoded to 8383 in tigera-operator conflicting with CNV #515

Closed krisiasty closed 4 years ago

krisiasty commented 4 years ago

Expected Behavior

Metrics port in tigera-operator should be configurable, and default value should be different than 8383 which is used by nmstate-handler pods from CNV (Container Native Virtualization in RedHat Openshift 4.x. Instead it is hardcoded to 8383 here: https://github.com/tigera/operator/blob/master/pkg/daemon/daemon.go#L35

Current Behavior

Currently it is not possible to properly deploy CNV (Container Native Virtualization) on RedHat Openshift 4.x with Calico and tigera-operator. Both tigera-operator and one of nmstate-handler pods (part of CNC) run with host network and try to bind to port 8383. Since tigera-operator is deployed first, nmstate-handler keeps crashing and CNV deployment cannot finish properly.

Possible Solution

  1. make metrics port configurable, and
  2. change default port 8383 to something else

Steps to Reproduce (for bugs)

  1. deploy RedHat Openshift 4.3.12 with Calico as specified in RedHat documentation and the following page: https://docs.projectcalico.org/getting-started/openshift/installation
  2. deploy CNV according to RedHat documentation: https://docs.openshift.com/container-platform/4.3/cnv/cnv_install/installing-container-native-virtualization.html
  3. list pods in openshift-cnv namespace and look for crashing nmstate-handler running on the same node as tigera-operator
  4. scale down tigera-operator to 0 replicas to stop tigera-operator pod
  5. restart failed nmstate-handler pod - it should be able to run successfully
  6. inspect status of hco/kubevirt-hyperconverged in openshift-cnv namespace - the last status condition should state "Reconcile completed successfully"

Context

Your Environment

caseydavenport commented 4 years ago

Totally agree this should be configurable in the tigera operator. We should probably also work with RH to make the CNV port configurable as well. Or is it configurable already? Maybe we should make changes to both.

Unfortunately, port conflicts are a fact of life for host-networked things :cry:

krisiasty commented 4 years ago

ARAIK latest upstream version of Kubevirt uses different ports, although last time I checked it was still not configurable. I don't know if/when RH is going to update CNV to also change this port, or make it configurable. I hoped Tigera team is much more flexible and would introduce necessary change much faster than RH.