coutinhop
commented
1 year ago Hi @mkaring! Could you share the packet captures you took with the MAC address discrepancies? (Also some details as to the IP and MAC addresses of the interfaces so that we can interpret the captures)
Could you also enable debug logging by setting logSeverityScreen to debug in the default FelixConfiguration? Then, for non-HPC install, you'd need remote access (e.g. RDP) to the Windows machine to get calico logs from there (should be in c:/CalicoWindows/logs. kubelet other logs in c:/k could also be useful.
Finally, is your cluster bare metal, or are you using a cloud service (AWS, GCP, etc)? If you are, could you tell us which one and any details of the setup?
Thanks!
mkaring
pramodsharma62
isarns
Expected Behavior
Following the guides of Calico to setup the Kubernetes node on Windows Server 2022 should result in a working network that is able to communicate with the rest of the cluster and the outside world just fine.
The guide followed can be found here: Install Calico for Windows
Current Behavior
The problem is that the communication works just fine outbound. I'm testing the command
nslookup google.comon a Windows Server 2022 container running on the node, setup by the Calico scripts. The CoreDNS on the control plane is receiving the request and responding to it, but the response never reaches the container that requested the DNS resolution.I'm using VXLAN as overlay for the connection. Reason for me choosing this is because the documentation of BGP sounds scatchy and I can't have the outside network know about my pods running in any way.
To be receive, no communication of any kind seems to be reaching the pod:
The general details about my nodes:
The IP Pool is setup correctly to my understanding:
The logs don't show any problems to me, but I'm running out of ideas what to look for. So if any log from any component would be helpful tracking the issue: Tell me, I get it.
Now what I did find out as potential issue is the following: I did capture the packages on the ethernet link on the Windows Server (not the Hyper-V connection, the physical one). This showed the packages of the DNS request encapsulated by VXLAN. The outbound packages appear fine. The actual package has the proper IP and MAC addresses for the physical servers hosting the kubernetes nodes. The headers inside the VXLAN encapsulation contain the IP and MAC of the containers communicating as expected. How ever the response package with the DNS resolution (it reaches the physical ethernet connection) it addressed inside the VXLAN encapsulation to the IP of the correct container, but to a different MAC. The destination mac matches the MAC of the Hyper-V switch. This seems wrong, but I have no idea what could be causing this issue.
For testing I did start a pod on the linux control-plane server and tried the DNS resolution here. This works just fine.
Steps to Reproduce (for bugs)
For the linux side, of things, the installation is done using the helm chart as described here: Install using Helm The additional configuration is:
Follow the guide here exactly: Install Calico for Windows The setup encapsulation is VXLAN, datastore is Kubernetes.
The alternative installation for calico using the HostProcess containers, also works and has the same outcome.
Context
In general I'm "only" trying the setup a Kubernetes with a Windows Node up.
Your Environment
Final words
At this point I have exhausted all ideas what the reason for this could be. I'd be thankful for any pointers where I could still look for problems.