search

F5Networks / k8s-bigip-ctlr

Repository for F5 Container Ingress Services for Kubernetes & OpenShift.
Apache License 2.0
357 stars 195 forks source link

ctlr cant get pods MAC #2400

Closed Jeremywenxuan closed 1 year ago

Jeremywenxuan commented 2 years ago

Setup Details

CIS Version : 2.8.1 Build: f5networks/k8s-bigip-ctlr:latest(2.8.1) BIGIP Version: Big IP 15.1.5.1 Build 0.0.14 AS3 Version: 3.36.0 Agent Mode: AS3 Orchestration: K8S Orchestration Version:1.20.12 Pool Mode: Cluster Additional Setup details: CNI Plugins: calico.vxlan Platform: CentOS Linux release 7.9.2009

Description

The CNI plugin is Calico.VXLAN, using port 4789. The F5 tunnels port has changed to 4789.

But when creating L7 YAML, ctlr cant get the pods MAC, but can get the pods IP, here is ctlrs log

2022/05/10 07:58:22 [DEBUG] [AS3] Response from BIG-IP: code: 200 --- tenant:kubernetes --- message: no change
2022/05/10 07:58:22 [DEBUG] [AS3] Response from BIG-IP: code: 200 --- tenant:l7_vs_demo --- message: success
2022/05/10 07:58:22 [DEBUG] [AS3] Preparing response message to response handler for arp and fdb config
2022/05/10 07:58:22 [DEBUG] [AS3] AppManager wrote endpoints to VxlanMgr
2022/05/10 07:58:22 [DEBUG] [AS3] Sent response message to response handler for arp and fdb config
2022/05/10 07:58:22 [DEBUG] [AS3] Posting AS3 Declaration
2022/05/10 07:58:22 [DEBUG] [AS3] posting request to https://10.8.2.67/mgmt/shared/appsvcs/declare/
2022/05/10 07:58:22 [ERROR] [VxLAN] Vxlan manager could not get VtepMac for 172.22.42.230's node.
2022/05/10 07:58:33 [DEBUG] [AS3] Response from BIG-IP: code: 200 --- tenant:kubernetes --- message: no change
2022/05/10 07:58:33 [DEBUG] [AS3] Response from BIG-IP: code: 200 --- tenant:l7_vs_demo --- message: success
2022/05/10 07:58:33 [DEBUG] [AS3] Preparing response message to response handler for arp and fdb config
2022/05/10 07:58:33 [DEBUG] [AS3] AppManager wrote endpoints to VxlanMgr
2022/05/10 07:58:33 [DEBUG] [AS3] Sent response message to response handler for arp and fdb config
2022/05/10 07:58:33 [ERROR] [VxLAN] Vxlan manager could not get VtepMac for 172.22.42.210's node.
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,103 __main__ DEBUG] config handler woken for reset
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,105 __main__ DEBUG] loaded configuration file successfully
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,105 __main__ DEBUG] NET Config: {"userFdbTunnels": [{"name": "calico-vxlan", "records": [{"name": "0a:0a:0a:1c:02:01", "endpoint": "10.28.2.1"}, {"name": "0a:0a:0a:1c:02:02", "endpoint": "10.28.2.2"}, {"name": "0a:0a:0a:1c:02:03", "endpoint": "10.28.2.3"}, {"name": "0a:0a:0a:1c:02:04", "endpoint": "10.28.2.4"}, {"name": "0a:0a:0a:1c:02:05", "endpoint": "10.28.2.5"}]}]}
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,105 f5_cccl.service.manager DEBUG] apply_net_config start
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,105 f5_cccl.service.validation DEBUG] Validating desired config against CCCL API schema.
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,105 f5_cccl.service.validation DEBUG] validate start
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,106 f5_cccl.service.validation DEBUG] validate took 0.00073 seconds.
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,106 icontrol.session DEBUG] get WITH uri: https://10.8.2.67:443/mgmt/tm/auth/partition/Common AND suffix:  AND kwargs: {}
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,108 urllib3.connectionpool DEBUG] Resetting dropped connection: 10.8.2.67
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,151 urllib3.connectionpool DEBUG] https://10.8.2.67:443 "GET /mgmt/tm/auth/partition/Common HTTP/1.1" 200 257
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,152 icontrol.session DEBUG] RESPONSE::STATUS: 200 Content-Type: application/json; charset=UTF-8 Content-Encoding: None
2022/05/10 07:58:45 [INFO] Text: '{"kind":"tm:auth:partition:partitionstate","name":"Common","fullPath":"Common","generation":0,"selfLink":"https://localhost/mgmt/tm/auth/partition/Common?ver=15.1.5.1","defaultRouteDomain":0,"description":"Repository for system objects and shared objects."}'
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,153 f5_cccl.bigip DEBUG] Refreshing the BIG-IP net cached state...
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,154 icontrol.session DEBUG] get WITH uri: https://10.8.2.67:443/mgmt/tm/auth/partition/Common AND suffix:  AND kwargs: {}
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,166 urllib3.connectionpool DEBUG] https://10.8.2.67:443 "GET /mgmt/tm/auth/partition/Common HTTP/1.1" 200 257
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,167 icontrol.session DEBUG] RESPONSE::STATUS: 200 Content-Type: application/json; charset=UTF-8 Content-Encoding: None
2022/05/10 07:58:45 [INFO] Text: '{"kind":"tm:auth:partition:partitionstate","name":"Common","fullPath":"Common","generation":0,"selfLink":"https://localhost/mgmt/tm/auth/partition/Common?ver=15.1.5.1","defaultRouteDomain":0,"description":"Repository for system objects and shared objects."}'
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,167 f5_cccl.bigip DEBUG] Retrieving arps from BIG-IP /Common...
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,168 icontrol.session DEBUG] get WITH uri: https://10.8.2.67:443/mgmt/tm/net/arp/ AND suffix:  AND kwargs: {'params': '$filter=partition+eq+Common'}
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,179 urllib3.connectionpool DEBUG] https://10.8.2.67:443 "GET /mgmt/tm/net/arp/?$filter=partition+eq+Common HTTP/1.1" 200 139
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,180 icontrol.session DEBUG] RESPONSE::STATUS: 200 Content-Type: application/json; charset=UTF-8 Content-Encoding: None
2022/05/10 07:58:45 [INFO] Text: '{"kind":"tm:net:arp:arpcollectionstate","selfLink":"https://localhost/mgmt/tm/net/arp?$filter=partition+eq+Common&ver=15.1.5.1","items":[]}'
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,180 f5_cccl.bigip DEBUG] Retrieving fdb tunnels from BIG-IP /Common...
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,181 icontrol.session DEBUG] get WITH uri: https://10.8.2.67:443/mgmt/tm/net/fdb/tunnel/ AND suffix:  AND kwargs: {}
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,194 urllib3.connectionpool DEBUG] https://10.8.2.67:443 "GET /mgmt/tm/net/fdb/tunnel/ HTTP/1.1" 200 1194
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,195 icontrol.session DEBUG] RESPONSE::STATUS: 200 Content-Type: application/json; charset=UTF-8 Content-Encoding: None
2022/05/10 07:58:45 [INFO] Text: '{"kind":"tm:net:fdb:tunnel:tunnelcollectionstate","selfLink":"https://localhost/mgmt/tm/net/fdb/tunnel?ver=15.1.5.1","items":[{"kind":"tm:net:fdb:tunnel:tunnelstate","name":"calico-vxlan","partition":"Common","fullPath":"/Common/calico-vxlan","generation":1,"selfLink":"https://localhost/mgmt/tm/net/fdb/tunnel/~Common~calico-vxlan?ver=15.1.5.1","recordsReference":{"link":"https://localhost/mgmt/tm/net/fdb/tunnel/~Common~calico-vxlan/records?ver=15.1.5.1","isSubcollection":true}},{"kind":"tm:net:fdb:tunnel:tunnelstate","name":"http-tunnel","partition":"Common","fullPath":"/Common/http-tunnel","generation":1,"selfLink":"https://localhost/mgmt/tm/net/fdb/tunnel/~Common~http-tunnel?ver=15.1.5.1","recordsReference":{"link":"https://localhost/mgmt/tm/net/fdb/tunnel/~Common~http-tunnel/records?ver=15.1.5.1","isSubcollection":true}},{"kind":"tm:net:fdb:tunnel:tunnelstate","name":"socks-tunnel","partition":"Common","fullPath":"/Common/socks-tunnel","generation":1,"selfLink":"https://localhost/mgmt/tm/net/fdb/tunnel/~Common~socks-tunnel?ver=15.1.5.1","recordsReference":{"link":"https://localhost/mgmt/tm/net/fdb/tunnel/~Common~socks-tunnel/records?ver=15.1.5.1","isSubcollection":true}}]}'
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,197 icontrol.session DEBUG] get WITH uri: https://10.8.2.67:443/mgmt/tm/net/fdb/tunnel/~Common~calico-vxlan/records/ AND suffix:  AND kwargs: {}
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,235 urllib3.connectionpool DEBUG] https://10.8.2.67:443 "GET /mgmt/tm/net/fdb/tunnel/~Common~calico-vxlan/records/ HTTP/1.1" 200 1475
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,236 icontrol.session DEBUG] RESPONSE::STATUS: 200 Content-Type: application/json; charset=UTF-8 Content-Encoding: None
2022/05/10 07:58:45 [INFO] Text: '{"kind":"tm:net:fdb:tunnel:records:recordscollectionstate","selfLink":"https://localhost/mgmt/tm/net/fdb/tunnel/~Common~calico-vxlan/records?ver=15.1.5.1","items":[{"kind":"tm:net:fdb:tunnel:records:recordsstate","name":"0a:0a:0a:1c:02:01","fullPath":"0a:0a:0a:1c:02:01","generation":1,"selfLink":"https://localhost/mgmt/tm/net/fdb/tunnel/~Common~calico-vxlan/records/0a:0a:0a:1c:02:01?ver=15.1.5.1","endpoint":"10.28.2.1%0"},{"kind":"tm:net:fdb:tunnel:records:recordsstate","name":"0a:0a:0a:1c:02:02","fullPath":"0a:0a:0a:1c:02:02","generation":1,"selfLink":"https://localhost/mgmt/tm/net/fdb/tunnel/~Common~calico-vxlan/records/0a:0a:0a:1c:02:02?ver=15.1.5.1","endpoint":"10.28.2.2%0"},{"kind":"tm:net:fdb:tunnel:records:recordsstate","name":"0a:0a:0a:1c:02:03","fullPath":"0a:0a:0a:1c:02:03","generation":1,"selfLink":"https://localhost/mgmt/tm/net/fdb/tunnel/~Common~calico-vxlan/records/0a:0a:0a:1c:02:03?ver=15.1.5.1","endpoint":"10.28.2.3%0"},{"kind":"tm:net:fdb:tunnel:records:recordsstate","name":"0a:0a:0a:1c:02:04","fullPath":"0a:0a:0a:1c:02:04","generation":1,"selfLink":"https://localhost/mgmt/tm/net/fdb/tunnel/~Common~calico-vxlan/records/0a:0a:0a:1c:02:04?ver=15.1.5.1","endpoint":"10.28.2.4%0"},{"kind":"tm:net:fdb:tunnel:records:recordsstate","name":"0a:0a:0a:1c:02:05","fullPath":"0a:0a:0a:1c:02:05","generation":1,"selfLink":"https://localhost/mgmt/tm/net/fdb/tunnel/~Common~calico-vxlan/records/0a:0a:0a:1c:02:05?ver=15.1.5.1","endpoint":"10.28.2.5%0"}]}'
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,238 icontrol.session DEBUG] get WITH uri: https://10.8.2.67:443/mgmt/tm/net/fdb/tunnel/~Common~http-tunnel/records/ AND suffix:  AND kwargs: {}
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,251 urllib3.connectionpool DEBUG] https://10.8.2.67:443 "GET /mgmt/tm/net/fdb/tunnel/~Common~http-tunnel/records/ HTTP/1.1" 200 165
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,252 icontrol.session DEBUG] RESPONSE::STATUS: 200 Content-Type: application/json; charset=UTF-8 Content-Encoding: None
2022/05/10 07:58:45 [INFO] Text: '{"kind":"tm:net:fdb:tunnel:records:recordscollectionstate","selfLink":"https://localhost/mgmt/tm/net/fdb/tunnel/~Common~http-tunnel/records?ver=15.1.5.1","items":[]}'
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,252 icontrol.session DEBUG] get WITH uri: https://10.8.2.67:443/mgmt/tm/net/fdb/tunnel/~Common~socks-tunnel/records/ AND suffix:  AND kwargs: {}
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,265 urllib3.connectionpool DEBUG] https://10.8.2.67:443 "GET /mgmt/tm/net/fdb/tunnel/~Common~socks-tunnel/records/ HTTP/1.1" 200 166
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,266 icontrol.session DEBUG] RESPONSE::STATUS: 200 Content-Type: application/json; charset=UTF-8 Content-Encoding: None
2022/05/10 07:58:45 [INFO] Text: '{"kind":"tm:net:fdb:tunnel:records:recordscollectionstate","selfLink":"https://localhost/mgmt/tm/net/fdb/tunnel/~Common~socks-tunnel/records?ver=15.1.5.1","items":[]}'
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,266 f5_cccl.bigip DEBUG] BIG-IP net refresh took 0.11281 seconds.
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,266 f5_cccl.service.manager DEBUG] Getting arp tasks...
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,266 f5_cccl.service.manager DEBUG] Getting tunnel tasks...
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,266 f5_cccl.service.manager DEBUG] Getting pre-existing tunnel update tasks...
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,267 f5_cccl.service.manager DEBUG] Building task lists...
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,267 f5_cccl.service.manager DEBUG] Service task queue length: 0
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,267 f5_cccl.service.manager DEBUG] Creating 0 resources...
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,267 f5_cccl.service.manager DEBUG] Updating 0 resources...
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,267 f5_cccl.service.manager DEBUG] Deleting 0 resources...
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,267 f5_cccl.service.manager DEBUG] apply_net_config took 0.16179 seconds.
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,267 __main__ DEBUG] loaded configuration file successfully
2022/05/10 07:58:45 [DEBUG] [2022-05-10 07:58:45,268 __main__ DEBUG] updating tasks finished, took 0.16430354118347168 seconds

1 2 3

trinaths commented 2 years ago

@Jeremygczhang - below is the error. 

2022/05/10 07:58:22 [ERROR] [VxLAN] Vxlan manager could not get VtepMac for 172.22.42.230's node.

CIS looking at flannel annotations in nodes. For calico setup see https://community.f5.com/t5/technical-articles/cis-and-kubernetes-part-1-install-kubernetes-and-calico/ta-p/291352 or https://clouddocs.f5.com/containers/latest/userguide/calico-config.html

vincentmli commented 2 years ago

@Jeremygczhang I suggest you try Cilium CNI instead of Calico, Cilium is suppose to replace Calico and works much better with BIG-IP, see https://github.com/f5devcentral/f5-ci-docs/blob/master/docs/kubernetes/cilium-bigip-info.rst

trinaths commented 1 year ago

Closing this issue. No update from user.