search

F5Networks / k8s-bigip-ctlr

Repository for F5 Container Ingress Services for Kubernetes & OpenShift.
Apache License 2.0
361 stars 195 forks source link

Can specify port 8443 for 1NIC scenarios? #783

Closed tkam8 closed 6 years ago

tkam8 commented 6 years ago

Description

This is likely an enhancement. Please advise.

I used below helm chart to successfully deploy the BIG-IP controller, but since my BIG-IP is deployed in GCP as a 1-NIC (accessed over port 8443), the connectivity fails. Any advice?

https://github.com/F5Networks/charts/tree/master/src/stable/f5-bigip-ctlr

Kubernetes Version

f5user@k8s-master:~$ kubectl version Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.1", GitCommit:"4ed3216f3ec431b140b1d899130a69fc671678f4", GitTreeState:"clean", BuildDate:"2018-10-05T16:46:06Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.1", GitCommit:"4ed3216f3ec431b140b1d899130a69fc671678f4", GitTreeState:"clean", BuildDate:"2018-10-05T16:36:14Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}

Controller Version

f5user@k8s-master:~$ kubectl logs auxiliary-meerkat-f5-bigip-ctlr-5877c74b9b-j99ph -n kube-system 2018/10/24 15:19:05 [INFO] Starting: Version: v1.7.0, BuildInfo: n1260-443736128

BIG-IP Version

admin@(localhost)(cfg-sync Standalone)(Active)(/Common)(tmos)# show sys version

Sys::Version Main Package Product BIG-IP Version 13.1.1 Build 0.0.4 Edition Final Date Fri Jul 20 17:55:49 PDT 2018

Diagnostic Information

f5user@k8s-master:~$ kubectl logs auxiliary-meerkat-f5-bigip-ctlr-5877c74b9b-j99ph -n kube-system
2018/10/24 15:19:05 [INFO] Starting: Version: v1.7.0, BuildInfo: n1260-443736128
2018/10/24 15:19:05 [INFO] ConfigWriter started: 0xc000380210
2018/10/24 15:19:05 [INFO] Started config driver sub-process at pid: 12
2018/10/24 15:19:05 [INFO] NodePoller (0xc0000b7680) registering new listener: 0x1829650
2018/10/24 15:19:05 [INFO] NodePoller started: (0xc0000b7680)
2018/10/24 15:19:05 [INFO] Watching ConfigMap resources.
2018/10/24 15:19:05 [INFO] Handling ConfigMap resource events.
2018/10/24 15:19:05 [INFO] Registered BigIP Metrics
2018/10/24 15:19:06 [INFO] Wrote 0 Virtual Server and 0 IApp configs
2018/10/24 15:19:06 [ERROR] [2018-10-24 15:19:06,378 __main__ ERROR] Encountered error: BIG-IP connection error: HTTPSConnectionPool(host='10.146.0.5', port=443): Max retries exceeded with url: /mgmt/shared/authn/login (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f4207a34a10>: Failed to establish a new connection: [Errno 111] Connection refused',)). Retrying for 1 seconds.
2018/10/24 15:19:07 [ERROR] [2018-10-24 15:19:07,383 __main__ ERROR] Encountered error: BIG-IP connection error: HTTPSConnectionPool(host='10.146.0.5', port=443): Max retries exceeded with url: /mgmt/shared/authn/login (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f4207a51090>: Failed to establish a new connection: [Errno 111] Connection refused',)). Retrying for 2 seconds.
2018/10/24 15:19:09 [ERROR] [2018-10-24 15:19:09,392 __main__ ERROR] Encountered error: BIG-IP connection error: HTTPSConnectionPool(host='10.146.0.5', port=443): Max retries exceeded with url: /mgmt/shared/authn/login (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f4207a51a90>: Failed to establish a new connection: [Errno 111] Connection refused',)). Retrying for 4 seconds.
2018/10/24 15:19:13 [ERROR] [2018-10-24 15:19:13,408 __main__ ERROR] Encountered error: BIG-IP connection error: HTTPSConnectionPool(host='10.146.0.5', port=443): Max retries exceeded with url: /mgmt/shared/authn/login (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f420773b090>: Failed to establish a new connection: [Errno 111] Connection refused',)). Retrying for 8 seconds.
2018/10/24 15:19:21 [ERROR] [2018-10-24 15:19:21,441 __main__ ERROR] Encountered error: BIG-IP connection error: HTTPSConnectionPool(host='10.146.0.5', port=443): Max retries exceeded with url: /mgmt/shared/authn/login (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f4207a51e90>: Failed to establish a new connection: [Errno 111] Connection refused',)). Retrying for 16 seconds.
2018/10/24 15:19:37 [ERROR] [2018-10-24 15:19:37,505 __main__ ERROR] Encountered error: BIG-IP connection error: HTTPSConnectionPool(host='10.146.0.5', port=443): Max retries exceeded with url: /mgmt/shared/authn/login (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f4207a34310>: Failed to establish a new connection: [Errno 111] Connection refused',)). Retrying for 32 seconds.
2018/10/24 15:20:09 [ERROR] [2018-10-24 15:20:09,629 __main__ ERROR] Encountered error: BIG-IP connection error: HTTPSConnectionPool(host='10.146.0.5', port=443): Max retries exceeded with url: /mgmt/shared/authn/login (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f4207754050>: Failed to establish a new connection: [Errno 111] Connection refused',)). Retrying for 64 seconds.
2018/10/24 15:21:13 [ERROR] [2018-10-24 15:21:13,878 __main__ ERROR] Encountered error: BIG-IP connection error: HTTPSConnectionPool(host='10.146.0.5', port=443): Max retries exceeded with url: /mgmt/shared/authn/login (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f420773b8d0>: Failed to establish a new connection: [Errno 111] Connection refused',)). Retrying for 128 seconds.
2018/10/24 15:23:22 [ERROR] [2018-10-24 15:23:22,352 __main__ ERROR] Encountered error: BIG-IP connection error: HTTPSConnectionPool(host='10.146.0.5', port=443): Max retries exceeded with url: /mgmt/shared/authn/login (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f4207752b50>: Failed to establish a new connection: [Errno 111] Connection refused',)). Retrying for 256 seconds.
2018/10/24 15:27:39 [ERROR] [2018-10-24 15:27:39,356 __main__ ERROR] Encountered error: BIG-IP connection error: HTTPSConnectionPool(host='10.146.0.5', port=443): Max retries exceeded with url: /mgmt/shared/authn/login (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f4207a34f10>: Failed to establish a new connection: [Errno 111] Connection refused',)). Retrying for 512 seconds.
vincentmli commented 6 years ago

Please try bigip-url with value "your-1nic-bigip-ip:8443", it should work

according to https://github.com/f5devcentral/f5-ctlr-agent/blob/master/f5_ctlr_agent/bigipconfigdriver.py#L696


    if 'url' not in bigip:
        raise ConfigError('Configuration file missing "bigip:url" section')
    if ('partitions' not in bigip) or (len(bigip['partitions']) == 0):
        raise ConfigError('Configuration file must specify at least one '
                          'partition in the "bigip:partitions" section')

    url = urlparse(bigip['url'])
    host = url.hostname
    port = url.port
    if not port:
        port = 443
tkam8 commented 6 years ago

thanks @vincentmli , i ended up using a GDM template to deploy a 2-NIC BIG-IP https://github.com/F5Networks/f5-google-gdm-templates/tree/master/supported/standalone/2nic/existing-stack/byol#prerequisites

I will give the above a try when I can.

tkam8 commented 6 years ago

hmm, I tried this but got an error:

      args: [
        "--bigip-username=$(BIGIP_USERNAME)",
        "--bigip-password=$(BIGIP_PASSWORD)",
        "--bigip-url=10.5.1.3:8443",
        "--bigip-partition=kubernetes",
f5user@k8s-master:~/agilitydocs/kubernetes$ kubectl logs k8s-bigip-ctlr-deployment2-6d786854cd-xp8lw -n kube-system
Error parsing url: parse 10.5.1.3:8443: first path segment in URL cannot contain colon
Usage of /app/bin/k8s-bigip-ctlr

I even tried single quotes

f5user@k8s-master:~/agilitydocs/kubernetes$ kubectl logs k8s-bigip-ctlr-deployment2-6fb947f864-7kwz7 -n kube-system
Error parsing url: parse '10.5.1.3:8443': first path segment in URL cannot contain colon

any other ideas? :)

amudukutore commented 6 years ago

You may need to specify an empty scheme to get past that field. Try "//10.5.1.3:8443"

tkam8 commented 6 years ago

thanks @amudukutore , I tried but that doesn't seem to work:

"--bigip-url=//10.5.1.3:8443",

f5user@k8s-master:~/agilitydocs/kubernetes$ kubectl logs k8s-bigip-ctlr-deployment2-67bbf999cb-9vj28 -n kube-system | head
BIGIP-URL path must be empty or '/'; check URL formatting and/or remove //10.5.1.3:8443 from path
Usage of /app/bin/k8s-bigip-ctlr

Even when I used the unicode escape char "--bigip-url=10.5.1.3&#58;:8443", the pod gets created but CC never creates a connection. It keeps trying to use 443

f5user@k8s-master:~/agilitydocs/kubernetes$ kubectl logs k8s-bigip-ctlr-deployment2-66d96c8695-94pp8 -n kube-system
2018/10/31 05:03:11 [INFO] Starting: Version: v1.7.0, BuildInfo: n1260-443736128
2018/10/31 05:03:11 [INFO] ConfigWriter started: 0xc00021ad20
2018/10/31 05:03:11 [INFO] Started config driver sub-process at pid: 13
2018/10/31 05:03:11 [INFO] NodePoller (0xc0001a2990) registering new listener: 0x1829650
2018/10/31 05:03:11 [INFO] NodePoller (0xc0001a2990) registering new listener: 0x18296c0
2018/10/31 05:03:11 [INFO] NodePoller started: (0xc0001a2990)
2018/10/31 05:03:11 [INFO] Watching ConfigMap resources.
2018/10/31 05:03:11 [INFO] Handling ConfigMap resource events.
2018/10/31 05:03:11 [INFO] Registered BigIP Metrics
2018/10/31 05:03:14 [INFO] Wrote 3 Virtual Server and 0 IApp configs
2018/10/31 05:03:31 [ERROR] [2018-10-31 05:03:31,786 __main__ ERROR] Encountered error: BIG-IP connection error: HTTPSConnectionPool(host='10.5.1.3&', port=443): Max retries exceeded with url: /mgmt/shared/authn/login (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f4751a62b90>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',)). Retrying for 1 seconds.

pls let me know if you'd like me to test another pattern

dgarrisonf5 commented 6 years ago

HI @tkam8. In order for the controller to correctly parse the --bigip-url parameter which specific a port number, you must use a full URL scheme e.g. https://10.5.1.3:8443.

Can you please test and report your findings?

Thanks

tkam8 commented 6 years ago

@dgarrisonf5 this works, thanks!

"--bigip-url=https://10.5.1.3:8443"

f5user@k8s-master:~/agilitydocs/kubernetes$ kubectl logs k8s-bigip-ctlr-deployment2-6bcb49d54c-tscms -n kube-system
2018/11/01 01:37:46 [INFO] Starting: Version: v1.7.0, BuildInfo: n1260-443736128
2018/11/01 01:37:46 [INFO] ConfigWriter started: 0xc0004add40
2018/11/01 01:37:46 [INFO] Started config driver sub-process at pid: 13
2018/11/01 01:37:46 [INFO] NodePoller (0xc0003003f0) registering new listener: 0x1829650
2018/11/01 01:37:46 [INFO] NodePoller (0xc0003003f0) registering new listener: 0x18296c0
2018/11/01 01:37:46 [INFO] NodePoller started: (0xc0003003f0)
2018/11/01 01:37:46 [INFO] Watching ConfigMap resources.
2018/11/01 01:37:46 [INFO] Handling ConfigMap resource events.
2018/11/01 01:37:46 [INFO] Registered BigIP Metrics
2018/11/01 01:37:47 [INFO] [2018-11-01 01:37:47,232 __main__ INFO] entering inotify loop to watch /tmp/k8s-bigip-ctlr.config258028959/config.json
2018/11/01 01:37:47 [INFO] Wrote 3 Virtual Server and 0 IApp configs
2018/11/01 01:37:50 [INFO] [2018-11-01 01:37:50,119 f5_cccl.resource.resource INFO] Creating ApiPool: /kubernetes/ingress_kube-system_f5-hello-world
2018/11/01 01:37:50 [INFO] [2018-11-01 01:37:50,233 f5_cccl.resource.resource INFO] Creating ApiInternalDataGroup: /kubernetes/https_redirect_dg
2018/11/01 01:37:50 [INFO] [2018-11-01 01:37:50,269 f5_cccl.resource.resource INFO] Creating ApiIRule: /kubernetes/http_redirect_irule_443
2018/11/01 01:37:50 [INFO] [2018-11-01 01:37:50,340 f5_cccl.resource.resource INFO] Creating ApiVirtualServer: /kubernetes/ingress_10-5-2-100_80
2018/11/01 01:37:50 [INFO] [2018-11-01 01:37:50,474 f5_cccl.resource.resource INFO] Creating ApiVirtualServer: /kubernetes/ingress_10-5-2-100_443
2018/11/01 01:37:50 [INFO] [2018-11-01 01:37:50,527 f5_cccl.resource.resource INFO] Creating ApiVirtualServer: /kubernetes/ingress_10-5-3-1_8080

maybe we should add a :: NOTE to https://clouddocs.f5.com/containers/v2/kubernetes/kctlr-app-install.html that says something like

If you deployed a Single-NIC BIG-IP, which does not use the standard management port, you must specify the full URL scheme including port number in the manifest file. For instance:

"--bigip-url=https://10.10.10.10:8443"

dgarrisonf5 commented 6 years ago

Happy to hear that it worked @tkam8!

I think it does make sense to update the docs, but adding a port to the "--bigip-url" parameter should be documented anyways as there can be any number of reasons that the port might be something other than 443.

Closing this issue