search

confluentinc / cp-docker-images

[DEPRECATED] Docker images for Confluent Platform.
Apache License 2.0
1.14k stars 705 forks source link

Dockerized connect - CONNECT_LISTENERS & CONNECT_REST_ADVERTISED_** #924

Closed mmowris closed 2 years ago

mmowris commented 2 years ago

Im attempting to run a Kafka Connect container on Heroku, inside a private space.

Im hoping for help setting: CONNECT_LISTENERS, CONNECT_REST_PORT, CONNECT_REST_ADVERTISED_HOST_NAME, CONNECT_REST_ADVERTISED_PORT.

From my understanding: CONNECT_LISTENERS - this is the external endpoint for Kafka REST.. thus CONNECT_LISTENERS="http://myapp.herokuapp.com:$PORT}

CONNECT_REST_PORT - If the process doesn't bind to the servers port [Heroku dynos] within 60s, the dyno reboots. So CONNECT_REST_PORT=${PORT}

CONNECT_REST_ADVERTISED_LISTENER="HTTP" - This is http as heroku routes via http

Here comes the issue.. Ive tried a variety of configs for the following..

CONNECT_REST_ADVERTISED_HOST_NAME - From the docs, this is the internal hostname/ip - so with (private space DNS discovery)[https://devcenter.heroku.com/articles/dyno-dns-service-discovery#usage], I can get an internal hostname of the form http://web.myapp.app.localspace: and with a private space, I can use a static port - say 5000.

The issue is, my REST API fails to bind...

2022-06-03T02:08:22.869326+00:00 app[web.1]: [2022-06-03 02:08:22,869] INFO jetty-9.4.43.v20210629; built: 2021-06-30T11:07:22.254Z; git: 526006ecfa3af7f1a27ef3a288e2bef7ea9dd7e8; jvm 11.0.13+8-LTS (org.eclipse.jetty.server.Server)
2022-06-03T02:08:22.894502+00:00 app[web.1]: [2022-06-03 02:08:22,891] ERROR Stopping due to error (org.apache.kafka.connect.cli.ConnectDistributed)
2022-06-03T02:08:22.894503+00:00 app[web.1]: org.apache.kafka.connect.errors.ConnectException: Unable to initialize REST server
2022-06-03T02:08:22.894503+00:00 app[web.1]: at org.apache.kafka.connect.runtime.rest.RestServer.initializeServer(RestServer.java:216)
2022-06-03T02:08:22.894504+00:00 app[web.1]: at org.apache.kafka.connect.cli.ConnectDistributed.startConnect(ConnectDistributed.java:101)
2022-06-03T02:08:22.894504+00:00 app[web.1]: at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:80)
2022-06-03T02:08:22.894505+00:00 app[web.1]: Caused by: java.io.IOException: Failed to bind to myapp.herokuapp.com/52.202.52.153:33635
2022-06-03T02:08:22.894505+00:00 app[web.1]: at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:349)
2022-06-03T02:08:22.894506+00:00 app[web.1]: at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:310)
2022-06-03T02:08:22.894506+00:00 app[web.1]: at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
2022-06-03T02:08:22.894507+00:00 app[web.1]: at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:234)
2022-06-03T02:08:22.894507+00:00 app[web.1]: at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
2022-06-03T02:08:22.894507+00:00 app[web.1]: at org.eclipse.jetty.server.Server.doStart(Server.java:401)
2022-06-03T02:08:22.894508+00:00 app[web.1]: at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
2022-06-03T02:08:22.894508+00:00 app[web.1]: at org.apache.kafka.connect.runtime.rest.RestServer.initializeServer(RestServer.java:214)
2022-06-03T02:08:22.894509+00:00 app[web.1]: ... 2 more
2022-06-03T02:08:22.894509+00:00 app[web.1]: Caused by: java.net.BindException: Cannot assign requested address
2022-06-03T02:08:22.894510+00:00 app[web.1]: at java.base/sun.nio.ch.Net.bind0(Native Method)
2022-06-03T02:08:22.894510+00:00 app[web.1]: at java.base/sun.nio.ch.Net.bind(Net.java:455)
2022-06-03T02:08:22.894510+00:00 app[web.1]: at java.base/sun.nio.ch.Net.bind(Net.java:447)
2022-06-03T02:08:22.894511+00:00 app[web.1]: at java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227)
2022-06-03T02:08:22.894511+00:00 app[web.1]: at java.base/sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:80)
2022-06-03T02:08:22.894512+00:00 app[web.1]: at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:344)
2022-06-03T02:08:22.894512+00:00 app[web.1]: ... 9 more
2022-06-03T02:08:27.675168+00:00 heroku[web.1]: Error R10 (Boo

What's the correct differentiation between CONNECT_LISTENERS and CONNECT_REST_ADVERTISED_HOST_NAME for managed instances? I can configure as I wish, my ssl handshake completes, I can ping my kafka instance.. but I cannot for the life of me stand up kafka connect