kafka 0.10.2.0 version CONNECTIVITY ERROR

[nile]

can upgrade to 0.10.2.0?

[Antwnis]

Hi @nile - Closing this one as the latest release 0.9.1 supports 0.10.2.0 It's available at https://github.com/Landoop/kafka-topics-ui/releases

[ghost]

I'm not sure what the exact issue was initially, but I'm getting this on the home page of kafka-topic-ui and there is not connectivity issue between kafka-topics-ui and the Kafka REST interface.

KAFKA REST http://kafka-rest:8082 CONNECTIVITY ERROR KAFKA-TOPICS-UI v0.9.2

I can wget the REST interface from the kafka-topics-ui container. Any other reason why this might be happening?

The kafka-topics-ui logs show nothing except:

2017-05-19T15:01:40.463958984Z Kafka REST Proxy URL to http://kafka-rest:8082.
2017-05-19T15:01:40.466414252Z 
2017-05-19T15:01:40.502393431Z Activating privacy features... done.
2017-05-19T15:01:40.502416615Z http://0.0.0.0:8000

[jglambed]

Try changing your KAFKA_REST url in your env.js file to localhost:8082 That should work

[ghost]

@jglambed Thx for the quick response! I did make that change, but the browser shows this now instead. So it shows the updated URL (with localhost) but still same ERROR msg.

KAFKA REST http://localhost:8082 CONNECTIVITY ERROR KAFKA-TOPICS-UI v0.9.2

[ghost]

@jglambed Plus, I'm getting:

# wget localhost:8082
--2017-05-19 15:53:01--  http://localhost:8082/
Resolving localhost... ::1, 127.0.0.1
Connecting to localhost|::1|:8082... failed: Connection refused.
Connecting to localhost|127.0.0.1|:8082... failed: Connection refused.

from the kafka-topics-ui container.

[andmarios]

I think you hit some CORS issues. You can either enable CORS at REST Proxy, or use our docker image's PROXY functionality to circumvent the checks.

Use your old address, which worked (KAFKA_REST_PROXY_URL=http://kafka-rest:8082) and add another environment variable to the Kafka Topics UI docker container: PROXY=true.

[andmarios]

Oh, another issue that isn't your case but may apply if you fix the error above. Kafka Topics UI versions 0.9.1 and 0.9.2 have some issues that we investigate currently.

You can safely use the 0.8.3 version which works with REST Proxy 0.10.2 but doesn't take advantage of its new API.

[ghost]

Hi @andmarios thx for your help. I applied you CORS fix with a PROXY=true env var, but now I get: KAFKA REST /api/kafka-rest-proxy CONNECTIVITY ERROR KAFKA-TOPICS-UI v0.9.2

In the browser. Next, I'm going to try version 0.8.3.

[andmarios]

Did you also set the address for REST Proxy? I believe in your setup (which I don't know but from what I understand) http://localhost:8082 is wrong.

[andmarios]

Also you can use your browser's console to detect errors. In chrome, while in the Kafka Topics UI tab, you would press CTRL+SHIFT+i and it would open (or from the menu -> More Tools -> Developer tools).

Maybe it could give us a hint to your issue.

[ghost]

@andmarios Oh yes, I did change env.js back to: Strange. I did set KAFKA_REST: "http://kafka-rest:8082" like it was before, but since changing to PROXY=true, I'm seeing this in env.js:

/ # cat /kafka-topics-ui/env.js 
var clusters = [
   {
     NAME:"default",
     KAFKA_REST: "/api/kafka-rest-proxy",
     MAX_BYTES: "?max_bytes=50000"
   }
]

[andmarios]

Yeap, this is due to the PROXY setting, everything seems normal. You could check the REST address in the file /caddy/Caddyfile if you'd like.

[ghost]

@andmarios I'm not familiar with Caddy:

/ # cat caddy/Caddyfile 
0.0.0.0:8000
tls off

root /kafka-topics-ui
log /access.log
proxy /api/kafka-rest-proxy http://kafka-rest:8082 {
    without /api/kafka-rest-proxy

}

[andmarios]

Everything is good here. We wanted to check the REST Proxy address which seem correct. Still not sure why you the connectivity error.

Did the developer tools from your browser provided any hint?

[ghost]

@andmarios There are a bunch of 404 errors in the dev console like this:

HTTP ERROR:  Failed at method [GET] [/api/kafka-rest-proxy/topics] with error: 
{"data":"404 page not found\n","status":404,"config":{"method":"GET","transformRequest":[null],"transformResponse":[null],"jsonpCallbackParam":"callback","url":"/api/kafka-rest-proxy/topics","dataType":"json","headers":{}},"statusText":""} 
DATA SENT: undefined

and

HTTP ERROR:  Failed at method [POST] [/api/kafka-rest-proxy/consumers/kafka_topics_ui_json_2017-05-19-01-31-39-279] with error: 
{"data":"404 page not found\n","status":404,"config":{"method":"POST","transformRequest":[null],"transformResponse":[null],"jsonpCallbackParam":"callback","url":"/api/kafka-rest-proxy/consumers/kafka_topics_ui_json_2017-05-19-01-31-39-279","data":"{\"name\": \"kafka-topics-ui-json\", \"format\": \"json\", \"auto.offset.reset\": \"earliest\", \"auto.commit.enable\": \"false\"}","dataType":"json","headers":{"Content-Type":"application/vnd.kafka.v2+json","Accept":""}},"statusText":""} 
DATA SENT: {"name": "kafka-topics-ui-json", "format": "json", "auto.offset.reset": "earliest", "auto.commit.enable": "false"}

[andmarios]

Hmm, it seems that your REST Proxy isn't listening at http://kafka-rest:8082 either? What's your setup?

[ghost]

My setup is Kubernetes. There is a Kafka cluster (3 pods), a Kafka reREST pod and a kafka-topics-ui pod. The Kafka REST pod talks to the cluster through a headless service successfully, showing in its logs that it discovers zookeeper and the kafka nodes. Then the kakfa-topic-ui pod can reach the Kafka REST pod via wget on 8082:

/ # wget kafka-rest:8082
--2017-05-19 17:36:03--  http://kafka-rest:8082/
Resolving kafka-rest... 10.114.89.104
Connecting to kafka-rest|10.114.89.104|:8082... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2 [application/vnd.kafka.v1+json]
Saving to: 'index.html'

     0K                                    100%  421K=0s

2017-05-19 17:36:03 (421 KB/s) - 'index.html' saved [2/2]

[andmarios]

From the length of the index.html I gather the response is correct? I.e {}?

Could you try to hit http://kafka-topics-ui:8000/api/kafka-rest-proxy and tell us what it returns? Replace the address with the proper one for you.

[ghost]

Thanks. When I hit https://[our-domain]/dev/[removed]sss/kafka-ui/kafka-ui/api/kafka-rest-proxy I get {} in the browser. Same with a curl. The URL I use to get the home page of kafka-topics-ui is: https://platform.bose.io/dev/svc-trio-model-builder/kafka-ui/kafka-ui/

[andmarios]

Oh, ok. I think the PROXY option introduced a second bug, due to not serving the container from the root folder (ie https://[our-domain]/ instead of https://[our-domain]/dev/[removed]sss/kafka-ui/kafka-ui/.

But even then, the Kafka Topics UI is a javascript application that runs in your browser. So your browser needs direct access to the REST Proxy when not proxying via our docker image, thus http://kafka-rest:8082 won't work either.

I can suggest something but I am not sure if it will work.

You could enable CORS for the REST Proxy. If you use Confluent's docker images, you should add these variables:

ACCESS_CONTROL_ALLOW_ORIGIN_DEFAULT="*"
ACCESS_CONTROL_ALLOW_METHODS_DEFAULT=GET,POST,PUT,DELETE,OPTIONS
ACCESS_CONTROL_ALLOW_ORIGIN=*
ACCESS_CONTROL_ALLOW_METHODS=GET,POST,PUT,DELETE,OPTIONS

And for Kafka Topics UI you should remove the PROXY variable and adjust the KAFKA_REST address to something like https://[our-domain]/dev/[removed]sss/kafka-rest.

Again, I am not sure if this will work (not sure if you can serve the REST Proxy from a non-root folder with default kubernetes proxying, we use a special proxy method in Caddy), but maybe worth a try.

[ghost]

Thanks again! So, I can make the kafka-rest endpoint available to the browser from within Kubernetes. I'll just put: https://[our-domain]/dev/[removed]sss/kafka-rest in KAFKA_REST and I think that should solve it. If that doesn't work, then I'll try the CORS and ACCESS_CONTROL... vars.

[andmarios]

Yes, try this with the PROXY option set to true. Not sure if it will work but it is an easy test.

[ghost]

@andmarios OK, it's all working now. So what I had to do was:

• set the KAFKA_REST_PROXY_URL var for the container to https://[our-domain]/dev/[removed]/[removed//kafka-rest
• not set PROXY=true
• Click Disable cache under Chrome developer tools, Network > Disable Cache (or open an incognito window, CTRL+F5 does not work.

When PROXY=true that redirect to /api/kafka-rest-proxy (I think through Caddy) gets activated but it's not needed for my setup.

Thanks so much for your help. I'm going to try the latest version of the container.

[Antwnis]

@Ian-Rossi-aimtheory Thanks for all the feedback

The Disable cache issue - we will soon resolve it as part of a next release of the UI tools - so we will instruct AngularJS to invalidate cached template pages

We are also investigating why 0.9.x is not working with cp-docker images

Don't forget to star the projects that you like / use 👍