search

camunda-community-hub / zeebe-full-helm

Zeebe Cluster + Operate Parent HELM Chart
Apache License 2.0
7 stars 13 forks source link

zeeqs pod crashing after http graphql request. #104

Closed Sargastico closed 3 years ago

Sargastico commented 3 years ago

I did a port fowarding from the zeeqs service running on port 9000 to my local machine, but when I sent a request (POST query or /graphiql), the zeeqs pod crashes with a Error with exit code 1 and restart.

Following the logs I found a error stacktrace.

Sargastico commented 3 years ago

I also finded other log entries that might be helpful to identify the problem:

hz.client_1 [dev] [4.1.1] Exception during initial connection to [localhost]:5701: com.hazelcast.core.HazelcastException: java.net.SocketException: Connection refused to address localhost/127.0.0.1:5701

it looks like the application is trying to connect to hazelcast on localhost. The service endpoint may not be resolved correctly.

salaboy commented 3 years ago

@Sargastico yes those are community charts so we might need some help to get them working as they should. It looks like a configuration is missing to point ZeeQS to the Zeebe Broker which has hazelcast enabled.

salaboy commented 3 years ago

@Sargastico have you looked at this: https://github.com/zeebe-io/zeebe-helm-profiles/blob/master/zeebe-dev-with-zeeqs-profile.yaml ?

Sargastico commented 3 years ago

@salaboy I tried this profile with helm, but the zeeqs still points to localhost and not to the hazelcast service.

I also tried to manually insert the env variable (hard coding it, replacing the helm resolver code) inside the zeebe-full-helm/charts/zeebe-zeeqs-helm/templates/deployment.yaml called ZEEBE_HAZELCAST_CONNECTION, but it had no effect.

Maybe the environment variables are not overriding the default address (localhost) of hazelcast on zeeqs.

salaboy commented 3 years ago

@Sargastico yes.. that might be the case.. can you look into the container.. maybe the env variable has changed.. Looking at the zeeqs repo.. I think that variable has changed to: ZEEBE_CLIENT_WORKER_HAZELCAST_CONNECTION can you give that a try?

https://github.com/zeebe-io/zeeqs/blob/master/app/src/main/resources/application.yml#L8

salaboy commented 3 years ago

remember that you can always do a kubectl edit deploy to change that in the cluster.. and if that works then we can send a PR to the Chart

Sargastico commented 3 years ago

@salaboy It's working! Changing the name of the env variable to ZEEBE_CLIENT_WORKER_HAZELCAST_CONNECTION fix the issue.

I sent the PR.

salaboy commented 3 years ago

@Sargastico awesome!!! glad to hear that helped!