dhiaayachi / temporal

Temporal service
https://docs.temporal.io
MIT License
0 stars 0 forks source link

Cassandra based docker compose fails to start #161

Open dhiaayachi opened 2 months ago

dhiaayachi commented 2 months ago

Expected Behavior

Temporal can be started using docker compose with cassandra, all services must be running

Actual Behavior

Temporal fails to start with the following error:

temporal              | Error: unable to health check "temporal.api.workflowservice.v1.WorkflowService" service: connection error: desc = "transport: Error while dialing: dial tcp 172.18.0.3:7233: connect: connection refused"
temporal              | ('export TEMPORAL_CLI_SHOW_STACKS=1' to see stack traces)
temporal              | Waiting for Temporal server to start...
temporal              | 2024/04/15 12:32:12 error: failed to connect to "[HostInfo hostname=\"172.18.0.2\" connectAddress=\"172.18.0.2\" peer=\"<nil>\" rpc_address=\"0.0.0.0\" broadcast_address=\"172.18.0.2\" preferred_ip=\"<nil>\" connect_addr=\"172.18.0.2\" connect_addr_source=\"connect_address\" port=9042 data_centre=\"datacenter1\" rack=\"rack1\" host_id=\"30e6c360-b68e-4d78-b863-9bb8e4b07646\" version=\"v3.11.9\" state=UP num_tokens=256]" due to error: Keyspace 'temporal_visibility' does not exist

temporal-admin-tools, temporal-ui, temporal-cassandra are Up and running, only temporal itself fails and exits

Steps to Reproduce the Problem

  1. Follow the instructions from https://github.com/temporalio/docker-compose?tab=readme-ov-file#how-to-use to clone the repo and get the compose file
  2. Start the compose using docker compose -f docker-compose-cass.yml up command
  3. Observe the logs and compose services status

Specifications

dhiaayachi commented 1 month ago

Expected Behavior

Temporal can be started using docker compose with cassandra, all services must be running

Actual Behavior

Temporal fails to start with the following error:

temporal              | Error: unable to health check "temporal.api.workflowservice.v1.WorkflowService" service: connection error: desc = "transport: Error while dialing: dial tcp 172.18.0.3:7233: connect: connection refused"
temporal              | ('export TEMPORAL_CLI_SHOW_STACKS=1' to see stack traces)
temporal              | Waiting for Temporal server to start...
temporal              | 2024/04/15 12:32:12 error: failed to connect to "[HostInfo hostname=\"172.18.0.2\" connectAddress=\"172.18.0.2\" peer=\"<nil>\" rpc_address=\"0.0.0.0\" broadcast_address=\"172.18.0.2\" preferred_ip=\"<nil>\" connect_addr=\"172.18.0.2\" connect_addr_source=\"connect_address\" port=9042 data_centre=\"datacenter1\" rack=\"rack1\" host_id=\"30e6c360-b68e-4d78-b863-9bb8e4b07646\" version=\"v3.11.9\" state=UP num_tokens=256]" due to error: Keyspace 'temporal_visibility' does not exist

temporal-admin-tools, temporal-ui, temporal-cassandra are Up and running, only temporal itself fails and exits

Steps to Reproduce the Problem

  1. Follow the instructions from https://github.com/temporalio/docker-compose?tab=readme-ov-file#how-to-use to clone the repo and get the compose file.
  2. Start the compose using docker compose -f docker-compose-cass.yml up command.
  3. Observe the logs and compose services status.

Specifications

Troubleshooting

The error message indicates that the temporal service is unable to connect to the Cassandra cluster for the Visibility store because the temporal_visibility keyspace does not exist.

Solution:

The docker-compose-cass.yml file does not create the temporal_visibility keyspace by default. You will need to add this to your YAML file.

To resolve this issue, follow these steps:

  1. Open docker-compose-cass.yml:

    vim docker-compose-cass.yml
  2. Add the following to the temporal service definition under the environment section:

    environment:
     - TEMPORAL_VISIBILITY_KEYSPACE=temporal_visibility
     - TEMPORAL_VISIBILITY_SETUP=true
     - SKIP_DB_CREATE=false
     # ... other environment variables
  3. Save the file.

  4. Restart the docker-compose stack:

    docker compose -f docker-compose-cass.yml down
    docker compose -f docker-compose-cass.yml up -d

This will create the temporal_visibility keyspace in your Cassandra cluster and the Temporal service should now start successfully.

Explanation:

By setting these environment variables, you ensure that the necessary keyspace and schema are created before the Temporal server starts.