Troubleshooting exercise 5

[RitaGlushkova]

Write-up is on Berkeli's board

[RitaGlushkova]

After a discussion with Radha, it looks like /etc/consul.d/consul.hcl is not the config file we need. The new path is /mnt/consul/consul/consul_config.json

sudo vim /mnt/consul/config/consul-config.json

[RitaGlushkova]

Create an environment variable named CONSUL_CONFIG_DIR and set it to your Consul configuration directory path. export CONSUL_CONFIG_DIR="/mnt/consul/config/"

Make sure the file is present: ls -1 ${CONSUL_CONFIG_DIR} Logs: consul-config.json

Validate config file: consul validate ${CONSUL_CONFIG_DIR}

Logs: Config validation failed: performance.raft_multiplier cannot be 11. Must be between 1 and 10

Change performance.raft_multiplier to 5 (default)

We can see that different servers have different numbers in bootstrap_expect. Mine (server 2) has 5 and Berkeli's (server 1) has 2. We know that we will have 3 servers in total, so we will change this value to 3 as well.

consul validate ${CONSUL_CONFIG_DIR}

Logs:

The 'acl.tokens.master' field is deprecated. Use the 'acl.tokens.initial_management' field instead.
bootstrap_expect > 0: expecting 3 servers
Configuration is valid!

_The 'acl.tokens.master' field is deprecated. Use the 'acl.tokens.initialmanagement' field instead. looks suspicious but it doesn't look like an error for now so we will leave it.

[RitaGlushkova]

Let's try to start consul:

consul agent -config-dir=${CONSUL_CONFIG_DIR}

Logs: ==> Failed to setup logging: open /var/log/consul-1673956064731056477.log: permission denied

This doesn't look right, my directory has to be /var/log/consul/ not /var/log/ Going back to consul-config.json and fixing "log_file":"/var/log/consul/"

[RitaGlushkova]

But still same error, I need to give permissions to consul user and group:

sudo chown consul:consul /var/log/consul
sudo chmod u+w,g+w,o+w /var/log/consul/*
consul agent -config-dir=${CONSUL_CONFIG_DIR}
ls -al /var/log/consul/

Logs:

drwxr-xr-x 2 consul consul     44 Jan 17 12:26 .
drwxr-xr-x 8 root   root     4096 Jan 17 12:04 ..
-rw-rw--w- 1 consul consul 111516 Jan 17 12:31 consul-1673958376420751130.log

now looks better

[RitaGlushkova]

{ "service": { "id": "service1", "name": "service1", "tags": ["primary"], "address": "172.31.51.18", "meta": { "meta": "I'm the first one" }, "token": "a15f4b82-4d0f-1a91-4a5b-8b27285dc13d" } }

{ "node_name": "consul-client1", "acl": { "enabled": true, "default_policy": "deny", "tokens": { "agent": "a9f5c75d-f43f-5dba-7c81-5e967e8628d1" } }, "advertise_addr": "172.31.51.18", "log_file": "/var/log/consul/", "log_json": true, "client_addr": "127.0.0.1 172.31.51.18", "data_dir": "/mnt/consul/data", "datacenter": "cyf5", "retry_join": ["172.31.87.243", "172.31.53.195", "172.31.53.179"], "server": false }