Open guzzijones opened 2 weeks ago
2024-07-01 11:07:48,299 INFO [-] Using config files: /etc/st2/st2.conf,/etc/st2/st2.docker.conf,/etc/st2/st2.user.conf
/etc/st2/st2.docker.conf has:
[coordination]
url = redis://
thanks @fdrab for your input. the st2 services are having problems locating the redis master. any help you or other people can provide is appreciated.
error message for st2api. my st2scheduler has the similar error message of not able to locate redis master
Traceback (most recent call last):
File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2api/cmd/api.py", line 99, in main
return _run_server()
File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2api/cmd/api.py", line 87, in _run_server
sock, app.setup_app(), custom_pool=worker_pool, log=LOG, log_output=False
File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2api/app.py", line 84, in setup_app
router.add_spec(spec, transforms=transforms)
File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2common/router.py", line 262, in add_spec
__import__(module_name)
File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2api/controllers/v1/keyvalue.py", line 533, in <module>
key_value_pair_controller = KeyValuePairController()
File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2api/controllers/v1/keyvalue.py", line 63, in __init__
self._coordinator = coordination.get_coordinator()
File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2common/services/coordination.py", line 253, in get_coordinator
COORDINATOR = coordinator_setup(start_heart=start_heart)
File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2common/services/coordination.py", line 224, in coordinator_setup
coordinator.start(start_heart=start_heart)
File "/opt/stackstorm/st2/lib/python3.8/site-packages/tooz/coordination.py", line 689, in start
super(CoordinationDriverWithExecutor, self).start(start_heart)
File "/opt/stackstorm/st2/lib/python3.8/site-packages/tooz/coordination.py", line 426, in start
self._start()
File "/opt/stackstorm/st2/lib/python3.8/site-packages/tooz/drivers/redis.py", line 450, in _start
self._server_info = self._client.info()
File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__
self.gen.throw(type, value, traceback)
File "/opt/stackstorm/st2/lib/python3.8/site-packages/tooz/drivers/redis.py", line 44, in _translate_failures
utils.raise_with_cause(coordination.ToozConnectionError,
File "/opt/stackstorm/st2/lib/python3.8/site-packages/tooz/utils.py", line 224, in raise_with_cause
excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
File "/opt/stackstorm/st2/lib/python3.8/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
tooz.coordination.ToozConnectionError: No master found for 'master'
here is my setup for redis st2 conf file:
[coordination]
url = redis://default:password@redis-headless.redis-dev.svc.cluster.local:26379?sentinel=master
redis values.yaml
## @param sentinel.masterSet Master set name
##
masterSet: master
in st2client pod, using redis-cli, i was able to locate my redis master
root@stackstorm-st2client-bb9b4c57c-zt7z7:/opt/stackstorm# redis-cli -h redis-headless.redis-dev.svc.cluster.local -p 26379
redis-headless.redis-dev.svc.cluster.local:26379> auth password
OK
redis-headless.redis-dev.svc.cluster.local:26379> sentinel masters
1) 1) "name"
2) "master"
3) "ip"
4) "redis-node-0.redis-headless.redis-dev.svc.cluster.local"
5) "port"
6) "6379"
can't say I'm an expert on any of this, what does your redis section of the values.yaml look like?
how does the st2actionrunner know what ip and port the redis server is running on? I would expect some sort of environment variable passed into the action runner container but I don't see that. Is the redis port somehow mapped to all pods?