Failed to connect stackstorm in k8s to MongoDB atlas

[yardengumai]

Receiving that err for mongo atlas connection with stackstorm in kubernetes: 2021-12-27 12:39:16,014 INFO [-] Patching sys.stdout and re-opening it with utf-8 encoding (originally opened with UTF-8 encoding)... 2021-12-27 12:39:16,016 INFO [-] Using Python: 3.6.9 (/opt/stackstorm/st2/bin/python) 2021-12-27 12:39:16,016 INFO [-] Using fs encoding: utf-8, default encoding: utf-8, locale: en_US.UTF-8, LANG env variable: en_US.UTF-8, PYTHONIOENCODING env variable: notset 2021-12-27 12:39:16,017 INFO [-] Using config files: /etc/st2/st2.conf,/etc/st2/st2.docker.conf,/etc/st2/st2.user.conf 2021-12-27 12:39:16,017 INFO [-] Using logging config: /etc/st2/logging.api.gunicorn.conf 2021-12-27 12:39:16,065 INFO [-] Connecting to database "st2" @ "USER.azure.mongodb.net:27017,USER.azure.mongodb.net:27017USERazure.mongodb.net:27017 (replica set)" as user "st2-t". 2021-12-27 12:39:19,149 ERROR [-] Failed to connect to database "st2" @ "USER.azure.mongodb.net:27017,USER.azure.mongodb.net:27017 (replica set)" as user "st2-t": connection closed,connection closed,connection closed 2021-12-27 12:39:19,150 ERROR [-] (PID=1) ST2 API quit due to exception. Traceback (most recent call last): File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2api/cmd/api.py", line 85, in main _setup() File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2api/cmd/api.py", line 59, in _setup service_registry=True, capabilities=capabilities) File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/service_setup.py", line 226, in setup db_setup() File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/database_setup.py", line 57, in db_setup connection = db_init.db_setup_with_retry(db_cfg) File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/persistence/db_init.py", line 76, in db_setup_with_retry ssl_match_hostname=ssl_match_hostname) File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/persistence/db_init.py", line 59, in db_func_with_retry return retrying_obj.call(db_func, *args, *kwargs) File "/opt/stackstorm/st2/lib/python3.6/site-packages/retrying.py", line 206, in call return attempt.get(self._wrap_exception) File "/opt/stackstorm/st2/lib/python3.6/site-packages/retrying.py", line 247, in get six.reraise(self.value[0], self.value[1], self.value[2]) File "/opt/stackstorm/st2/lib/python3.6/site-packages/six.py", line 696, in reraise raise value File "/opt/stackstorm/st2/lib/python3.6/site-packages/retrying.py", line 200, in call attempt = Attempt(fn(args, kwargs), attempt_number, False) File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/models/db/init.py", line 170, in db_setup ssl_match_hostname=ssl_match_hostname) File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/models/db/init.py", line 152, in _db_connect raise e File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/models/db/init.py", line 145, in _db_connect connection.admin.command('ismaster') File "/opt/stackstorm/st2/lib/python3.6/site-packages/pymongo/database.py", line 730, in command read_preference, session) as (sock_info, slave_ok): File "/usr/lib/python3.6/contextlib.py", line 81, in enter return next(self.gen) File "/opt/stackstorm/st2/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1298, in _socket_for_reads server = self._select_server(read_preference, session) File "/opt/stackstorm/st2/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1253, in _select_server server = topology.select_server(server_selector) File "/opt/stackstorm/st2/lib/python3.6/site-packages/pymongo/topology.py", line 235, in select_server address)) File "/opt/stackstorm/st2/lib/python3.6/site-packages/pymongo/topology.py", line 193, in select_servers selector, server_timeout, address) File "/opt/stackstorm/st2/lib/python3.6/site-packages/pymongo/topology.py", line 209, in _select_servers_loop self._error_message(selector)) pymongo.errors.ServerSelectionTimeoutError: connection closed,connection closed,connection closed

My conf: [database] host = "mongodb+srv://st2-t:f45ew2.mongodb.net/st2-t?retryWrites=true&w=majority" username = st2-t password = f45ew2 db_name = st2-t port = 27017 ssl = False

env: stackstorm 3.4

[sandesvitor]

I'm currently having a similar issue in my release when trying to connect with MongoDB Atlas (we debug the VPC peering between Atlas and our subnet and we were able to connect to Atlas cluster using mongosh). My st2.config regarding the [database] is similar to @yardengumai.

Checking st2 requirements.txt and fixed-requirements.txt, maybe the issue lies in installing also pymongo[srv] driver? Like:

dnspython==2.2.0
pymongo[srv]==4.0.1

[nzlosh]

What is preventing you from listing the mongodb nodes as per the official St2 documentation https://docs.stackstorm.com/install/config/config.html?highlight=mdb_node_1#configure-mongodb rather than depending on DNS records to resolve them?

[sandesvitor]

Wow! I've just resolve it by setting the "ssl" key to true, this way the srv driver was able to work (thanks for the doc @nzlosh, I was able to find it here https://docs.mongodb.com/manual/reference/connection-string/#:~:text=string%20of%20options.-,DNS%20Seed%20List%20Connection%20Format,-New%20in%20version). My st2.config:

[database]
host = mongodb+srv://<username>:<password>@<host>/<database>?retryWrites=true&w=majority
db_name = <database>
port = 27017
ssl = true

The host is the same when you click connect to your MongoDB Atlas cluster.

@yardengumai maybe set ssl to true in your st2.config?