Closed ajdavis closed 1 year ago
Note we should also address the stdout/stderr issues described in https://github.com/10gen/mongo-orchestration/issues/272 when fixing this issue.
This issue is fixed. The current behavior is to raise an error immediately. For example when passing the "invalidOption" option:
$ python3 ~/launch.py repl single
{'members': [{'procParams': {'bind_ip': '127.0.0.1,::1',
'invalidOption': True,
'ipv6': True,
'logappend': True,
'port': 27017,
'setParameter': {'enableTestCommands': 1}}}]}
Traceback (most recent call last):
File "/Users/shane/launch.py", line 333, in <module>
cluster.start()
File "/Users/shane/launch.py", line 209, in start
return self._init_from_response(self._make_post_request())
File "/Users/shane/launch.py", line 99, in _make_post_request
raise RuntimeError(
RuntimeError: Error sending POST to cluster: Traceback (most recent call last):
File "/Users/shane/git/mongo-orchestration/mongo_orchestration/servers.py", line 338, in start
self.proc, self.hostname = process.mprocess(
File "/Users/shane/git/mongo-orchestration/mongo_orchestration/process.py", line 233, in mprocess
if wait_for(proc, port, timeout):
File "/Users/shane/git/mongo-orchestration/mongo_orchestration/process.py", line 159, in wait_for
raise OSError("Process started, but died immediately")
OSError: Process started, but died immediately
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/shane/git/mongo-orchestration/mongo_orchestration/apps/__init__.py", line 66, in wrap
return f(*arg, **kwd)
File "/Users/shane/git/mongo-orchestration/mongo_orchestration/apps/replica_sets.py", line 80, in rs_create
result = _rs_create(data)
File "/Users/shane/git/mongo-orchestration/mongo_orchestration/apps/replica_sets.py", line 37, in _rs_create
rs_id = ReplicaSets().create(params)
File "/Users/shane/git/mongo-orchestration/mongo_orchestration/replica_sets.py", line 648, in create
repl = ReplicaSet(rs_params)
File "/Users/shane/git/mongo-orchestration/mongo_orchestration/replica_sets.py", line 75, in __init__
config_members = [f.result() for f in futures]
File "/Users/shane/git/mongo-orchestration/mongo_orchestration/replica_sets.py", line 75, in <listcomp>
config_members = [f.result() for f in futures]
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/_base.py", line 458, in result
return self.__get_result()
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
raise self._exception
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/Users/shane/git/mongo-orchestration/mongo_orchestration/replica_sets.py", line 325, in member_create
server_id = self._servers.create(
File "/Users/shane/git/mongo-orchestration/mongo_orchestration/servers.py", line 522, in create
server.start(timeout)
File "/Users/shane/git/mongo-orchestration/mongo_orchestration/servers.py", line 364, in start
reraise(TimeoutError,
File "/Users/shane/git/mongo-orchestration/mongo_orchestration/compat.py", line 21, in reraise
raise exctype(str(value)).with_traceback(trace)
mongo_orchestration.errors.TimeoutError: Could not start Server. Please check the mongo-orchestration log in /Users/shane/mongo-orchestration/server.log for more details.
And the server log contains the error:
2023-03-14 14:33:11,910 [DEBUG] mongo_orchestration.apps:63 - rs_create((), {})
2023-03-14 14:33:11,910 [DEBUG] mongo_orchestration.apps.replica_sets:77 - rs_create()
2023-03-14 14:33:11,910 [DEBUG] mongo_orchestration.servers:167 - Server.__init__(mongod, {'replSet': 'eb8cc0f8-061f-48a1-b23d-1ce0a31c2a8f', 'logappend': True, 'ipv6': True, 'bind_ip': '127.0.0.1,::1', 'invalidOption': True, 'setParameter': {'enableTestCommands': 1}, 'port': 27017}, {}, None, None, None)
2023-03-14 14:33:11,910 [DEBUG] mongo_orchestration.servers:66 - Creating log file for mongod: /var/folders/88/rvpt81v95lxcw1t9_7089vkr0000gp/T/mongod-kjr_sj_c/mongod.log
2023-03-14 14:33:11,910 [DEBUG] mongo_orchestration.servers:223 - ('mongod', '--version')
2023-03-14 14:33:12,085 [DEBUG] mongo_orchestration.process:203 - mprocess(name='mongod', config_path='/var/folders/88/rvpt81v95lxcw1t9_7089vkr0000gp/T/mongo-59ur1_s0', port=27017, timeout=300)
2023-03-14 14:33:12,086 [DEBUG] mongo_orchestration.process:217 - execute process: mongod --config /var/folders/88/rvpt81v95lxcw1t9_7089vkr0000gp/T/mongo-59ur1_s0 --port 27017
2023-03-14 14:33:12,088 [DEBUG] mongo_orchestration.process:153 - wait for 27017
Error parsing INI config file: unrecognised option 'invalidOption'
try 'mongod --help' for more information
2023-03-14 14:33:12,304 [DEBUG] mongo_orchestration.process:158 - process is not alive
2023-03-14 14:33:12,304 [ERROR] mongo_orchestration.servers:362 - Could not start Server
Traceback (most recent call last):
File "/Users/shane/git/mongo-orchestration/mongo_orchestration/servers.py", line 338, in start
self.proc, self.hostname = process.mprocess(
File "/Users/shane/git/mongo-orchestration/mongo_orchestration/process.py", line 233, in mprocess
if wait_for(proc, port, timeout):
File "/Users/shane/git/mongo-orchestration/mongo_orchestration/process.py", line 159, in wait_for
raise OSError("Process started, but died immediately")
OSError: Process started, but died immediately
....
This was fixed by #280
If you pass an unrecognized config option to a mongod or mongos, or if disk is already full when they start, or various other misconfigurations, the mongod or mongos processes that Orchestration starts will exit quickly with nonzero code.
Orchestration should detect this, clean up, and return a useful error to the caller, rather than waiting 120 seconds to decide that starting the cluster has generically "timed out".