rueckstiess / mtools

A collection of scripts to set up MongoDB test environments and parse and visualize MongoDB log files.
Apache License 2.0
1.89k stars 403 forks source link

Version detection is using global mongod not the configured one #866

Closed p-mongo closed 2 years ago

p-mongo commented 2 years ago

When I use --binarypath option, the binary I am specifying is not used by mlaunch for server version detection and it tries to obtain the version (only) from global mongod.

I don't know how to get this sort of output out of unmodified mlaunch, I added this patch:

serene2% git diff
diff --git a/mtools/mlaunch/mlaunch.py b/mtools/mlaunch/mlaunch.py
index f16b9d2..683c300 100755
--- a/mtools/mlaunch/mlaunch.py
+++ b/mtools/mlaunch/mlaunch.py
@@ -907,6 +907,7 @@ class MLaunchTool(BaseCmdLineTool):
             binary = os.path.join(self.args['binarypath'], binary)

         try:
+            import pdb;pdb.set_trace()
             out = check_mongo_server_output(binary, '--version')
         except Exception:
             return "0.0"
@@ -2116,6 +2117,7 @@ class MLaunchTool(BaseCmdLineTool):

     def _construct_mongod(self, dbpath, logpath, port, replset=None, extra=''):
         """Construct command line strings for mongod process."""
+        print(self.current_version)
         rs_param = ''
         if replset:
             rs_param = '--replSet %s' % replset

This prints the detected version when launching mongod.

On my system there isn't a mongod in PATH.

Expected behavior

The binary I referenced used for version detection.

Actual/current behavior

The global mongod is used, which doesn't exist on my system, version ends up being set to 0.0 which I think is a separate problem - this would cause mlaunch to assume behavior (e.g. with respect to tls/ssl options) that may be incorrect.


Running mlaunch --replicaset --name ruby-driver-rs --auth --username dev --password dev --dir /mnt/zram/mongodb/4.0-rs-auth --setParameter diagnosticDataCollectionEnabled=false --wiredTigerCacheSizeGB 0.25 --setParameter honorSystemUmask=true --networkMessageCompressors snappy,zlib --setParameter enableTestCommands=1 --filePermissions 0666 --binarypath /usr/local/m/versions/4.0 --port 27150 --verbose
> /home/m/.local/lib/python3.10/site-packages/mtools/mlaunch/mlaunch.py(911)getMongoDVersion()
-> out = check_mongo_server_output(binary, '--version')
(Pdb) c
Failed to launch mongod: [Errno 2] No such file or directory: 'mongod'
creating directory: /mnt/zram/mongodb/4.0-rs-auth/ruby-driver-rs/rs1/db
0.0
creating directory: /mnt/zram/mongodb/4.0-rs-auth/ruby-driver-rs/rs2/db
0.0
creating directory: /mnt/zram/mongodb/4.0-rs-auth/ruby-driver-rs/rs3/db
0.0
writing .mlaunch_startup file.
Generating keyfile: /mnt/zram/mongodb/4.0-rs-auth/keyfile
waiting for nodes to shutdown...
launching: "/usr/local/m/versions/4.0/mongod" --replSet ruby-driver-rs --dbpath "/mnt/zram/mongodb/4.0-rs-auth/ruby-driver-rs/rs1/db" --logpath "/mnt/zram/mongodb/4.0-rs-auth/ruby-driver-rs/rs1/mongod.log" --port 27150 --fork --auth --keyFile "/mnt/zram/mongodb/4.0-rs-auth/keyfile" --setParameter "diagnosticDataCollectionEnabled=false" --wiredTigerCacheSizeGB "0.25" --setParameter "honorSystemUmask=true" --networkMessageCompressors "snappy,zlib" --setParameter "enableTestCommands=1" --filePermissions "0666" 
launching: "/usr/local/m/versions/4.0/mongod" --replSet ruby-driver-rs --dbpath "/mnt/zram/mongodb/4.0-rs-auth/ruby-driver-rs/rs2/db" --logpath "/mnt/zram/mongodb/4.0-rs-auth/ruby-driver-rs/rs2/mongod.log" --port 27151 --fork --auth --keyFile "/mnt/zram/mongodb/4.0-rs-auth/keyfile" --setParameter "diagnosticDataCollectionEnabled=false" --wiredTigerCacheSizeGB "0.25" --setParameter "honorSystemUmask=true" --networkMessageCompressors "snappy,zlib" --setParameter "enableTestCommands=1" --filePermissions "0666" 
launching: "/usr/local/m/versions/4.0/mongod" --replSet ruby-driver-rs --dbpath "/mnt/zram/mongodb/4.0-rs-auth/ruby-driver-rs/rs3/db" --logpath "/mnt/zram/mongodb/4.0-rs-auth/ruby-driver-rs/rs3/mongod.log" --port 27152 --fork --auth --keyFile "/mnt/zram/mongodb/4.0-rs-auth/keyfile" --setParameter "diagnosticDataCollectionEnabled=false" --wiredTigerCacheSizeGB "0.25" --setParameter "honorSystemUmask=true" --networkMessageCompressors "snappy,zlib" --setParameter "enableTestCommands=1" --filePermissions "0666" 
waiting for nodes to start...
initializing replica set 'ruby-driver-rs' with configuration: {'_id': 'ruby-driver-rs', 'members': [{'_id': 0, 'host': 'localhost:27150'}, {'_id': 1, 'host': 'localhost:27151'}, {'_id': 2, 'host': 'localhost:27152'}]}
replica set 'ruby-driver-rs' initialized.
waiting for nodes to start...
waiting for primary to add a user.
added user dev on admin database
Username "dev", password "dev"
done.

Steps to reproduce the actual/current behavior

Run patched mlaunch printing detected version.

Environment

Software Version
mtools develop
Python 3.10
MongoDB server 4.0
Operating system Debian
p-mongo commented 2 years ago

Use of incorrect mongod causes mlaunch to fail when starting sharded clusters. This condition is responsible for setting csrs for 3.6+ servers:

        # add the 'csrs' parameter as default for MongoDB >= 3.3.0
        if (LooseVersion(self.current_version) >= LooseVersion("3.3.0") or
                LooseVersion(self.current_version) == LooseVersion("0.0.0")):
            self.args['csrs'] = True

Since self.current_version is erroneously set to 0.0, the csrs flag isn't set, and subsequently the following condition:

        # CSRS config servers (MongoDB >=3.1.0)
        if self.args['csrs']:
            config_string.append(self._construct_config(self.dir, nextport,
                                                        "configRepl", True))
        else:
            for name in config_names:
                self._construct_config(self.dir, nextport, name)
                config_string.append('%s:%i' % (self.args['hostname'],
                                                nextport))
                nextport += 1

... goes into else branch instead of if and produces a broken configdb parameter (host names only without the replica set name).

The else branch doesn't look right also because the value of _construct_config isn't used anywhere.

p-mongo commented 2 years ago

Workaround for sharded clusters on 3.6+: https://github.com/p-mongodb/mtools-legacy/commit/90b1bc19aed3729d13409193f829a0516e62a7c9