TypeError: expected str, bytes or os.PathLike object, not int on self._mongod.start()

stevenmz commented 7 months ago

Describe the bug subprocess fails to load mongod on os x

To Reproduce Steps to reproduce the behavior:

import pytest
from pymongo_inmemory import MongoClient

if not os.path.exists("mongo_dl_cache/download") or not os.path.exists("mongo_dl_cache/extract"):
    os.makedirs("mongo_dl_cache/download")
    os.makedirs("mongo_dl_cache/extract")
    os.makedirs("mongo_dl_cache/data")

os.environ["PYMONGOIM__DOWNLOAD_FOLDER"]="mongo_dl_cache/download"
os.environ["PYMONGOIM__MONGOD_PORT"]= "27017"
os.environ["PYMONGOIM__EXTRACT_FOLDER"]="mongo_dl_cache/extract"
os.environ["PYMONGOIM__MONGOD_DATA_FOLDER"]="mongo_dl_cache/data"
os.environ["PYMONGOIM__DOWNLOAD_URL"]="https://fastdl.mongodb.org/osx/mongodb-macos-x86_64-6.0.12.tgz"

client = MongoClient()

Expected behavior MongoClient should connect.

Logs If applicable, add any kind of collected logs to help us understand the problem.

../../opt/anaconda3/envs/py311/lib/python3.11/site-packages/pymongo_inmemory/_pim.py:15: in init self._mongod.start() ../../opt/anaconda3/envs/py311/lib/python3.11/site-packages/pymongo_inmemory/mongod.py:142: in start self._proc = subprocess.Popen(boot_command) ../../opt/anaconda3/envs/py311/lib/python3.11/subprocess.py:1026: in init self._execute_child(args, executable, preexec_fn, close_fds, self = <Popen: returncode: None args: ['mongo_dl_cache/extract/c42eeac2efb615d0fe85...>

args = ['mongo_dl_cache/extract/c42eeac2efb615d0fe8562483f94326dcfeeb5a97cdeb3b4263c92a8ff27d04e/mongodb-macos-x86_64-6.0.12/bin/mongod', '--dbpath', 'mongo_dl_cache/data', '--logpath', 'mongo_dl_cache/data/mongod.log', '--port', ...] executable = b'mongo_dl_cache/extract/c42eeac2efb615d0fe8562483f94326dcfeeb5a97cdeb3b4263c92a8ff27d04e/mongodb-macos-x86_64-6.0.12/bin/mongod' preexec_fn = None, close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None, creationflags = 0 shell = False, p2cread = -1, p2cwrite = -1, c2pread = -1, c2pwrite = -1, errread = -1, errwrite = -1 restore_signals = True, gid = None, gids

= None, uid = None, umask = -1, start_new_session = False, process_group = -1

Offending line: self.pid = _fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, process_group, gid, gids, uid, umask, preexec_fn, _USE_VFORK) E TypeError: expected str, bytes or os.PathLike object, not int

Context:

OS: OS X Ventura 13.5.1 Intel based
Version of pymongo_inmemory 0.4.0
Version of mongo you are downloading [e.g. 4.0] I have tried 6.0.12 and the latest 7.0.4

ekarademir commented 6 months ago

Thank you so much for the report @stevenmz I'll investigate as soon as possible.

ammar-almuwallad commented 6 months ago

I'm having the same problem.

Issue Description

When attempting to run python3 run_mongo.py, encountered the following error:

Traceback (most recent call last):
  File "/home/almowld/Desktop/penny/official/penny/webhooks/sendgrid/set_mongo.py", line 6, in <module>
    client = MongoClient()  # No need to provide host
  File "/home/almowld/.cache/pypoetry/virtualenvs/app-aYfvlb12-py3.10/lib/python3.10/site-packages/pymongo_inmemory/_pim.py", line 15, in __init__
    self._mongod.start()
  File "/home/almowld/.cache/pypoetry/virtualenvs/app-aYfvlb12-py3.10/lib/python3.10/site-packages/pymongo_inmemory/mongod.py", line 142, in start
    self._proc = subprocess.Popen(boot_command)
  File "/usr/lib/python3.10/subprocess.py", line 971, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.10/subprocess.py", line 1796, in _execute_child
    self.pid = _posixsubprocess.fork_exec(
TypeError: expected str, bytes or os.PathLike object, not int

Steps to Reproduce

Modify setup.cfg with the following configurations:

[pymongo_inmemory]
mongod_port = 28000
mongo_version=7.0
operating_system=ubuntu
os_version=22

Contents of run_mongo.py:

from pymongo_inmemory import MongoClient

client = MongoClient()  # No need to provide host
db = client['testdb']
collection = db['test-collection']
# etc., etc.
client.close()

Run python3 run_mongo.py

Expected Behavior

The script should run successfully and create an in-memory mongo instance

Additional Information

Operating System: Ubuntu 22.04.3 LTS
Python Version: 3.10.12
pymongo_inmemory Version: 0.4.0

NOTE: no matter what the settings are in setup.cfg i keep getting the same error. I also tried reverting back to 0.3.1 and it had the same error so I reverted way back to 0.2.11 and it works.

Thank you for your work

ercaronte commented 4 months ago

I created a PR that should fix this issue. There is an additional PR (#101) that would make this good library useful with MongoDB 7+. While waiting for the maintainers to handle them, I solved the bug by subclassing the MongoClient and Context classes as follows:

from pymongo_inmemory import Mongod, MongoClient
from pymongo_inmemory.context import Context, conf

class MyContext(Context):
    def __init__(self, os_name=None, version=None, os_ver=None, ignore_cache=False) -> None:
        super(InMemoryContext, self).__init__(os_name, version, os_ver, ignore_cache)
        self.storage_engine = conf("storage_engine", "ephemeralForTest")
        self.mongod_port = str(self.mongod_port)

class MyMongoClient(MongoClient):
    def __init__(self, host=None, port=None, **kwargs):
        self._pim_context: Context = MyContext()
        if port is not None:
            self._pim_context.mongod_port = port
        if host is not None:
            self._pim_context.mongo_client_host = host
        self._mongod = Mongod(self._pim_context)
        self._mongod.start()
        super(MongoClient, self).__init__(self._mongod.connection_string, **kwargs)

ercaronte commented 4 months ago

@ekarademir Thank you for latest release - 0.4.1 - which solves this issue. I guess we can close this.

kaizendorks / pymongo_inmemory