kaizendorks / pymongo_inmemory

A mongo mocking library with an ephemeral MongoDB running in memory.
MIT License
38 stars 12 forks source link

pymongo_inmemory doesn't work in docker container alpine #54

Open trolliama opened 2 years ago

trolliama commented 2 years ago

Describe the bug I am having troubles in running my tests inside my docker alpine container. I installed the pymongo_inmemory properly and started the tests and this was returned to me:

FileNotFoundError: [Errno 2] No such file or directory: '/usr/src/venv/lib/python3.9/site-packages/pymongo_inmemory/downloader/../.cache/extract/mongodb-linux-x86_64-4.0.23-tgz/mongodb-linux-x86_64-4.0.23/bin/mongod'

I verified it and the mongod binary is there. So i tried to run the subprocess.Popen to just ls the folders and indeed the error appeared. With a little bit of research I've found that using the shell=True option in the Popen() class like this: subprocess.Popen(["ls path/to/bin"], shell=True) worked, but when I tried to run the mongod with this command it gave me this error:

/bin/sh: /usr/src/venv/lib/python3.9/site-packages/pymongo_inmemory/downloader/../.cache/extract/mongodb-linux-x86_64-4.0.23-tgz/mongodb-linux-x86_64-4.0.23/bin/mongod: not found

Obs: In my machine works if I don't use the docker container

To Reproduce Steps to reproduce the behavior:

  1. Create tests (I used pytest)
  2. Create a dockerfile (I used the alpine):
    
    FROM python:3.9-alpine AS builder

RUN pip install --user pipenv

WORKDIR /usr/src

RUN apk update RUN apk add postgresql-dev gcc python3-dev musl-dev RUN apk add postgresql

ADD Pipfile.lock Pipfile /usr/src/

RUN /root/.local/bin/pipenv lock --dev -r > requirements.txt

RUN python -m venv venv RUN venv/bin/pip install -r requirements.txt

FROM python:3.9-alpine AS runtime

RUN mkdir -v /usr/src/venv COPY --from=builder /usr/src/venv /usr/src/venv

RUN apk update RUN apk add gcc postgresql

WORKDIR /usr/src

RUN ln -snf /usr/share/zoneinfo/America/Sao_Paulo \ /etc/localtime && \ echo "America/Sao_Paulo" > /etc/timezone

COPY ./src/app ./app COPY ./src/tests ./tests COPY ./src/main.py ./

Copy the crontab configuration file

COPY ./cronjobs/execute_updater.dev ./cronjobs/execute_updater RUN /usr/bin/crontab ./cronjobs/execute_updater RUN touch /var/log/cron.log

path the python venv bin - tru not to change since its needed in the ci pipeline

CMD /usr/sbin/crond -f -l 8 & tail -f /var/log/cron.log

CMD ["python", "-m", "http.server", "8080"]



3. Run the container and then run the tests (I ran inside the container itself)
4. The error appears

**Expected behavior**
It's expected that the tests pass

**Screenshots**
![image](https://user-images.githubusercontent.com/21108976/162024186-53ec646a-f806-4ca1-bacc-f7ed14d065fb.png)

**Context:**
 - Version of pymongo_inmemory [~=0.2.5]
 - python3.9-alpine docker image
trolliama commented 2 years ago

I tried to use the ubuntu:20.04 image and it worked as expected, but would be great if was possible to use the alpine since its lightweight

ekarademir commented 2 years ago

Hi @trolliama,

How do you set up the version in your docker image? The library itself doesn't do any compilation. It basically downloads a suitable version. See https://github.com/kaizendorks/pymongo_inmemory#available-mongodb-versions

It looks like Apline linux is not supported by MongoDB itself. Probably then it's falling back to generic version. That being said, FileNotFound error looks fishy. I'll publisha fix recently merged to master. Maybe that would fix this problem too.

ekarademir commented 2 years ago

Maybe related to #52 and maybe closed by #53

sebastus commented 1 year ago

Our test engine is configured by default to run tests in parallel. As I understand it, pymongo_inmemory downloads mongo on first access. If a test running in parallel with that first request attempts to access the database, it's no longer the first access, so rather than trying to download mongo, it attempts to run the query. But mongo isn't installed yet.

Is there a way to pre-load mongo so that on first access, the query can just go ahead and run?

ekarademir commented 1 year ago

Hey @sebastus this is a very interesting use case. To summarise what I understood

I am in the middle of refactoring how this download and running business works. I will try to incorporate that as well. But in the mean time I think there is a workaround.

With #65 you should be able to point to a "local" mongod instance. So you can pre-download the binary and point to it for the tests.

That being said, I think there is still not a good way to provide that path. So that's something I should add to that refactor.