search

kaizendorks / pymongo_inmemory

A mongo mocking library with an ephemeral MongoDB running in memory.
MIT License
40 stars 13 forks source link

Error in initialization of client with macOS #56

Closed ChingEn-Lin closed 2 years ago

ChingEn-Lin commented 2 years ago

ValueError: invalid literal for int() with base 10: b'2022-04-21T17:34:12.398+0200 I CONTROL [js] machdep.cpu.extfeatures unavailable\n0\n' when trying to instantiate client with client = MongoClient()

To Reproduce

from pymongo_inmemory import MongoClient
client = MongoClient()

gives the error ValueError: invalid literal for int() with base 10: b'2022-04-21T17:34:12.398+0200 I CONTROL [js] machdep.cpu.extfeatures unavailable\n0\n'

Logs

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "./lib/python3.7/site-packages/pymongo_inmemory/_pim.py", line 9, in __init__
    self._mongod.start()
  File "./python3.7/site-packages/pymongo_inmemory/mongod.py", line 101, in start
    while not self.is_healthy:./python3.7/site-packages/pymongo_inmemory/mongod.py", line 140, in is_healthy
    if int(uptime) > 0:
ValueError: invalid literal for int() with base 10: b'2022-04-21T17:34:12.398+0200 I CONTROL  [js] machdep.cpu.extfeatures unavailable\n0\n'

Context:

ekarademir commented 2 years ago

Hi @ChingEn-Lin I'm going to look into this.

ekarademir commented 2 years ago

Hi @ChingEn-Lin could you please post the output of 

python -c "import platform; print(platform.uname())"

I don't have an M1 machine, so I need your help debugging this. The version of MongoDB is determined by the output of a similar Python function. I wonder if the output has changed for M1 chips.

ChingEn-Lin commented 2 years ago

Dear Ertugrul,

Thank you for looking into this. The output is as follow:

import platform; print(platform.uname()) uname_result(system='Darwin', node=‘xxxx-MacBook-Pro.local', release='21.4.0', version='Darwin Kernel Version 21.4.0: Fri Mar 18 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000', machine='x86_64', processor='i386’)

Regards, Lin

Ertugrul Karademir @.**@.>> 於 2022年5月1日 下午1:08 寫道：

python -c "import platform; print(platform.uname())"

ekarademir commented 2 years ago

Hi @ChingEn-Lin

Thank you for the output. I think it's not about selecting the version, but how the code decides if MongoDB server is running or not. So the debugging will be harder. Below steps is basically what pymongo_inmemory does. The aim is to see if the commands it's using are producing the output that it's expecting. When you have time could you do the following please:

ChingEn-Lin commented 2 years ago

Dear Ertugrul, I followed the instructions, The output is as follow

db.serverStatus().uptime 68

I believe this is sth that is expected?

Regards, Lin

Ertugrul Karademir @.**@.>> 於 2022年5月2日 上午9:10 寫道：

Hi @ChingEn-Linhttps://github.com/ChingEn-Lin

Thank you for the output. I think it's not about selecting the version, but how the code decides if MongoDB server is running or not. So the debugging will be harder. Below steps is basically what pymongo_inmemory does. The aim is to see if the commands it's using are producing the output that it's expecting. When you have time could you do the following please:

mkdir data ./mongod --storageEngine ephemeralForTest --dbpath ./data

./mongo

db.serverStatus().uptime

— Reply to this email directly, view it on GitHubhttps://github.com/kaizendorks/pymongo_inmemory/issues/56#issuecomment-1114560621, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARWAPJ6KWXW25GLJQP3Y7KDVH55VZANCNFSM5T7UY2QA. You are receiving this because you were mentioned.Message ID: @.***>

ekarademir commented 2 years ago

Indeed, that is the expected output. However this might be related to your particular case: https://jira.mongodb.org/browse/SERVER-56416

Apparently MongoDB server outputs an extra line of logging with CPU features summary.

It looks like it has been fixed in 4.4.10 and 5.0.4, I'm guessing also in other minor versions. I'm going to publish a new version of pymongo_inmemory which will include option to download these versions. Could you try again after the next release?

If you are tied to this version, I will try to blindly create a patch and I might need your help testing it.

ChingEn-Lin commented 2 years ago

Yes I’ll be gladly to try again after the next release, as well as to help test it.

Ertugrul Karademir @.**@.>> 於 2022年5月3日 上午9:31 寫道：

Indeed, that is the expected output. However this might be related to your particular case: https://jira.mongodb.org/browse/SERVER-56416

Apparently MongoDB server outputs an extra line of logging with CPU features summary.

It looks like it has been fixed in 4.4.10 and 5.0.4, I'm guessing also in other minor versions. I'm going to publish a new version of pymongo_inmemory which will include option to download these versions. Could you try again after the next release?

If you are tied to this version, I will try to blindly create a patch and I might need your help testing it.

— Reply to this email directly, view it on GitHubhttps://github.com/kaizendorks/pymongo_inmemory/issues/56#issuecomment-1115813594, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARWAPJ2HNBQGDTHUQGPWVPDVIDI3XANCNFSM5T7UY2QA. You are receiving this because you were mentioned.Message ID: @.***>

ekarademir commented 2 years ago

I pushed a new release few ours ago. Could you try if the problem is resolved with 4.4 or 5.0?

ChingEn-Lin commented 2 years ago

I am using MongoDB shell version v5.0.7 and pymongo-inmemory version: 0.2.7, still get the same

 Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "./lib/python3.7/site-packages/pymongo_inmemory/_pim.py", line 9, in __init__
    self._mongod.start()
  File "./lib/python3.7/site-packages/pymongo_inmemory/mongod.py", line 112, in start
    while not self.is_healthy:
  File "./lib/python3.7/site-packages/pymongo_inmemory/mongod.py", line 154, in is_healthy
    if int(uptime) > 0:
ValueError: invalid literal for int() with base 10: b'2022-05-03T15:25:41.744+0200 I CONTROL  [js] machdep.cpu.extfeatures unavailable\n0\n'

for the code

ekarademir commented 2 years ago

Ok, thanks for trying out. I'll try creating a patch on a branch then I might ask you to test it. It will take me few days :)

ekarademir commented 2 years ago

Hi @ChingEn-Lin ,

As mentioned above, I created a fix for this issue. However now we need to see if it solves your issue.

Could you do the following please?

  1. Reproduce the issue, or, make sure the issue is still there. For this you'd need at least poetry installed on the system. Or, you can install the only runtime dependency, pymongo via pip. The choice is yours.

    git clone https://github.com/kaizendorks/pymongo_inmemory.git
cd pymongo_inmemory
# Only if you have poetry
poetry install --no-root
poetry shell
# If you don't have poetry
pip install pymongo

    Then,

    python -m pymongo_inmemory.mongod

    Or you can create the same test file you've mentioned in this PR and see if the problem is still there.

  2. Now we observed that the problem is there, we checkout the fix and run the code again, in the same repository directory

    git checkout Issue56

    and run the same test and see if the problem is still there.

ChingEn-Lin commented 2 years ago

Hello, After the testing, I had a weird finding. This

from pymongo_inmemory import MongoClient client = MongoClient() actually works if I run it under the repo of pymongo_inmemory, regardless of the fix branch or not.

Ertugrul Karademir @.**@.>> 於 2022年5月9日 下午3:11 寫道：

Hi @ChingEn-Linhttps://github.com/ChingEn-Lin ,

As mentioned above, I created a fix for this issue. However now we need to see if it solves your issue.

Could you do the following please?

  1. Reproduce the issue, or, make sure the issue is still there. For this you'd need at least poetryhttps://python-poetry.org/ installed on the system. Or, you can install the only runtime dependency, pymongo via pip. The choice is yours.

git clone https://github.com/kaizendorks/pymongo_inmemory.git cd pymongo_inmemory

Only if you have poetry

poetry install --no-root poetry shell

If you don't have poetry

pip install pymongo

Then,

python -m pymongo_inmemory.mongod

Or you can create the same test file you've mentioned in this PR and see if the problem is still there.

  1. Now we observed that the problem is there, we checkout the fix and run the code again, in the same repository directory

git checkout Issue56

and run the same test and see if the problem is still there.

— Reply to this email directly, view it on GitHubhttps://github.com/kaizendorks/pymongo_inmemory/issues/56#issuecomment-1121083161, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARWAPJ3UL6LETLOCO5YGM63VJEFIPANCNFSM5T7UY2QA. You are receiving this because you were mentioned.Message ID: @.***>

ekarademir commented 2 years ago

Ok. I'll create a new version anyway, because I think similar problems arose in Windows as well. After that you can update the version and see if the problem is still there.

ekarademir commented 2 years ago

Hi @ChingEn-Lin I published a new version. Could you update the version you are using and try to see if your problem is solved?

ChingEn-Lin commented 2 years ago

Hello, the problem is solved, thank you very much!

ekarademir commented 2 years ago

Great to hear!