CPU runaway using `-f` option

[hedefalk]

I get around 150% CPU running the docker image on Mac with level 2 logging. I think it worked better the other day so maybe something that comes after a while. Hard to debug for me… Will try to use the python version natively instead now…

[hedefalk]

Follow-up:

I installed via easy_install instead but still got 150% CPU on my Docker process in OSX.

So it turns out it's the MONGO process that get's heavy CPU usage only when tailing it with this tool. (I'm running this Mongo within Docker too)

Still don't know how to debug this well…

[mrsarm]

Hello @hedefalk ,

Yes, you are alright, some times when you execute the Mongotail with the -f option, it take a lot of CPU consumption, I had the same issue with two different versions of Linux, but I didn't have time to profile the error to find where is the problem. I think the problem comes from the PyMongo driver, I'll try to profile Mongotail soon.

[mrsarm]

@hedefalk,

As I said, I experimented the same issue in the past. Now I'm trying to reproduce again but tailing with the -f option is working Ok, maybe it was solved n the latest PyMongo driver. Which version are you using? Did you try with the latest 3.4.0 version?

My test environment:

• PyMongo 3.4.0
• Mongotail 2.1.1
• MongoDB 3.2.1
• Ubuntu 14.04 x64
• Python 2.7 and 3.4

[hedefalk]

MongoDB: 3.2.12 Python: 2.7.12

PyMongo, not a clue?

OSX 10.12.4. But MongoDB running from official docker image which is debian:jessie I think. I get this behaviour running mongotail as docker image too so should be pretty non-system dependent?

[mrsarm]

To se the mongotail version:

mongotail --version

To see the PyMongo version:

python -c 'import pymongo; print pymongo.__version__'

Both versions are not specified in the Dockerfile (should be fixed), so depends of the day you run the first time the Docker container the versions of PyMongo and Mongotail are installed. Running outside docker, execute first sudo pip install --upgrade pymongo or sudo easy_install --upgrade pymongo to update the library to the latest version. Same for Mongotail.

[mrsarm]

@hedefalk , I found when the problem happens: it just occurs when you are connecting with local databases. I couldn't reproduce the bug before because I was trying with a remote server, after tried with two different local servers, the problem happened again.

Profiling mongotail I found that the problem looks like a driver issue, so I reported the bug to the PyMongo project: PYTHON-1276.

[hedefalk]

@mrsarm Cool!

Sorry for not getting back, I seem to have missed this. Here's just for completeness:

~ ❯❯❯ mongotail --version
mongotail 2.1.1 <https://github.com/mrsarm/mongotail>
~ ❯❯❯ python -c 'import pymongo; print pymongo.__version__'
3.4.0

[mrsarm]

Patch from @ShaneHarvey works Ok! Please @hedefalk test it. Released in Mongotail 2.1.2

[hedefalk]

@mrsarm Awesome!

Python noob question, is this published in any of the channels I've used?

~ ❯❯❯ sudo easy_install mongotail
Searching for mongotail
Best match: mongotail 2.1.1
Processing mongotail-2.1.1-py2.7.egg
mongotail 2.1.1 is already the active version in easy-install.pth
Installing mongotail script to /usr/local/bin

Using /usr/local/lib/python2.7/site-packages/mongotail-2.1.1-py2.7.egg
Processing dependencies for mongotail
Finished processing dependencies for mongotail

I've only used this and docker…

[mrsarm]

Try this:

~ ❯❯❯ sudo easy_install --upgrade mongotail