mrsarm / mongotail

Command line tool to log all MongoDB queries in a "tail"able way
https://pypi.org/project/mongotail/
GNU General Public License v3.0
192 stars 17 forks source link

Cannot get queries to log; only DROP and INSERT show in the log #32

Closed drauschenbach closed 3 years ago

drauschenbach commented 3 years ago

Is there special configuration required to get queries to log? I don't ever see any QUERY entries in the log, which is what I'm most interested in looking at. I can only see inserts and drops (the setup of my test case, but not my test case).

$ mongotail myserver/mydb -l 2
Profiling level set to level 2

$ mongotail myserver/mydb -f
2021-01-06 23:53:08.418 DROP      [mycoll] :
2021-01-06 23:53:08.559 INSERT    [mycoll] : 1 inserted.
2021-01-06 23:53:08.624 INSERT    [mycoll] : 1 inserted.

I'm using a Mongo 4.4.2 standalone server on Linux.

drauschenbach commented 3 years ago

Oh by the way my Mongo queries are aggregation pipelines. I don't know if that's a factor in why they're not getting logged.

mrsarm commented 3 years ago

Hey, sorry I never responded back, I was on vacations when you posted your question, so my reaction on that time was to answer once back but then I forgot to do so.

I have never tested Mongotail with Mongo 4.4, last version was 4.2, and it is common that among different versions of Mongo the log format change. Mongotail adapts the way it reads these logs depending of the engine version, I'll test Mongotail + MongoDB 4.4 soon to see if changes are needed.

Aggregation is supported since 2.0.x version, and I guess you are using the last 2.4.0 version, tough, could you post here what is the query? I don't need the schema nor the exact names of the fields in the query, you can change that, but if you can pass me the query it can help me to see if there is a particular problem with it.

Also, try execute mongotail DBNAME --verbose to see if it's logged with that option.

Thanks!

drauschenbach commented 3 years ago

I knew about the --verbose switch -- thanks.

Here's the commands that I can get to log into the mongod log, which I can't see through the more convenient mongotail:

{
  "command": {
    "aggregate": "mytable",
    "pipeline": [{
      "$match":{
        "$and":[{
          "a":{"$exists":true,"$ne":null}
        },{
          "a":2
        }]
      }
    },{
      "$project":{"_id":1,"a":1,"b":1,"c":1}
    }], ...
}
mrsarm commented 3 years ago

Nice, I'll give it a try soon, along with MongoDB 4.4. Thanks!

mrsarm commented 3 years ago

@drauschenbach , I confirmed the log of aggregation queries is not working, not sure why, must be a bug introduced in one of the latest versions, but not related with the DB used, actually I tested Mongotail against MongoDB 4.4 and worked without problems. I'll try find a slot this week to fix the issue, should be simple to fix.

mrsarm commented 3 years ago

Hey @drauschenbach , I think I fixed the issue, at least it works in my local env. Could you try the fix with the alpha version I released? You can upgrade your current version with the alpha release with:

$ pip3 install --upgrade https://github.com/mrsarm/mongotail/archive/refs/tags/2.4.1-alpha.1.zip

Replace pip3 with pip if your are using Python 2, but I would recommend to uninstall the Python 2 version and install the version for Python 3 instead.

mrsarm commented 3 years ago

Fixed and released in version 2.4.1