Getting OperationFailure error when trying to find using the $near operator

[santigandolfo]

Bug

I'm getting a pymongo.errors.OperationFailure: $geoNear, $near, and $nearSphere are not allowed in this context error when trying to find using the $near operator.

Current Behavior

If I have defined the following classes:

class GeoJSON(BaseModel):
    type: str = "Point"
    coordinates: tuple[float, float]

class Instruction(Model):
    name: str
    geo_json: GeoJSON

And execute this query:

engine.find(Instruction, {
            "geo_json": {
                "$near": {
                    "$geometry": {
                        "type": "Point",
                        "coordinates": [
                            -65.49250524795319,
                            -24.732399067642003
                        ]
                    },
                    "$maxDistance": 500
                }
            }
        })

The following exception is raised:

pymongo.errors.OperationFailure: $geoNear, $near, and $nearSphere are not allowed in this context, full error: {'ok': 0.0, 'errmsg': '$geoNear, $near, and $nearSphere are not allowed in this context', 'code': 2, 'codeName': 'BadValue', '$clusterTime': {'clusterTime': Timestamp(1676945612, 1), 'signature': {'hash': b'\x8b2\xd80ipA\x82,R\x1bN\xa7a\xad\x7f\xcb\xaeI\x84', 'keyId': 7155114180230512642}}, 'operationTime': Timestamp(1676945612, 1)}

Expected behavior

The query should return a list of Instructions.

Environment

• ODMantic version: 0.9.2
• MongoDB version: 5.0.14 (Mongo Atlas)
• Pydantic infos (output of python -c "import pydantic.utils; print(pydantic.utils.version_info())):

             pydantic version: 1.10.5
            pydantic compiled: True
                 install path: /usr/local/lib/python3.11/site-packages/pydantic
               python version: 3.11.1 (main, Dec 23 2022, 09:39:26) [Clang 14.0.0 (clang-1400.0.29.202)]
                     platform: macOS-13.1-x86_64-i386-64bit
     optional deps. installed: ['dotenv', 'email-validator', 'typing-extensions']

• Version of additional modules (if relevant):
  • motor: 3.1.1
• Python version: Docker python:3.10-buster

Additional context

In my DB i have a collection "instruction" with objects like:

{
  "_id": { "$oid": "63f4162924a0a5952a0904ef" },
  "name": "Some name",
  "geo_json":
    {
      "type": "Point",
      "coordinates":
        [
          { "$numberDouble": "-65.49250524795319" },
          { "$numberDouble": "-24.732399067642003" }
        ]
    }
}

And I've added the index:

{ "geo_json": "2dsphere" }

When I try to use the Find on Mongo Atlas, if I add this on the search field:

{
    "geo_json": {
        "$near": {
            "$geometry": {
                "type": "Point",
                "coordinates": [
                    -65.49250524795319,
                    -24.732399067642003
                ]
            },
            "$maxDistance": 500
        }
    }
}

It returns the object correctly.

Also, if I change the python code to this:

        collection = engine.get_collection(Instruction)
        response = collection.find({
            "geo_json": {
                "$near": {
                    "$geometry": {
                        "type": "Point",
                        "coordinates": [
                            -65.49250524795319,
                            -24.732399067642003
                        ]
                    },
                    "$maxDistance": 500
                }
            }
        })
        instructions = await response.to_list(length=None)
        return [Instruction.parse_doc(instruction) for instruction in instructions]

Then the search works.

Is the find functionality not working fine or I'm just using it wrong?

[santigandolfo]

After reading the source code and mongo's docs (and with the help of ChatGPT) I've fould the following:

The reason why when I do the engine.find it fails is because the find method uses internally an aggregation pipeline where the find queries are set inside a $match stage. And because "You cannot use $near or $nearSphere in $match queries as part of the aggregation pipeline" (from the MongoDB docs) then that's why the query fails (It would be nice if the OperationFailure error said this insead of the more generic "...are not allowed in this context"). The docs suggest to: " Use $geoNear stage instead of the $match stage.

Use $geoWithin query operator with $center or $centerSphere in the $match stage. "

Is there a way to add the geoNear stage to the _prepare_find_pipeline method? I don't know what the best way to do it, if the queries should go in the queries parameter or in a separate one (or if there should be a different find method that allows this kind of queries).