Performing Projection to return matched embedded documents

[fahimmahmoodmir]

I am having trouble in returning only matched embedded document. I am new to ODM concept and following is a document structure that is inserted in collection called Project:

{ 
    "_id" : ObjectId("59f889e46803fa3713454b5d"), 
    "projectName" : "usecase-updated", 
    "classes" : [
        {
            "_id" : ObjectId("59f9d7776803faea30b895dd"), 
            "className" : "OLA"
        }, 
        {
            "_id" : ObjectId("59f9d8ad6803fa4012b895df"), 
            "className" : "HELP"
        }, 
        {
            "_id" : ObjectId("59f9d9086803fa4112b895de"), 
            "className" : "DOC"
        }, 
        {
            "_id" : ObjectId("59f9d9186803fa4212b895de"), 
            "className" : "INVOC"
        }
    ]
}

Now lets suppose that fromProject collection, i wanted to return a Project with specific _id and within that selected Project document, i wanted to get only the matched embedded Classes documents (with a given _id). I tried creating this query like this:

$collection = $PHPMongoDBInstance->getCollection("Project"); //get the Project Collection
$cursor = $collection->findAsArray()->whereAnd(
            $collection->expression()->where('_id', new MongoId("59f889e46803fa3713454b5d")),
            $collection->expression()->whereElemMatch("classes", $collection->expression()->where("_id", 
            new MongoId("59f9d7776803faea30b895dd"))));

I was expecting the result would be something like this:

{ 
   "_id" : ObjectId("59f889e46803fa3713454b5d"), 
    "projectName" : "usecase-updated", 
    "classes" : [
          {
            "_id" : ObjectId("59f9d7776803faea30b895dd"), 
            "className" : "OLA"
          }
     ]
}

But its not like this. Thing is i am able to achieve the desired field level projection with native collection instance of mongo driver as follows:

$result = $collection->getMongoCollection("Project")
                                 ->findOne(array("_id" => new MongoId("59f889e46803fa3713454b5d")), 
                                                  array("classes" => array('$elemMatch' => array( "_id" => new MongoId("59f9d7776803faea30b895dd")))));

So i wonder if we have similar thing in the library where i can pass the fields of the results to return within the chaining of getCollection method.

[fahimmahmoodmir]

I tried using aggregation framework and comes up with following query:

$result = $collection->aggregate(array(
    array(
        '$match' => array(
            "_id" => new MongoId("59f889e46803fa3713454b5d")
        )
    ),
    array(
        '$project' => array(
            'classes' => array(
                '$filter' => array(
                    'input' => '$classes',
                    'as' => 'classItem',
                    'cond' => array(
                        '$eq' => array('$$classItem._id' => new MongoId("59f9d7776803faea30b895dd"))
                    )
                )

            )

        )
    )

));

its giving me following exception:

Sokil\\Mongo\\Exception\nMessage: Aggregate error: Unrecognized expression '$$classItem._id'\nFile:

is there any syntax error?

[sokil]

Hello.

ODM tries to map mongo's document to object. Any field of document in collection mapped to property of Sokil\Mongo\Document object, any embedded document mapped to Document property, also as field of embedded document.

If you do not configure your Document class to map embedded document to some Structure, it will be just array.

But you can do some tricks to get embedded document like object: https://github.com/sokil/php-mongo#embedded-documents, so in this way your Document became aggregate of another embedded documents.

In first example, when you query collection using $collection->findAsArray() with whereElemMatch, you get whole aggregate with all embedded documents inside, but only if this whole aggregat contain some fields in it's embedded documentys. Because you filtered by _id and embedded classes containing other id.

In second example, when you call native findOne, you pass second projection parameter: https://docs.mongodb.com/v3.2/reference/method/db.collection.findOne/.

<?php
$result = $collection->getMongoCollection("Project")->findOne(
    // query
    array(
        "_id" => new MongoId("59f889e46803fa3713454b5d")
    ), 
    // projection
    array(
        "classes" => array(
            '$elemMatch' => array(
                "_id" => new MongoId("59f9d7776803faea30b895dd")
            )
        )
    )
);

This code find document by _id and project some fields.

Find by id with bool projection:

> db.test.findOne({"_id": ObjectId("59f889e46803fa3713454b5d")}, {classes: 1})
{
    "_id" : ObjectId("59f889e46803fa3713454b5d"),
    "classes" : [
        {
            "_id" : ObjectId("59f9d7776803faea30b895dd"),
            "className" : "OLA"
        },
        {
            "_id" : ObjectId("59f9d8ad6803fa4012b895df"),
            "className" : "HELP"
        },
        {
            "_id" : ObjectId("59f9d9086803fa4112b895de"),
            "className" : "DOC"
        },
        {
            "_id" : ObjectId("59f9d9186803fa4212b895de"),
            "className" : "INVOC"
        }
    ]
}

Due to https://docs.mongodb.com/manual/tutorial/project-fields-from-query-results/#project-specific-array-elements-in-the-returned-array you can also project by some expressions:

Find by id and id of embedded document of "classes":

> db.test.find({"_id": ObjectId("59f889e46803fa3713454b5d")}, {"classes": {$elemMatch: {"_id" : ObjectId("59f9d9086803fa4112b895de")}}}).pretty()
{
    "_id" : ObjectId("59f889e46803fa3713454b5d"),
    "classes" : [
        {
            "_id" : ObjectId("59f9d9086803fa4112b895de"),
            "className" : "DOC"
        }
    ]
}

Currently $elemMatch not supported. You can only $slice and project specific fields. And this feature require to beimplemented.

[fahimmahmoodmir]

Thank you for the detailed answer. I wanted to move toward the mapping of ODM to a class so i will try it with the mapping functionality. Moreover as you mentioned that $elemMatch is not yet implemented so this mean this post can be labeled as enhancement for the development of $elemMatch.

One more update is that I tried with aggregation query and somehow i was able to retrieve desired results. I was making a mistake in the syntax of the aggregation query. Following is my updated aggregation query for retrieving matched embedded document (i am using $filter array aggregation function):

$collection = $PHPMongoDBInstance->getCollection("Project");
$result = $collection->aggregate(array(
    array(
        '$match' => array(
            "_id" => new MongoId('59f889e46803fa3713454b5d')
        )
    ),
    array(
        '$project' => array(
            'classes' => array(
                '$filter' => array(
                    'input' => '$classes',
                    'as' => 'class',
                    'cond' => array(
                        '$eq' => array('$$class._id', new MongoId("59faff836803fa3b1203d2e5"))
                    )
                )
            ),
            'projectName' => 1
        )
    )
));

the problem was in the syntax of the following expression: '$eq' => array('$$classItem._id' => new MongoId("59f9d7776803faea30b895dd"))

I learned that it shouldn't be a associative array rather each element in the expression should be a separate array item like this:

'$eq' => array('$$class._id', new MongoId("59faff836803fa3b1203d2e5"))

With this conclusion i am able to get the results correctly.

[sokil]

yes, this feature will be implemented. Thanks for interesting lib )

[sokil]

Implemented in v.1.22.2

Useage:

public function testElemMatch()
    {
        // create new document
        $this->collection
            ->createDocument(array(
                "_id"=> new \MongoId("59f889e46803fa3713454b5d"),
                "projectName" => "usecase-updated",
                "classes" => array(
                    array(
                        "_id" => new \MongoId("59f9d7776803faea30b895dd"),
                        "className" => "OLA"
                    ),
                    array(
                        "_id" => new \MongoId("59f9d8ad6803fa4012b895df"),
                        "className" => "HELP"
                    ),
                    array(
                        "_id" => new \MongoId("59f9d9086803fa4112b895de"),
                        "className" => "DOC"
                    ),
                    array(
                        "_id" => new \MongoId("59f9d9186803fa4212b895de"),
                        "className" => "INVOC"
                    )
                )
            ))
            ->save();
        // filter embedded documents
        $filteredClasses = $this->collection
            ->find()
            ->elemMatch(
                'classes',
                function(Expression $e) {
                    $e->where('_id', new \MongoId("59f9d9186803fa4212b895de"));
                }
            )
            ->one()
            ->classes;
        $this->assertEquals(
            'INVOC',
            $filteredClasses[0]['className']
        );
    }