Calling $regex against a property that does not exist on the collection evaluates it to "True"

[john-yick]

Prerequisites

• [X] I have written a descriptive issue title
• [X] I have searched existing issues to ensure the bug has not already been reported

Mongoose version

6.5.2

Node.js version

14.18.2

MongoDB server version

5.0.10

Description

When querying a collection through mongoose using a $regex command if the field does not exist, it will treat it as "true" so that in the case of an $or :[$regex:{}] that block would always be treated as "true"

let conditions = {
            $or: [
          {
            _id: entity._id.toString(),
          },
             path: {
                $regex: `${entity._id.toString()}`,
            },
]}

In the above condition, the _id would never need to be matched if the "path" property did not exist on the collection.

This didn't used to work as expected on mongoose versions 5.x.x and also through Mongo Compass.

Steps to Reproduce

Create a new collection with properties

• firstName
• lastName

Add some data

send through the query:

let conditions = {
            $or: [
          {
            firstName: 'i do not exist',
          },
             i_do_not_exist: {
                $regex: `randomValue`,
            },
]}

When doing so it would return back all the rows of data from the collection

Expected Behavior

if a property does not exist on the collection for the $regex, it should just evaluate it to "False" like Mongo Compass and previous versions of Mongoose.

[mssodhi]

Unfortunately this is the expected behavior. As of Mongoose 6.0.10, we brought back the strictQuery option. However, strictQuery is tied to strict by default. This means that, by default, Mongoose will filter out query filter properties that are not in the schema. https://mongoosejs.com/docs/migrating_to_6.html#strictquery-is-removed-and-replaced-by-strict

Fortunately, you can disable it per query like this:

await UserModel.find({
  $or: [{ email: 'a@b.com' }, { email_path_dne: { $regex: 'abcd' } }],
}, null, { strictQuery: false });

or globally like this mongoose.set('strictQuery', false);

[john-yick]

@mssodhi So actually if that was the case, shouldn't the $regex evaluate to "False" instead of "True" ? Because in such a situation of the $or or $and operands we would want Mongoose to ignore the property if it was not part of the schema.

For your example it would return all users, since the field "abcd" does not exist.

[mssodhi]

@john-yick Looks like what's happening here is that since mongoose filters out properties that are not in your schema, the query that mongoose sends to the db is below. Notice the {} obj instead of { email_path_dne: { $regex: 'abcd' } }.

await UserModel.find({
$or: [{ email: 'a@b.com' }, {}],
}, null, { strictQuery: false });

Confirmed in mongo shell that db.getCollection('users').find({ $or: [{email: 'i@i.com'}] }); returns only one doc (as expected), and db.getCollection('users').find({ $or: [{email: 'i@i.com'}, {}] }); with the {} returns all docs.

So the regex isn't evaluating to True. It's not even there. You can check what query/options mongoose is sending to the db by

mongoose.set('debug', (collectionName, method, ...args) => {
  console.log(`___MONGOOSE___: ${collectionName}.${method}, ${JSON.stringify(args, null, 2)}`);
});

[john-yick]

@mssodhi Thanks for the update that makes sense with the empty $or:[] issue is that for Mongo Compass when sending through such a query it would evaluate to "False" and return no documents.

This is what was throwing me off, as I would expect queries in Mongoose to also work for Mongo Compass, when default settings are applied to both.

See below:

[IslandRhythms]

If this issue is resolved please close it.

[github-actions[bot]]

This issue is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 5 days

[github-actions[bot]]

This issue was closed because it has been inactive for 19 days since being marked as stale.