.findOne() is different from .findOne().lean()

[sibelius]

Prerequisites

• [X] I have written a descriptive issue title
• [X] I have searched existing issues to ensure the bug has not already been reported

Mongoose version

6.8.2

Node.js version

16.18.0

MongoDB server version

6.0.1

Typescript version (if applicable)

4.9.4

Description

related to https://github.com/Automattic/mongoose/issues/12905

nested paths are not working well when using just .findOne, but works well with lean

example

this breaks

const user = await User.findOne();

user.my.nested.path // this is undefined

this do not breaks

const user = await User.findOne().lean()

user.my.nested.path // not undefined

we can see nested path in mongosh and robo 3t

{
   "my": { "nested": { "path": "value } } 
}

it does not work on 6.8.2, 6.8.0 it works on 6.7.0

I will try to find the exactly version that breaks

this is a regression

Steps to Reproduce

const user = await User.findOne({})

  const userLean = await User.findOne({})

user.my.nested.path !== userLean.my.nested.path

Expected Behavior

it should resolve properly with and without .lean()

[sibelius]

the weird thing is

when I console.log(user)

it shows the

{ my: { nested: { path: 'value' } } }

however when I try to do user.my it is already undefined

is there any kind of getter that change this behavior somehow?

[sibelius]

user.toJSON().my // works
user.my // do not work

[sibelius]

user.my // do not works
user.get('my') // works

[MohOraby]

Can you show more context like your DB documents and your model?

This seems to be working fine for me @6.8.2, 6.8.3 and 6.8.0

'use strict';

import mongoose, { Schema } from "mongoose";

await mongoose.connect('mongodb://localhost:27017/nested');

const User = mongoose.model('User', new Schema({
  my: {
    nested: {
      path: String
    }
  }
}));

await User.create({
  my: {
    nested: {
      path: 'test'
    }
  }
});

const user = await User.findOne({})
const user1 = await User.findOne({}).lean()

console.log(user.my.nested.path, user1.my.nested.path);

[sibelius]

is there an email where I can send you some private data?

how can I debug mongoose getters?

like, where is the code when user.my is called ?

[MohOraby]

@sibelius I don't think you should send private data, but it's just that the test case you shared to replicate the issue seem to be working fine, so if you could share a little more context to replicate the issue

maybe try on a new DB with the schema you currently have and see if the issue still persists

[sibelius]

in a clean database, it works well

but for this specific item it is breaking

[MohOraby]

Then it's not really a problem with findOne itself. could you provide a code that reproduces the error

[sibelius]

this is failing on this specific item

export const run = async () => {
  const _id = '63bf33de88954ec32e13bbbe';
  const user = await User.findOne({
    _id,
  });

  const userLean = await User.findOne({
    _id
  }).lean();

  console.log({
    p: user.my,
    o: user.my?.nested,
    d: user.my?.nested?.path,
    p1: userLean.my,
    o1: userLean.my?.nested,
    d1: userLean.my?.nested?.path,
  });

  console.log({
    user,
    userLean
  });

  // console.log(diffString(user, userLean));
}

[MohOraby]

@sibelius Can you show the document (mongo compass or studio 3t)? blur any data, just wanna see how it looks in the DB

[sibelius]

this is the robo3t - the real nested path is user.preferences.openpix.defaultCompanyBankAccount, preferences is already undefined

given this console.log

const user = await User.findOne({
    _id,
  });

  const userLean = await User.findOne({
    _id
  }).lean();

console.log({
    user,
    p: user.preferences,
    o: user.preferences?.openpix,
    d: user.preferences?.openpix?.defaultCompanyBankAccount,
    p1: userLean.preferences,
    o1: userLean.preferences?.openpix,
    d1: userLean.preferences?.openpix?.defaultCompanyBankAccount,
  });

check the image below

[sibelius]

when I print the mongoose document without lean

it has the preferences in the _doc

example

-  $__: {
-    activePaths: {
-      paths: {
-        _id: "init"
-        createdAt: "init"
-        updatedAt: "init"
-        __v: "init"
-      }
-      states: {
-        init: {
-          _id: true
-          createdAt: true
-          updatedAt: true
-          __v: true
-        }
-      }
-    }
-    skipId: true
-  }
-  $isNew: false
-  _doc: {
-    _id: {
-    }
 -    preferences: {
-      openpix: {
-        defaultChargeExpiration: 86400
-        defaultCompanyBankAccount: {
-        }
-        defaultCompanyPixKey: null
-      }

[MohOraby]

can you check if the answers in #7441 help?

[sibelius]

it does not

this json

{
    "_id" : ObjectId("63bf33de88954ec32e13bbbe"),
    "preferences" : {
        "openpix" : {
            "defaultCompanyBankAccount" : ObjectId("63c6b351f33c59502dc43603")
        }
    },
    "createdAt" : ISODate("2023-01-11T22:10:38.929+0000"),
    "updatedAt" : ISODate("2023-01-22T00:06:22.982+0000"),
    "__v" : NumberInt(0)
}

is breaking for me, can you give a try?

[sibelius]

is preferences a reserved word on mongoose?

I'm trying to debug the issue

when I try to get preferences property from Document.prototype.get, the this._doc does not have preferences field, but it has the other ones.

https://github.com/Automattic/mongoose/blob/master/lib/document.js#L1854

[sibelius]

where and when the this._doc is built ?

[IslandRhythms]

const mongoose = require('mongoose');

const testSchema = new mongoose.Schema({
  familyTree: {
    greatGrandfather: {
      grandFather: {
        father: {
          son: String
        }
      }
    }
  }
});

const Test = mongoose.model('Test', testSchema);

async function run() {
  await mongoose.connect('mongodb://localhost:27017');
  await mongoose.connection.dropDatabase();

  await Test.create({
    familyTree: {
      greatGrandfather: {
        grandFather: {
          father: {
            son: 'Test Testerson'
          }
        }
      }
    }
  });

  const user = await Test.findOne();
  const leanUser = await Test.findOne().lean();

  console.log(user.familyTree.greatGrandfather.grandFather.father.son);
  console.log('==================================================');
  console.log(leanUser.familyTree.greatGrandfather.grandFather.father.son);
}

run();

[IslandRhythms]

provide a script that demos your issue and I will run it again

[sibelius]

try this schema

import mongoose from 'mongoose';

const { ObjectId } = mongoose.Schema.Types;
export const openpix = {
  defaultCompanyBank: {
    type: ObjectId,
    ref: 'CompanyBank',
  },
  defaultCompanyBankAccount: {
    type: ObjectId,
    ref: 'CompanyBankAccount',
  },
  defaultCompanyPixKey: {
    type: ObjectId,
    ref: 'CompanyPixKey',
  },
};

const CompanyPreferences = new mongoose.Schema(
  {
    openpix,
  },
  {
    _id: false,
  },
);

const Schema = new mongoose.Schema(
  {
    preferences: {
      type: CompanyPreferences,
      description: 'Company preferences',
    },
  },
  {
    collection: 'Company',
    timestamps: true,
  },
);

const CompanyModel = mongoose.models.Company || mongoose.model(
  'Company',
  Schema,
);

export default CompanyModel;

I think the issue is in schema inside schema

can you try to create the data directly in the database? instead of creating directly in the script ?

[IslandRhythms]

I'll run it in a bit, however, at first glance this seems like an issue.

const CompanyPreferences = new mongoose.Schema(
  {
    openpix,
  },
  {
    _id: false,
  },
);

this is making the schema definition look like

{{
  defaultCompanyBank: {
    type: ObjectId,
    ref: 'CompanyBank',
  },
  defaultCompanyBankAccount: {
    type: ObjectId,
    ref: 'CompanyBankAccount',
  },
  defaultCompanyPixKey: {
    type: ObjectId,
    ref: 'CompanyPixKey',
  },
}};

as opposed to

{
  defaultCompanyBank: {
    type: ObjectId,
    ref: 'CompanyBank',
  },
  defaultCompanyBankAccount: {
    type: ObjectId,
    ref: 'CompanyBankAccount',
  },
  defaultCompanyPixKey: {
    type: ObjectId,
    ref: 'CompanyPixKey',
  },
};

[IslandRhythms]

const mongoose = require('mongoose');

const { ObjectId } = mongoose.Schema.Types;
const openpix = {
  defaultCompanyBank: {
    type: ObjectId,
    ref: 'CompanyBank',
  },
  defaultCompanyBankAccount: {
    type: ObjectId,
    ref: 'CompanyBankAccount',
  },
  defaultCompanyPixKey: {
    type: ObjectId,
    ref: 'CompanyPixKey',
  },
};

const CompanyPreferences = new mongoose.Schema(
    openpix,
  {
    _id: false,
  },
);

const Schema = new mongoose.Schema(
  {
    preferences: {
      type: CompanyPreferences,
      description: 'Company preferences',
    },
  },
  {
    collection: 'Company',
    timestamps: true,
  },
);

const CompanyModel =  mongoose.model(
  'Company',
  Schema,
);

async function run() {
  await mongoose.connect('mongodb://localhost:27017');
  await mongoose.connection.dropDatabase();

  await CompanyModel.create({
    preferences: {
      defaultCompanyBank: mongoose.Types.ObjectId(),
      defaultCompanyBankAccount: mongoose.Types.ObjectId(),
      defaultCompanyPixKey: mongoose.Types.ObjectId()
    }
  });

  const res = await CompanyModel.findOne();

  console.log(res.preferences.defaultCompanyBank);
};

run();

Try removing the curly braces around openpix in the CompanyPreferences schema and see if that resolves your issue.

[sibelius]

it was a typo while copying the code

can you point it out, when and how the this._docis populated ?

[IslandRhythms]

this refers to the document and _doc is essentially doc. So if you console.log() the output, so res._doc, it should look like the document. Check to see if that's what it looks like for you.

const res = await CompanyModel.findOne();

  console.log(res.preferences.defaultCompanyBank);
  console.log(res._doc);

new ObjectId("63ceba97aea86a6ead764904")
{
  _id: new ObjectId("63ceba97aea86a6ead764907"),
  preferences: {
    defaultCompanyBank: new ObjectId("63ceba97aea86a6ead764904"),
    defaultCompanyBankAccount: new ObjectId("63ceba97aea86a6ead764905"),
    defaultCompanyPixKey: new ObjectId("63ceba97aea86a6ead764906")
  },
  createdAt: 2023-01-23T16:49:27.097Z,
  updatedAt: 2023-01-23T16:49:27.097Z,
  __v: 0
}

[sibelius]

mongoose create 1 Document per each Schema? even shema of schema ?

when I console.log({res._doc}) it does not have preferences, just the rest of data and properties

• res.preferences // do not work
• res.toJSON().preferences // do not work
• res.toObject().preferences // do not work
• CompanyModel.findOne().lean().preferences works
• res.get('preferences') // works

[IslandRhythms]

No, mongoose can create several documents per schema. Documents are part of the class 'Document' and so have access to a bunch of functions defined on the Document like populate() as well as properties like isNew. Have you tried updating to latest? Its possible this issue could have been resolved in an update. I'm unable to reproduce it on my machine so that indicates there is a problem with your setup, whether is be definition, document creation, or even the version of mongoose you are using.

[sibelius]

• [ ] mongoose@6.8.4
• [ ] mongodb@4.12.1

[IslandRhythms]

So you have updated? In the issue description it says you're on mongoose@6.8.2 and does not mention trying latest mongoose. Double check your schema definitions, I am unable to reproduce this on latest.

[sibelius]

the wrong schema definition can mess this up?

where in the mongoose codebase I can think the raw data returned from findOne, before the mongoose magic ?

[IslandRhythms]

The schema definition is how you define document paths. So if there is a typo or incorrect configuration, that would explain why when you attempt to access preferences you get undefined.

This is where the magic begins to happen https://github.com/Automattic/mongoose/blob/master/lib/model.js#L2343

[vkarpov15]

@sibelius it looks like defaultCompanyBankAccount is an empty object, so I think it is getting stripped out by the minimize option. Try adding minimize: false to your schema options and see if that helps.

[sibelius]

minimize didn't fixed

but I got a fix

removed

preferences: {
      type: CompanyPreferences,
      description: 'Company preferences',
      }

to inline

preferences: {
      openpix,      
      }