search

Automattic / mongoose

MongoDB object modeling designed to work in an asynchronous environment.
https://mongoosejs.com
MIT License
26.97k stars 3.84k forks source link

Running into 16MB BSON limit with Mongoose Populate #5890

Open mayeaux opened 6 years ago

mayeaux commented 6 years ago

I am doing a really large Mongoose call with nested population, and then doing some processing on the data.

However, because of the nested populate call I am hitting the 16MB limit for a BSON document in Mongoose.

What's a solid way around this? 

let allDocuments = await exampleModel.find({
    condition: true
  }).populate({path: 'tvShows', populate: {path: 'views'}});

enter image description here

Maybe I can break into into multiple calls? But I'm not sure a logical way to do that. Thanks.

vkarpov15 commented 6 years ago

Populate currently doesn't have support for this. We will fix this bug but will take some time. In the meantime I recommend looping over allDocuments.tvShows and executing a separate query for each ones views

mhombach commented 5 years ago

Bump: What's the plan with this? @vkarpov15 would really appreciate if this would be somehow fixed (or is it already fixed? Didn't run into that problem yet) :) To all others, who are running into that problem, you can just query without populating and then call populate() on the subdocuments during runtime, so they get "loaded" with sepreate calls after the parent-document is loaded.