search

Meteor-Community-Packages / meteor-collection2

A Meteor package that extends Mongo.Collection to provide support for specifying a schema and then validating against that schema when inserting and updating.
https://packosphere.com/aldeed/collection2
MIT License
1.02k stars 108 forks source link

Performance impact (esp. with subdocuments) #338

Closed abesto closed 6 years ago

abesto commented 8 years ago

Hi! Today I started investigating slow updates in my app, found https://github.com/aldeed/meteor-simple-schema/issues/347, and built an app to measure performance along a number of axes (is the schema attached?, plus the validate and bypassCollection2 flags, number of DB entries and subdocuments).

I've found that performance degrades rapidly as even a small number (50 - 100) of subdocuments are added, as well as when inserting a relatively small number of DB entries (again, 50 - 100) when the schema is attached and bypassCollection2 is false. This is hinted at in the documentation, but even so, it feels like performance issues should crop up only when the (sub)document count is a few orders of magnitude higher. I realize improvements like this require major effort, my expectations are set accordingly, but maybe even starting a conversation is useful.

The repo to reproduce (just clone and start with meteor): https://github.com/abesto/meteor-simple-schema-347-repro

A screenshot of various measurements:

screenshot 2016-07-10 16 20 59

Taakn commented 8 years ago

Hey @abesto I'm also running into performance issues. My bigger worry right now is that this repo is dead (like Simple Schema) and that @aldeed is not maintaining it anymore. I'm about to launch my product and I'm already running into a bunch of performance problems. Simple Schema/Collection2 do not seem to be scalable.

For me the main issue is that clean is killing it, taking several seconds to complete. I'd be curious to hear about your tests or results if you've gone further in your analysis.

abesto commented 8 years ago

@Taakn nope, this is the full extend of my research. I guess one work-around is not using subdocuments. Another is not checking the schema at runtime. Can't think of any others.

For me this was used in a side-project only.

Taakn commented 8 years ago

Thanks @abesto MongoDB is offering document validation now so I might move towards that https://docs.mongodb.com/manual/release-notes/3.2/#document-validation

dthree commented 7 years ago

I have an app that has to regularly load 140kb of JSON into a collection while the user is waiting. This was taking 30 seconds! After removing SimpleSchema it took a quarter of a second.

I had to remove the schema until something gets fixed on this. ☹️

aldeed commented 6 years ago

Closing. I'm guessing this was fixed by SimpleSchema 1.2.0 last year. See https://github.com/aldeed/simple-schema-js/issues/206