search

Tarang / Meteor-Analytics

A super cool analytics package for meteor
https://tail.sh
64 stars 5 forks source link

DB performance / non-reactive views #19

Open dcsan opened 10 years ago

dcsan commented 10 years ago

the service has slowed to a crawl today, i assume your DB is getting crunched again?

I can't imagine how mongo will scale if those are reactive/observed views

would it be a big help for performance if you were to use non-reactive views? for me it's kind of cute that the tracking is updating in realtime, but not that essential. i mostly go in right now and look at a users detailed flow through the app well after they have left. and for aggregate queries (today's DAU etc) realtime doesn't add much

so this is a thing: https://github.com/richsilv/meteor-dumb-collections

Tarang commented 10 years ago

Mongo's working quite well with the observable stuff. Its only observed if someone is viewing the page.

Because its analytics, not very many people view it relative to the amount of data being inserted, so that bits quite ok, the number of observers is relatively low.

Its just we have a few huuugee users that sometimes come out of nowhere and choke the db with the amount of data coming in.

For example my db between yesterday and today has hit up 400mb larger from a single user, even though there is '2 day data retention'.

I'll have to work on a way to process the data coming in fast enough, at the moment thats the bottleneck i'm with.

dcsan commented 10 years ago

you could charge per event like mixpanel does. your system by default currently sends all events tho...