search

mozilla-services / loads

SUPERSEDED BY https://github.com/loads
https://github.com/loads
107 stars 17 forks source link

Create a mongodb backend for the broker #157

Open almet opened 11 years ago

almet commented 11 years ago

Because we store too much little lines, redis uses all the memory on the server and then everything gets unresponsive.

We should consider other backends, such as mongodb where everything is not written in memory.

tarekziade commented 11 years ago

We are dealing with time series here since every line of a test comes with a timestamp, so I wonder how good is mongo db with this kind of data.

Do you have some xp in Mongo ?

I know Elastic Search has extended features to deal with time series, like group bys etcs, that would let us write an efficient db backend - but I am curious to see how this would work with Mongo

almet commented 11 years ago

I have used mongo in the past but I don't think it's especially good for time series. It's good to store a lot of data, though; so that will be better than the actual redis.

If we really want to store and index lots of data, because we have an expertise with ES we might use it instead. That would be interesting to have a look at the features mongo has and eventually use it.