search

ioBroker / AdapterRequests

This Place is used to track the status of new Adapter-Requests.
248 stars 36 forks source link

MongoDB #573

Open SimonFischer04 opened 3 years ago

SimonFischer04 commented 3 years ago

If you have a lot of sensors that produces a lot of data a relational database isn't ideal. Think a MongoDB integration would be great.

https://www.mongodb.com/

There's also a free self hosted version: https://www.mongodb.com/try/download/community

Implementation ideas: https://www.w3schools.com/nodejs/nodejs_mongodb.asp

Apollon77 commented 3 years ago

Your idea is to have amongodb as "statistic" db like influxdb and sql?

Is mongodb stsructure and purpose really the right one for such kind of data?

SimonFischer04 commented 3 years ago

"Your idea is to have amongodb as "statistic" db like influxdb and sql?" Basically Yes, but...

My initial idea about this was when I saw that the sql adapter stores all the number objects in one table. And "many" data from "many" objects can easily sum up to big table -> decreasing performance. Honestly and don't know better way of doing this in an relational database without realtering the table often or storing each object in its own table <- both ideas really not ideal.

Also to get the real potential out of NoSQL 1 idea would be to group the data from 1 adapter or 1 objectfolder or ... into 1 Document and because of the way NoSQL works is that you can easily add more data(object-values) to them over time.

So, in fact this was also a bit of an idea / i don't have a propper solution on how this could implemented. Any ideas are welcome here. Furthermore I don't know if it's the best place to discuss this here? Maybe it's better to create a forum post?

DutchmanNL commented 3 years ago

So, in fact this was also a bit of an idea / i don't have a propper solution on how this could implemented. Any ideas are welcome here. Furthermore I don't know if it's the best place to discuss this here? Maybe it's better to create a forum post?

would be easier to discuss, In fact the issues your are mentioning with the SQL way of storing data is the benefit of having InfluxDB. So i am wondering if adding a new data logger makes sense, or we should advise what is the best here.

anyway, having mongo DB supported makes sense in basis, but isn't that just same as SQL query engine and logic so MySQL adapter could use that ?

Excodibur commented 2 years ago

I don't know if it still relevant, but here is an article that discourages the use of MongoDB to store time-series data (what I would understand as "statistic" data in context of iobroker): https://medium.com/timescale/how-to-store-time-series-data-mongodb-vs-timescaledb-postgresql-a73939734016

The current db-adapters (sql/influxdb) persist state-history in a database, meaning the information which value a state had at a certain point in time. That is basically the core purpose of time-series databases and that is what makes specialized db solutions like influxdb, or similar perfect for this use-case. Trying to recreate this by extending document-structures in an NoSQL db like MongoDB does seem like trying to reinvent the wheel, with little chance of gaining any advantage over specialized time-series solutions from it.

Don't get me wrong, I think MongoDB is a great DB for a lot of use-cases that a relational DB with fixed DB-structures can't effectively handle and professionally I work on a daily basis with it, but for tracking relatively simple ioBroker states it seems overkill.

SimonFischer04 commented 2 years ago

yeah, just thought about it again and you're absolutely right that MongoDB is probably not the right choice for time series data this is why influxdb exists.

I thought about: maybe someone has data in a mongodb and wants to integrate it into their iobroker smart home system. or any other usecase where an document-structure is useful. but i can't how that would be somehow beneficial / any usecase for iobroker currently

so close?