home-assistant / architecture

Repo to discuss Home Assistant architecture
313 stars 100 forks source link

Support external RDBMS or NoSQL DB for the use case of .storage #472

Closed gsdevme closed 1 year ago

gsdevme commented 3 years ago

Context

When deploying HA in containerised environment this would further ease a high availability deployment strategy for Home Assistant by keeping the container volume immutable. Furthermore it would mean running two or more or even a beta version locally would be easier.

The recorder already supports some RDBMS options so there is certainly users who have this already ready to go without any extra deployments required. Further keeps parity with its featureset also as the recorder has been able to do this for a while.

Few more points

Proposal

Support external RDBMS or NoSQL for the use case of .storage as a more advanced configuration.

Consequences

Not exactly sure here, Id imagine the implementation might be difficult depending how pesudo style flat file database is implemented.

Work Arounds

My current workaround is to run a NFS share that can be access from multiple nodes. It's not as stable as a native database though as its subject to other pitfails.

I'm also trying to use the CLI to do certain tasks as a "init" step before my container boots but there is limited support for things although sometimes the non-secret things can be put into VCS but its abit of guess work to know those.

BradleyFord commented 3 years ago

Agree, this would be awesome and allow storage for a much longer time period.

We already have the ability to push days to InfluxDB, but this is then totally separate to the HA implementation.

Switching to InfluxDB as a default would be very good and easy to manage via the supervisor structure.

On Mon, Dec 14, 2020, 3:33 AM Gavin Staniforth notifications@github.com wrote:

Context

When deploying HA in containerised environment this would further ease a high availability deployment strategy for Home Assistant by keeping the container volume immutable. Furthermore it would mean running two or more or even a beta version locally would be easier.

The recorder already supports some RDBMS options so there is certainly users who have this already ready to go without any extra deployments required. Further keeps parity with its featureset also as the recorder has been able to do this for a while.

Few more points

  • A Raspberry Pi deployment having both the recorder & .storage external really would help reduce the problems you run into I/O wise on the SD Card.
  • Existing industry standard database tools could be utilised including for backup.
  • Running beta versions on another host or locally is very simple without manually interacting with a filesystem of an undocumented nature (from a user point of view).
  • I/O read/write on the host needs to be catered for.
  • Mixture of 'secrets/configuration' might be easier to restructure outside of a big folder with files re: #370 https://github.com/home-assistant/architecture/issues/370
  • Very often its confusing from a user prospective what we even need to do with .storage (see #370 https://github.com/home-assistant/architecture/issues/370)
  • Automated deployments via Kubenetes, ansible, docker etc are all trivial as you don't need to worry about the .storage folder.
  • Migration when upgrading could utilize advanced RDBMS features to provide rollbacks if the configuration check was to fail (i.e. abort an upgrade and prevent mutating the database)

Proposal

Support external RDBMS or NoSQL for the use case of .storage as a more advanced configuration. Consequences

Not exactly sure here, Id imagine the implementation might be difficult depending how pesudo style flat file database is implemented.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/home-assistant/architecture/issues/472, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACOZ7ZFJYAMQNI66BD4SU3SUVMOZANCNFSM4UZ5MNPQ .

gsdevme commented 3 years ago

Switching to InfluxDB as a default would be very good and easy to manage via the supervisor structure.

As in switching the native recorder to Influx? yeah Id perhaps like that although this is specifically about the .storage/ rather than the recorder thats sqlite.db by default. You can already put the recorder in a RDBMS

BradleyFord commented 3 years ago

Ah yes you are correct, sorry for disturbing.

On Mon, 14 Dec 2020 at 18:48, Gavin Staniforth notifications@github.com wrote:

Switching to InfluxDB as a default would be very good and easy to manage via the supervisor structure.

As in switching the native recorder to Influx? yeah Id perhaps like that although this is specifically about the .storage/ rather than the recorder thats sqlite.db by default. You can already put the recorder in a RDBMS

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/home-assistant/architecture/issues/472#issuecomment-744528813, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACOZ76HXLPKAYMZB4KWUZLSUYXUZANCNFSM4UZ5MNPQ .

shbatm commented 3 years ago

I like this idea as well. The json structure of the /.storage/core.* files seems like it'd lend itself to move to MongoDB or other NoSQL style, which could be spun up in its own docker container under the Supervisor or extended to another server instance for advanced users.

frenck commented 1 year ago

This architecture issue is old, stale, and possibly obsolete. Things changed a lot over the years. Additionally, we have been moving to discussions for these architectural discussions.

For that reason, I'm going to close this issue.

../Frenck