search

The-Randalorian / Open-Assistant

An open source voice assistant
GNU General Public License v3.0
3 stars 1 forks source link

Thing retention #7

Closed The-Randalorian closed 3 years ago

The-Randalorian commented 3 years ago

Thing objects should be able to be remembered between restarts. This will require 3 main things to happen

  1. Thing objects will need to be serializable.
  2. Thing object data will need to be storable in a searchable format.
  3. Thing object data will need to be automatically saved and loaded.

Currently, the plan is to use Camel/YAML with custom load and dump functions for serialization. Separate plugins will be used for storage to allow for different APIs to be developed in the future. However, for testing and development an SQLAlchemy based SQL storage system will be made. This can then be used with most any SQL system; SQLite for local use or MySQL for multiple-device testing.

The-Randalorian commented 3 years ago

Part 1 is basically done. Camel was used, however zlib compression was added to the packaged Thing format (PTF? Need some sort of shorthand) to hopefully reduce the storage space used and increase transmission speed. With smaller Thing objects, it actually adds a few bytes. Hopefully the savings with larger Thing objects will make it worth while. Out of the compression algorithms provided by the python standard library, zlib added the least overhead for small Thing objects, so that was picked.

While Thing subclasses can make custom load and dump functions (replace _loader and _dumper methods), developers may find it easier to add any data that should be saved to the _saveables dictionary. The keys to that dictionary should be the arguments used to create the initial object.

Code will be uploaded soon.

The-Randalorian commented 3 years ago

Parts 2 and 3 are usable. Encryption still needs to be implemented to help protect user data, but otherwise most features described are working.