Local data storage method - options

ElliotTheRobot / LILACS-mycroft-core

Mycroft Core, the Mycroft Artificial Intelligence platform - LILACS Fork

https://mycroft.ai

GNU General Public License v3.0

4 stars 1 forks source link

Local data storage method - options #1

Closed HeinzSchmidt closed 7 years ago

HeinzSchmidt commented 7 years ago

What are we using for the relational concept data storage? Considerations:

doesn't overly complicate the core installation procedure
is fast i.t.o data io
supports multi-node relationships
is robust
easy to backup and restore if a data corruption occurs

I'm thinking sqllite3 ? Any other ideas?

JarbasAI commented 7 years ago

my first approach would be saving all concepts as json, its simple and stuctures without introducing dependencies, but this is not my area so feedback is welcome

HeinzSchmidt commented 7 years ago

For the data structure, if we start with JSON for the sake of illustration:

'{ { "name": "Cow", "parents": [ { "gen1" : "herbivore", "gen2" : "animal" } ], "data_dict" : "Cow is the nickname for cattle, and the name of adult female cattle." }, { "name": "Herbivore", "parents": [ { "gen1" : "animal", "gen2" : "life" }, "data_dict" : "A herbivore is an animal anatomically and physiologically adapted to eating plant material, for example foliage, for the main component of its diet. As a result of their plant diet, herbivorous animals typically have mouthparts adapted to rasping or grinding. Horses and other herbivores have wide flat teeth that are adapted to grinding grass, tree bark, and other tough plant material." ] } }'

JarbasAI commented 7 years ago

i think we could also go with data BackEnds, so the user can config how he wants data stored, this backend would also be usefull for other skills that save data to use

this also brings the question, where will data be stored? should we create a database folder or inside LILACS skill folder is ok?

HeinzSchmidt commented 7 years ago

Yes, it would have to be in a location where it won't be messed with. Inside the skills folder in a ".db" subfolder should work?

ElliotTheRobot commented 7 years ago

In future, the config file will have the storage location of the LILACS datastore / db file and it can point to:

a local file on disk
a personal webservice on local network
a personal webservice hosted at mycroft.ai
a community/global webservice / knowledge store hosted at mycroft.ai

The possibilities are endless

JarbasAI commented 7 years ago

a storage service was created, currently defaults to json, opening an issue for each new backend

https://github.com/ElliotTheRobot/LILACS-mycroft-core/tree/dev/mycroft/storage