carolynvs
commented
3 years ago Closing now that we've validated that this works well in Porter and that we all agreed that we don't need the data access layer in cnab-go. Each tool will implement retrieving/storing documents themselves.
When CNAB was first created, there was only a single document stored, claims. The default storage was the filesystem which was fine at the time. The Store interface was created with this simplicity in mind.
When I added more document types (installations, claims, results, outputs, credential and parameter sets), I tweaked the Store interface to use "groups" (poor man's foreign key) so that it was implementable by the filesystem storage provider.
As I am implementing the Installation State specification with support for namespaces and labels, I am struggling with making this work for the filesystem store. What we need is efficient queryable storage, which the filesystem store can't do (without essentially implementing a database). The filesystem store was always the lowest common denominator and at this point it's holding us back from a better storage abstraction.
In the CNAB meeting today I proposed that we switch the filesystem store with a local filesystem database, like sqlite. It would still let developers quickly use CNAB without requiring additional services, without limiting our storage interface. For example, we could query for the last claim associated with an installation without having to list all the claims first. Or we could query for all installations within a namespace, or that has a particular label. A bonus would be that we can use an in-memory sqlite database for the mock store as well.
Ultimately we would then have just two store implementations:
I am aware of two active implementations of CNAB: Porter and Data Dog. Porter can handle migrating from the filesystem store to a sqlite database. I believe DataDog isn't using the storage interface from cnab-go. Right, @jlegrone?
Would there be any concerns or objections to changing the filesystem store to use sqlite?