akclace
commented
8 months ago The plan for data persistence support in Clace is two fold. First (managed) is a built-in solution and second (external) uses plugins for accessing external databases. The external approach will mean plugins for any database, exposing the database native api (like SQL), but the tables/collections have to be managed externally by the user. Clace plugins just provide the API to Starlark to work with the external storage.
The first (managed) solution involves Clace managing the storage abstraction, automatically creating the tables and providing the migration required as schema evolves. The plan is to implement this solution first. The second external solution will come after the support for external plugin loading is added.
The goals for the managed persistence solution are:
- Storage automatically managed by Clace, no manual table management required.
- Portable across storage providers, without requiring application code changes. Initial target is sqlite and postgres, mongodb should be possible later.
- Avoid implementing an ORM
The current design to achieve these goals is:
- Implement a document store approach. The documents are persisted as sqlite tables, postgres tables or later mongodb collections. JSON(B) support in the db is used to implement the sqlite/postgres solution.
- Provide schema support, such that the document store has a structured schema instead of being a data dump. The schema is defined by the app developer, it is persisted to the storage as json. Nested lists and dicts are supported. The database schema is not expected to be normalized. Joins are not supported.
- Each app can have a linked storage location. One store can be shared across apps. Each store has one main app, which manages the schema migration. Other secondary apps can work with the store as per their defined permissions. An admin app will be provided later which allows administering the storage.
- Schema migration support using object update functions. No database changes are required for schema migration, since a JSON document is used for storage
- CRUD API's are exposed for use from Starlark code, loosely based on the mongodb python API pymongo.
- Indexes can be defined in the schema to improve performance and set uniqueness constraints.
The end user interface should be simple. An app create call on an app which used the managed persistence plugin should automatically create the tables in sqlite (default), no additional work should be required.
The current status is the schema loading support was added few days back https://github.com/claceio/clace/commit/693fd626d300db161e71a0f335d526799696d8cb. The next release 0.4 will have the full managed sqlite support in a few weeks.
The goal of Clace is to prove a complete solution for building hypermedia based web applications, backend and UI. The storage API will be exposed as a Starlark call for use in the backend Starlark application code. There is no plan to automatically create JSON REST API's for use from external applications. The starlark code is the only client. That code can implement a JSON api if desired (though Hypermedia clients expose HTML apis generally). In that way, Clace is different from Firebase and Pocketbase type backend as a service solutions, which prove storage and automatically expose JSON REST API's.
Would appreciate any inputs on the proposed solution.
gedw99
Hey @akclace
Can we call the Sqlite DB to connect them to the htmx layer ?
I use Pocketbase for htmx apps in golang at the moment. https://github.com/pocketbase/pocketbase