Geni has dump and restore commands now we can tap into.
This gives you the db schema back from the current DB, which is very useful to help with development.
Fancy tools can do a diff on a DB schema between 2 databases.
So really we need to get to the point where you gen the DB off the golang model to make things predictable.
then you can easily do a DB Dif an gen the migraions.
Just need to specialise to work with genji. which is not hard.
The above will work...
But we can also plan for the future also....
Untyped no code development to make it fast and easy for devs to builds apps.
We will always have our core API and Apps and they can use the typed system we have now. These will build up over time and be designed as a foundation. They will be able to be composed into highe order apps and / or run on their own.
Will we definitly move to a pub sub pattern there you can subscribe to changes to a genji table, and get a event of changes that match your subscription filters. The filter os course just match what the original query was ! SO you essentially are doing an "query and subscription in one" to the system and saying i want the data for this query, and to be told whenever it changes.
How we do that will be a mix of Badger subscribe, genji hooks, NATS, Liftbridge.
You will be saying " but this is graphql, so lets use that !". Lets not. GraphQL is a nice idea and but the tooling, eco system sucks, and we can easily do better. Proto IDL can have tags, and so we can add subscription meta tags to good old proto.
Learning proto is easy. There are awesome GUI and reflection tools.
But graphQL allows oyu to do deep queries ! SO can we, because we model the relationships between one table and another in the Proto IDL tags. We already have a way of doing a sub query using an event system, that we can extend. This allows use to run a single binary or a bg microservice. Those tags can tell use at runtime where something is in terms of a Foreign table to the main table, and our engine can easily do that.
This is where the migration aspects comes in...
We should use Protocol IDL to our advantage and work towards having a one to one mapping of the proto IDL match the DB.
This then sets us up for the world of typed and untyped.
In the typed world we generate the DAO layer that maps the Proto to the DB.
In the untyped world we do exactly the same but using reflection.
we need a way to do migrations of the DB
normally you need to deal wth:
Geni has dump and restore commands now we can tap into.
https://dev.to/techschoolguru/generate-crud-golang-code-from-sql-and-compare-db-sql-gorm-sqlx-sqlc-560j https://sqlc.dev/
https://github.com/scylladb/gocqlx
Just need to specialise to work with genji. which is not hard.
The above will work... But we can also plan for the future also.... Untyped no code development to make it fast and easy for devs to builds apps.
We will always have our core API and Apps and they can use the typed system we have now. These will build up over time and be designed as a foundation. They will be able to be composed into highe order apps and / or run on their own.
Will we definitly move to a pub sub pattern there you can subscribe to changes to a genji table, and get a event of changes that match your subscription filters. The filter os course just match what the original query was ! SO you essentially are doing an "query and subscription in one" to the system and saying i want the data for this query, and to be told whenever it changes. How we do that will be a mix of Badger subscribe, genji hooks, NATS, Liftbridge. You will be saying " but this is graphql, so lets use that !". Lets not. GraphQL is a nice idea and but the tooling, eco system sucks, and we can easily do better. Proto IDL can have tags, and so we can add subscription meta tags to good old proto. Learning proto is easy. There are awesome GUI and reflection tools. But graphQL allows oyu to do deep queries ! SO can we, because we model the relationships between one table and another in the Proto IDL tags. We already have a way of doing a sub query using an event system, that we can extend. This allows use to run a single binary or a bg microservice. Those tags can tell use at runtime where something is in terms of a Foreign table to the main table, and our engine can easily do that.
This is where the migration aspects comes in... We should use Protocol IDL to our advantage and work towards having a one to one mapping of the proto IDL match the DB. This then sets us up for the world of typed and untyped. In the typed world we generate the DAO layer that maps the Proto to the DB. In the untyped world we do exactly the same but using reflection.