SQLite: allow generate from schema file

[emarj]

Sometimes is handy to generate files from schema files (see for example https://github.com/go-jet/jet/issues/136).

In general this is hard to do. In SQLite it is so easy to create an in-memory throw-away database that this can be easily accomplished.

This PR propose to add a flag to specify a schema file that will be used to generate files. If no DSN is specified, an temporary in-memory database will be used, otherwise the schema file will be executed in the specified database.

In order for this to work, a change to the test data is needed: https://github.com/go-jet/jet-test-data/pull/2

The README/Wiki should also be updated with this new flag.

What do you think?

[houten11]

I'm not sure what benefits bring such a feature. Currently, my development cycle looks like this: 1) Add a new migration file. 2) Apply migration to the local test database 3) Generate jet files from running local test database 4) Add/update business logic 5) Add tests for new business logic 6) Push changes for PR

The only two reasons I can see to generate jet files from SQL migration files are in case developers don't have a local development setup. If there is no local test database, then how business logic is tested? This is such a bad practice that I don't see a point supporting it.

Of course, there might be some other use cases that I'm missing.

[emarj]

Well, my use case is the following.

In the early stages of development I use an in-memory SQLite, which will be automatically destroyed and recreated at every run. I want to populate it with test data at every startup and I would prefer to do it from my business logic layer since there I can use the already present logic and I have compile time checks (thanks to jet). In order to do this of course I need to have generated files. Since there is no physical database, in order to run jet, I need to create a temporary one and then destroy it. This is why I decided to modify the code (and then decided to share it here).

Of course I could use a physical db from the beginning, and in fact I often do this, but the situation is more or less the same since, again in early development, I want to destroy it and recreate it at every run. Nonetheless, there might be people (?) running SQLite as an in-memory database even in production, and this allows them to use easily jet.

In general, my personal point of view is that the application comes first, and it is often responsible to create and manage the database. And since jet is a code generator, it seems natural to me to run it from a schema, even before the database exists. Of course, this is just my own opinion.

[go-jet]

In the early stages of development I use an in-memory SQLite, which will be automatically destroyed and recreated at every run...

You can achieve the same thing using a script that drops the whole schema, runs the migration script, and then runs a jet generator.

Nonetheless, there might be people (?) running SQLite as an in-memory database even in production, and this allows them to use easily jet.

For an in-memory SQLite database, you can call jet generator from the code. Just pass it a in-memory db connection.

In general, my personal point of view is that the application comes first, and it is often responsible to create and manage the database. And since jet is a code generator, it seems natural to me to run it from a schema, even before the database exists. Of course, this is just my own opinion.

IMO it's migration -> database(jet gen) -> application, where migration is responsible for the creation and management of the database. I know there are libraries(like GORM), which try to reverse this process, but jet is not built on those principles.  

Honestly, even if someone comes up with a valid scenario for jet to parse DDL, the probability of including such a feature in jet is very low.

[emarj]

You can achieve the same thing using a script that drops the whole schema, runs the migration script, and then runs a jet generator.

Yes, for sure, for physical DBs this is possible.

For an in-memory SQLite database, you can call jet generator from the code. Just pass it a in-memory db connection.

True, but since the generator has to run before compilation this basically amounts to rewrite (fork) a tool like cmd/jet but with this feature built-in.

IMO it's migration -> database(jet gen) -> application, where migration is responsible for the creation and management of the database. I know there are libraries(like GORM), which try to reverse this process, but jet is not built on those principles.

Honestly, even if someone comes up with a valid scenario for jet to parse DDL, the probability of including such a feature in jet is very low.

I don't have strong opinion on these matters, so I will not try to argue further.

Sounds like a reject at this point. Feel free to close this.

[go-jet]

True, but since the generator has to run before compilation this basically amounts to rewrite (fork) a tool like cmd/jet but with this feature built-in.

The ability to call a jet generator from the code was initially added because it was hard to anticipate every possible use case. In-memory db could be one of those use cases.