search

Electron100 / butane

An ORM for Rust with a focus on simplicity and on writing Rust, not SQL
Apache License 2.0
83 stars 11 forks source link

Automatic cleaning of .butane/migrations/current #216

Open jayvdb opened 3 months ago

jayvdb commented 3 months ago

This is sort of a continuation of https://github.com/Electron100/butane/issues/44 , and a devils advocate for one half of https://github.com/Electron100/butane/issues/60 which says

The proc macro runs independently on each #[model], and Cargo does not appear to provide any environment variables with a unique identifier for the build. Thus each .table file is generated independently and there isn't a great way to reap dead ones (that I've identified).

In the last task, one of the outcomes was to create cargo clean which is described as

Clean current migration state. Deletes the current migration working state which is generated on each build. This can be used as a workaround to remove stale tables from the schema, as Butane does not currently auto-detect model removals. The next build will recreate with only tables for the extant models

This is useful, but to be a solution, needs to be either

  1. very clearly documented as a required step when removing tables or types,
  2. automated, either auto-cleaning, or auto-detecting cleaning is needed.

https://github.com/Electron100/butane/issues/44 proposes extra effort is going to be required of developers. If so, it would be easy for developers to create a build.rs and set a unique build identifier variable.

Another approach is to derive a unique-enough id from all files under src/ (and tests/ if CARGO_CFG_TEST is set). Could be file modification timestamps or hashes.

So if we had a build identifier, what can we do? For it to be of any use, it would need to be stored somewhere under .butane/. As it will change frequently, it would need to be a file which isnt checked into VCS.

The "solution" of creating a build.rs to auto-clean .butane/migrations/current caused https://github.com/Electron100/butane/issues/56 - Using an IDE while running tests causes test failures. The fix in https://github.com/Electron100/butane/pull/215 only works because in that project all of the models are in tests/. It isnt a general solution.

To use this build-id to reap dead tables/types, I guess the either a build.rs or proc-macros would detect that that there is a new build, and delete all the current files - esp types.json which is always json-appended to. Unless done carefully, this is break in the "IDE + running tests" scenario.

What would be interesting is if the proc-macros could write the new files into .butane/builds/<id>, and then some process known to run at the end be injected to replace the current migration dir files if a change was detected.

jayvdb commented 2 months ago

Perhaps we can catch the problem at "make-migration" stage.

It is very easy to auto-clean types.json - remove any type which is no longer mentioned in any table.

To get the current list of tables, we may be able to use CrateRoot.impls to get all the implementations of DataObject.

https://doc.rust-lang.org/nightly/nightly-rustc/rustc_metadata/rmeta/struct.CrateRoot.html https://doc.rust-lang.org/nightly/nightly-rustc/rustc_metadata/rmeta/struct.TraitImpls.html

These files are generated during each build. We would need to find the most recently written one at

$ find target/ -name 'getting_started.rmeta'