search

golang-migrate / migrate

Database migrations. CLI and Golang library.
Other
15.3k stars 1.4k forks source link

New Driver Interface #14

Open dhui opened 6 years ago

dhui commented 6 years ago

Goals:

  1. Configurable timeouts
    • Granularity of configuration? e.g. different timeouts for different operations
  2. No possibility of dangling locks. See: https://github.com/golang-migrate/migrate/pull/13#issuecomment-372577783

Possible Goals:

  1. Composability
    • Allow drivers to be composed of another driver(s)
      • maybe expose a methods for getting:
        • the db connection (*sql.Conn or *sql.DB or some interface) - for overwriting migration management queries
        • the config - for shared config
  2. Safety
    • Struct fields should not be exported unnecessarily. e.g many DB driver Config structs have all of their fields exported. This is dangerous since the DB driver has a pointer to the Config, whose exported field values may be changed inadvertently. Another option would be to keep a copy of the Configstruct in the DB driver, but if any of the Config struct fields are pointers, we have the same problem...
      • Add NewConfig() method for creating configs with unexported struct fields.
      • Unexported Config struct fields may make sharing config between composed/dependent drivers harder

Ideas:

  1. Require context.Context as the first param for Driver receiver functions
    1. Which receiver functions should require context?
    2. How does a context timeout/cancel affect the DB state? Is it consistent across all DBs?
    3. Which DBs support context.Context?
    4. Will break DB drivers not explicitly maintained by this repo
  2. Add retry with timeout helper for DBs that don't support acquiring a lock within a timeout
  3. Make the Open method on the Driver interface take a url.URL struct instead of a string
  4. Run each migration in a transaction managed by migrate. See: https://github.com/golang-migrate/migrate/issues/196
  5. Add EnsureVersionTable() method
  6. Add a Metadata() method
  7. Remove Drop() method
  8. Create DB
    • See: https://github.com/golang-migrate/migrate/issues/347
    • Would require migrating migration storage to it's own DB in a DBMS and tracking the migration status for a DB. e.g. the schema migration table would need to track the latest migration, the db, and if it's dirty
      • To be backwards compatible, we'd need to support migrations for the schema migration table
  9. Use migrate to manage version and metadata table schemas
  10. Return concrete types in DB driver implementation with compile time type checks
Teddy-Schmitz commented 6 years ago

I would at least to start. Only add contexts to the Lock and Unlock functions. Since there is already configuration to have a Timeout but this seems to be very inconsistently applied across the drivers. With mysql, setting its own timeout of 10seconds, and postgres only returning instantly. I haven't checked the others.

dhui commented 5 years ago

Should support for DROP be removed? https://github.com/golang-migrate/migrate/issues/193#issuecomment-475848534