search

thbar / kiba

Data processing & ETL framework for Ruby
https://www.kiba-etl.org
Other
1.75k stars 87 forks source link

Deprecate non-live Sequel connection passing #79

Closed thbar closed 4 years ago

thbar commented 5 years ago

Today it is possible in Kiba Pro to pass a simple database url string to SQL components (SQL Source, SQL single record Upsert destination, SQL bulk insert / upsert destination), like this:

database: ENV.fetch('DATABASE_URL')

For the same reasons the kiba command line is getting deprecated, I'll instead encourage to pass live connections here, created via a surrounding block:

Sequel.connect(xxx) do |db|
  Kiba.run(ETL::MyJob.setup(db: db))
end

This will allow automatic connection closing, which is a better pattern, and also brings more control over how the connection can be tweaked (e.g. extensions).

Keeping this note as a roadmap item.

thbar commented 4 years ago

Kiba Pro v1.5 (to be released today) deprecates the use of database url. Closing.