search

thbar / kiba

Data processing & ETL framework for Ruby
https://www.kiba-etl.org
Other
1.75k stars 87 forks source link

Kiba now defaults to "StreamingRunner" for processing jobs #83

Closed thbar closed 4 years ago

thbar commented 4 years ago

This PR ensures the StreamingRunner becomes the default for Kiba v3.0.0.

This change is expected to be backward compatible, but the new runner is more powerful and will be the default in Kiba v3.0.0, so it's time to make it the default.

This will be reflected in the upcoming documentation rewrite (#77).

What is the StreamingRunner?

Kiba v2.0.0 introduced a new "runner" (the code responsible for running the ETL jobs) named StreamingRunner.

https://github.com/thbar/kiba/releases/tag/v2.0.0

This runner allows the creation of more powerful and more reusable ETL components.

Note on legacy Runner

It's still possible to revert to the previous Runner using configuration like this:

Kiba.parse do
  extend Kiba::DSLExtensions::Config
  config :kiba, runner: Kiba::Runner
  # ...
end

It is likely, though, that this legacy Runner will be removed in Kiba v4.