search

thbar / kiba

Data processing & ETL framework for Ruby
https://www.kiba-etl.org
Other
1.75k stars 87 forks source link

Passing arguments on the command line #39

Closed pmackay closed 7 years ago

pmackay commented 7 years ago

I think this would be useful but dont currently see a way to do it. I'm wrapping Kiba scripts in Rake tasks for convenience and to be able to do other actions. So the paths to input/output file locations are in Rakefiles and duplicated in ETL files. Passing in from the command line would be handy.

thbar commented 7 years ago

If you are mostly relying on command line, I can recommend passing ENV variables (as described in this StackOverflow answer.

Still on the command line, if you have more elaborated needs, you can pass an ENV variable giving a path to a YML/JSON file with more configuration.

thbar commented 7 years ago

@pmackay I've also added a link to that question in the readme, since it seems like a FAQ. Thanks for asking, let me know if you have further questions!

pmackay commented 7 years ago

Very helpful, thanks! It might be worth adding a small note in the How do you run your ETL jobs? section of README pointing to the FAQ as well, as that section is about running jobs.

Much appreciated :)

thbar commented 7 years ago

Good suggestion! I just added it. Thanks for your help improving the onboarding experience :smile:

thbar commented 6 years ago

FYI @pmackay in addition, Kiba v2 is now out (https://github.com/thbar/kiba/releases/tag/v2.0.0) and officially supports a programmatic invocation. This means that you can implement your own form of CLI very easily, pass more elaborate data (live instance etc).