search

treasure-data / digdag

Workload Automation System
https://www.digdag.io/
Apache License 2.0
1.3k stars 221 forks source link

Using varaible filenames in !import #920

Open nickhall opened 5 years ago

nickhall commented 5 years ago

I'd like to define a workflow like this:

_export:
  pg:
    !include : 'config/${environment}.dig'

But the variable is not expanded, with or without quotes. Is this possible?

hiroyuki-sato commented 5 years ago

Hello, @nickhall

Have you ever tried the following?

digdag run -a work3.dig -P digdag_env_test.yml or digdag server -P digdag_env_test.yml

It seems that this issue is similar. https://github.com/treasure-data/digdag/issues/847#issuecomment-415707929

nickhall commented 5 years ago

@hiroyuki-sato Thanks. Looking over that it does seem that I have the same issue. Using a param file like that would probably work, but as noted in the other issue it would require a restart whenever changing parameters. Over there using the parameter store is floated as a solution, but from the docs it doesn't seem possible to set params from the command line, so I'm not sure exactly what process he's using.

For now I'll go with the parameter file, and I left a comment on that issue as well.