Open tandav opened 1 year ago
Thanks @tandav for sharing this suggestion. Right now the equivalent workaround is to generate the export commands yourself:
@task()
def align_dna_reads_to_genome(key1, key2):
return script(f"""
export KEY_1={key1}
export KEY_2={key2}
my_command
"""
)
Is your suggestion to have script()
help prepend such export
lines when env
is given? I could see that being helpful since there is some subtlety in properly escaping the env var values.
Thanks for workaround example!
I see that scripting.py calls subprocess.run, which have env
argument
https://github.com/insitro/redun/blob/8df1415e9f3848d4f430f52df5077995e15bc976/redun/scripting.py#L78-L80
It would be nice to support passing env dictionary from script()
to that subrocess call. I agree that solution which requires prepending export
lines is not robust.
Our team actually has this implemented along with a couple other changes to redun's scripting:
script()
has an environ=
arg that works as suggested aboveA limitation to just generating the exports
directly in your scripts is that not all scripts use shell. E.g. some of our scripts use R too. So we would need to write this logic differently for each language we want to use. If instead the wrapper bash script does the env var exports itself, it generically works for any executable script that it calls.
If there's interest we can factor just the above changes into a PR here.
Edit: For clarity, I'm part of an external team that uses redun and we maintain a private fork, always happy to contribute stuff back.
It would be handy to pass a dict with environment variables: