search

Kitware / kwiver

Pulls Together Computer Vision Algorithms into Highly-Modular Run-Time Configurable Systems
Other
189 stars 83 forks source link

pipe-config ENV variables #1313

Open readicculus opened 3 years ago

readicculus commented 3 years ago

There may be a reason for how it is but would be consistent to have the pipe-config target compile environment variables as well and may be as simple as adding a call to pipe.process_env() from the pipe-config code.

It seems that anyways pipe-config will fail if required ENV variables are not defined when called but doesn't actually compile/inject them into the resulting pipeline.

I could see the decision here being wanting ENV to remain dyanmic(ie. pipe-runner runtime only) and settings to be used if compiling pipelines with pep-config but then it seems it shouldn't fail if there are missing ENV variables.

readicculus commented 3 years ago

Also might be helpful but will just briefly describe my use case. I'm creating a batch running tool to run a pipeline on many datasets and there are attributes specific to each dataset. Since I want the process to be reproducible I'm initially compiling one pipeline per dataset then will run them one by one, but the benefit here is the pipeline that created each set of results will be archived along with the results so that results can be reproduced/more transparency with parameters used etc...

I'll use settings instead of env here so not a major issue, just initially developed with env and then this issue/inconsistency came up so figured I'd share.