search

NDCMS / lobster

A userspace workflow management tool for harnessing non-dedicated resources for high-throughput workloads.
MIT License
3 stars 14 forks source link

Timestamps in directory names cause errors when updating config #632

Open rebucci opened 6 years ago

rebucci commented 6 years ago

Including a timestamp in the directory name results in runtime errors when config.py is edited mid-run.

Example of timestamp:

project = 'PROJECT_'
version = datetime.datetime.now().strftime('%Y%m%d_%H%M’)
"hdfs://eddie.crc.nd.edu:19000/store/user/$USER/" + project + version,

When config.py is changed, the timestamp is updated and all outputs are routed to the new storage directory, which doesn't exist.

Proposed quick solution: Change the example configuration on the lobster webpage so that it does not use the timestamp. Optionally, include information about how lobster handles immutable parameters.