Current behaviour looks in ~/.config/dask-ms/storage_options.json. This is a little awkward, as (a) this keeps S3 credentials openly listed in a file, and (b) this isn't necessarily a static user default -- the storage options will potentially differ from run to run.
Controlling this via environment variables would be useful. In particular, I would suggest
DASK_MS_STORAGE_OPTIONS_JSON="json text"
...as that makes it easy to set on a per-process basis from within stimela, and also keeps credentials from being written to any disk.
Current behaviour looks in
~/.config/dask-ms/storage_options.json. This is a little awkward, as (a) this keeps S3 credentials openly listed in a file, and (b) this isn't necessarily a static user default -- the storage options will potentially differ from run to run.Controlling this via environment variables would be useful. In particular, I would suggest
...as that makes it easy to set on a per-process basis from within stimela, and also keeps credentials from being written to any disk.