search

spedas / pyspedas

Python-based Space Physics Environment Data Analysis Software
https://pyspedas.readthedocs.io/
MIT License
148 stars 57 forks source link

Externalize configuration #613

Open Beforerr opened 11 months ago

Beforerr commented 11 months ago

Now the code logic is highly coupled to the configuration part in load function. We can actually create a instrument_config.yamlfile and put the configuration part in the file.

By introducing this, it provides modularity, and gives users space when they only want a configuration file (use case: now downloading is not parallel and very slow if you want to download a lot of files, user may write a function but stil want the configuration)

Examples

# instrument_config.yaml
fgm:
  pathformat: 'mfi/mfi_{datatype}/%Y/wi_{datatype}_mfi_%Y%m%d_v??.cdf'
  masterfile: 'wi_{datatype}_mfi_00000000_v01.cdf'

swe:
  pathformat: 'swe/swe_{datatype}/%Y/wi_{datatype}_swe_%Y%m%d_v??.cdf'
  masterfile: 'wi_{datatype}_swe_00000000_v01.cdf'

sms:
  pathformat: 'sms/{datatype}/sms_{datatype}/%Y/wi_{datatype}_sms_%Y%m%d_v??.cdf'
  masterfile: 'wi_{datatype}_sms_00000000_v01.cdf'

# ... and similar entries for other instruments
Beforerr commented 11 months ago

Also I feel like that the load functions in different missions are more or less similar, but externalizing configuration we may provide a unified load function.