mrghg / agage-archive

Code for producing AGAGE archival files
MIT License
3 stars 0 forks source link

Documentation on the data selection files needed #48

Open mrghg opened 3 months ago

mrghg commented 3 months ago

Need to explain how to use the various data selection spreadsheets

mrghg commented 1 month ago

Also need to document the attributes and attributes_site json files, etc.

Amsoht commented 1 month ago

Draft outline from email on 2024-07-11

  1. Create individual instrument data files (not combined!):

    a) read in data_release_schedule.xlsx; for each site, each instrument has its own release_schedule. This means that for each instrument, each substance that should be released needs to be listed here.

    b) read in data_exclude.xlsx; again for each site, each instrument has its own info on which data to exclude

    c) apply this information on individual instruments to output final archive data.

  2. Create combined data files for each station:

    a) use information from 1a) and 1b)

    b) read in data_combination.xlsx; for each station, determine when to switch from one instrument to the next.

    c) apply this information on each station to output final combined archive data

Additional info: we are now using data_exclude to interleave two instruments. In this case, there’s an extra column you can add called “Combined_only”, which will only exclude data from the combined file, not the individual instrument file (it’s not necessarily bad data, just not preferred for some period).