Updates to data combination and release schedule, June 2024

[mrghg]

Code changes:

• Removes duplicate timestamps if two overlapping instruments happen to coincide. Defaults to instrument with latest start date
• Error checking if an instrument is not included in release schedule

Data selection changes:

• Updates from Jens Mühle and Chris Harth on dates to switch between instruments (22/5/24). These have been applied to many instruments, based partly on Paul Krummel's MD to Medusa recommendations for CH3CCl3 and CFC-113
• Overlap for CMN from Jgor Arduini (17/5/24), but these may be subject to change following the AGAGE meeting
• Addition of TAC from Joe Pitt (22/5/24)

[joe-pitt]

Data selection changes:

• Added hcfc-124 to data_combination for ADS/Medusa at CGO, MHD and ZEP (but exclude ADS data for CGO and ZEP until confirmed)
• Added hcfc-141b to data_combination for JFJ
• Changed ADS/Medusa switch date for hfc-365mfc at CGO and MHD (Medusa record starts later for this gas)
• Minor change to ADS/Medusa switch date for all other gases at MHD (to start of Medusa record)
• Change ADS/Medusa switch date for hfc-134a and hfc-245fa at JFJ (later Medusa start date)

[joe-pitt]

Data selection changes:

• Include ZEP ADS data for hcfc-124 following email from Chris Lunder (2024-06-13)

[joe-pitt]

Data selection changes:

• Include CGO ADS data for hcfc-124 following email from Paul Krummel (2024-06-13)

[mrghg]

This last commit fixes a few inconsistencies in the TOB data. But important to note that I also changed the default naming for some variables, to address inconsistencies. E.g., benzene -> c6h6, etc.

[mrghg]

Last commit includes Paul Krummel's instrument changeover dates. His methodology is as follows:

NOTE – the current (June 2024) ALE and GAGE data in gccompare are on LOCAL time for each of the sites!

CH4 – GAGE/AGAGE transition when AGAGE starts. Transition from AGAGE to Picarro when Picarro starts. Also, want to use Picarro2 for whenever Picarro1 data are missing – maybe use Picarro2 whenever there is more than 12 hours of missing Picarro1 data?

CH3CCl3 – Ale/GAGE transition date selected for when records almost match – there is a largish offset prior to this date. GAGE/AGAGE transition date based on when the AGAGE precision improved considerably. There are also large jumps in the early AGAGE record that correspond to tank changes.

CFC-11 – have based the cutoff dates on CFC-11S. Both records have various steps which must be cal tank changes. Picked ALE/GAGE transition when the two records first matched relatively closely. GAGE/AGAGE transition when AGAGE started.

CFC-12 - Picked ALE/GAGE transition when the two records first matched relatively closely. GAGE/AGAGE transition when AGAGE started.

CFC-113 - GAGE/AGAGE transition when AGAGE started. Might revisit this someday …

CCl4 – initial AGAGE data quite noisy, pushed transition date forward by a couple months.

N2O – Dang the early N2O data are variable and ugly! Picked ALE/GAGE transition when the two records first matched relatively closely. GAGE/AGAGE transition when AGAGE started. **When we are completely happy with the Picarro N2O data we will set a transition date to move to that instrument – it will require a scaling factor between SIO-16 and NOAA-2006A scales.

ADS to Medusa transitions – mostly based on time when overlap looked the best. Most of them are when the Medusa starts. Some species have small steps at the transitions.

NOTE – use GCMD CHCl3 as the primary record – I changed this in the spreadsheet. This should be the case at ALL 5 core AGAGE sites that have GCMD instruments!

SF6 – GC ECD data from 2001 until transition date. Have placed the transition date in the GCMD column, but might need a separate one for GCECD?

H2 – switch from GC-MRD to GC-PDD when PDD data start

[mrghg]

Am going to close this PR, as it's become a bit of a mess. I'll try to be more disciplined about not mixing changes to the data files and code in the future.