Work on BGC from 2024 LOV visit

[gmaze]

Place holder for work kicked off during 2024 LOV visit. Closes #361 Adressed #363

What's in this PR:

• Implement machinery to be able to post-process data according to user modes for BGC from the erddap
  • This includes a new transform_data_mode, that is doing what was doing the filter_data_mode method before, but this method was very badly named: merge PARAM with PARAM_ADJUSTED according to DATA_MODE values
  • The already existing filter_data_mode is now really doing what is was supposed to do: filter points according to data mode values
  • Refactor the postprocessing method in the data fetcher facade, so that it clearly implement user mode requirements for 'phy' and 'bgc' dataset
• Implement support for another index file, the auxiliary index in the `etc GDAC folder
• Implement nutrients and carbonate system predictions for the Mediterranean Sea using the CANYON-MED model. This brings in a new submodule in argopy to easily support argo accessor extensions.

Poke @Sauzede and @catsch

Breaking changes in the API:

• Argo xarray dataset attribute history is replaced by Processing_history
• change in signature for the xarray.argo accessor method filter_data_mode
• refactorised datafetcher prototype method filter_data_mode as transform_data_mode

[gmaze]

Hi folks, @catsch @Sauzede @grgdll @HCBScienceProducts,

Could I have some advice with regard to how can I easily identify all variables related to Radiometry in the Erddap list of variables below (the BGC dataset filling the Erddap is from synthetic profiles).

I would need this in order to filter out radiometry variables not in R data mode for the "Standard user mode", following recommendation in this issue https://github.com/euroargodev/argopy/issues/280#issuecomment-1617498660

List of variables to filter

List of erddap BGC dataset from synthetic profile files: ``` ['wmo_inst_type', 'data_type', 'date_update', 'pi_name', 'bbp470', 'bbp470_qc', 'bbp470_adjusted', 'bbp470_adjusted_qc', 'bbp470_adjusted_error', 'bbp532', 'bbp532_qc', 'bbp532_adjusted', 'bbp532_adjusted_qc', 'bbp532_adjusted_error', 'bbp700', 'bbp700_qc', 'bbp700_adjusted', 'bbp700_adjusted_qc', 'bbp700_adjusted_error', 'bbp700_2', 'bbp700_2_qc', 'bbp700_2_adjusted', 'bbp700_2_adjusted_qc', 'bbp700_2_adjusted_error', 'bisulfide', 'bisulfide_qc', 'bisulfide_adjusted', 'bisulfide_adjusted_qc', 'bisulfide_adjusted_error', 'cdom', 'cdom_qc', 'cdom_adjusted', 'cdom_adjusted_qc', 'cdom_adjusted_error', 'chla', 'chla_qc', 'chla_adjusted', 'chla_adjusted_qc', 'chla_adjusted_error', 'cndc', 'cndc_qc', 'cndc_adjusted', 'cndc_adjusted_qc', 'cndc_adjusted_error', 'cp660', 'cp660_qc', 'cp660_adjusted', 'cp660_adjusted_qc', 'cp660_adjusted_error', 'downwelling_par', 'downwelling_par_qc', 'downwelling_par_adjusted', 'downwelling_par_adjusted_qc', 'downwelling_par_adjusted_error', 'down_irradiance380', 'down_irradiance380_qc', 'down_irradiance380_adjusted', 'down_irradiance380_adjusted_qc', 'down_irradiance380_adjusted_error', 'down_irradiance412', 'down_irradiance412_qc', 'down_irradiance412_adjusted', 'down_irradiance412_adjusted_qc', 'down_irradiance412_adjusted_error', 'down_irradiance443', 'down_irradiance443_qc', 'down_irradiance443_adjusted', 'down_irradiance443_adjusted_qc', 'down_irradiance443_adjusted_error', 'down_irradiance490', 'down_irradiance490_qc', 'down_irradiance490_adjusted', 'down_irradiance490_adjusted_qc', 'down_irradiance490_adjusted_error', 'down_irradiance555', 'down_irradiance555_qc', 'down_irradiance555_adjusted', 'down_irradiance555_adjusted_qc', 'down_irradiance555_adjusted_error', 'down_irradiance665', 'down_irradiance665_qc', 'down_irradiance665_adjusted', 'down_irradiance665_adjusted_qc', 'down_irradiance665_adjusted_error', 'down_irradiance670', 'down_irradiance670_qc', 'down_irradiance670_adjusted', 'down_irradiance670_adjusted_qc', 'down_irradiance670_adjusted_error', 'doxy', 'doxy_qc', 'doxy_adjusted', 'doxy_adjusted_qc', 'doxy_adjusted_error', 'doxy2', 'doxy2_qc', 'doxy2_adjusted', 'doxy2_adjusted_qc', 'doxy2_adjusted_error', 'doxy3', 'doxy3_qc', 'doxy3_adjusted', 'doxy3_adjusted_qc', 'doxy3_adjusted_error', 'nitrate', 'nitrate_qc', 'nitrate_adjusted', 'nitrate_adjusted_qc', 'nitrate_adjusted_error', 'ph_in_situ_total', 'ph_in_situ_total_qc', 'ph_in_situ_total_adjusted', 'ph_in_situ_total_adjusted_qc', 'ph_in_situ_total_adjusted_error', 'pres', 'pres_qc', 'pres_adjusted', 'pres_adjusted_qc', 'pres_adjusted_error', 'psal', 'psal_qc', 'psal_adjusted', 'psal_adjusted_qc', 'psal_adjusted_error', 'temp', 'temp_qc', 'temp_adjusted', 'temp_adjusted_qc', 'temp_adjusted_error', 'turbidity', 'turbidity_qc', 'turbidity_adjusted', 'turbidity_adjusted_qc', 'turbidity_adjusted_error', 'up_radiance412', 'up_radiance412_qc', 'up_radiance412_adjusted', 'up_radiance412_adjusted_qc', 'up_radiance412_adjusted_error', 'up_radiance443', 'up_radiance443_qc', 'up_radiance443_adjusted', 'up_radiance443_adjusted_qc', 'up_radiance443_adjusted_error', 'up_radiance490', 'up_radiance490_qc', 'up_radiance490_adjusted', 'up_radiance490_adjusted_qc', 'up_radiance490_adjusted_error', 'up_radiance555', 'up_radiance555_qc', 'up_radiance555_adjusted', 'up_radiance555_adjusted_qc', 'up_radiance555_adjusted_error', 'latitude', 'longitude', 'position_qc', 'time', 'time_qc', 'direction', 'platform_number', 'platform_type', 'cycle_number', 'config_mission_number', 'data_centre', 'profile_bbp470_qc', 'profile_bbp532_qc', 'profile_bbp700_2_qc', 'profile_bbp700_qc', 'profile_bisulfide_qc', 'profile_cdom_qc', 'profile_chla_qc', 'profile_cndc_qc', 'profile_cp660_qc', 'profile_down_irradiance380_qc', 'profile_down_irradiance412_qc', 'profile_down_irradiance443_qc', 'profile_down_irradiance490_qc', 'profile_down_irradiance555_qc', 'profile_down_irradiance665_qc', 'profile_down_irradiance670_qc', 'profile_downwelling_par_qc', 'profile_doxy_qc', 'profile_doxy2_qc', 'profile_doxy3_qc', 'profile_nitrate_qc', 'profile_ph_in_situ_total_qc', 'profile_pres_qc', 'profile_psal_qc', 'profile_temp_qc', 'profile_turbidity_qc', 'profile_up_radiance412_qc', 'profile_up_radiance443_qc', 'profile_up_radiance490_qc', 'profile_up_radiance555_qc'] ```

[HCBScienceProducts]

All variables that contain "up_radiance", "downwelling_par" or "down_irradiance" are radiometry-related. "up_radiance" and "down_irradiance" are followed by 3 numerical digits that give the associated wavelength in nm, if you want to refine your filter.

[gmaze]

awesome ! thanks @HCBScienceProducts !

[gmaze]

one more question @catsch @Sauzede @grgdll @HCBScienceProducts :

what about CDOM ? it's all in real time data mode, so how can we consider it in "standard" user mode ?

[catsch]

We are presently waiting for new calibration factors for CDOM or a list of QC=4 from Seabird so presently, it should not be used at all, more to come at the next ADMT...

[gmaze]

so presently, it should not be used at all,

thanks @catsch !

[gmaze]

do you think we can keep Radiometry in the 'research' user mode ?

[catsch]

Most of the time, Radiometry is good data (in R mode)
and Radiometry exists in D and (1,2,5,8) is Ok Do I answer the question ?

[HCBScienceProducts]

do you think we can keep Radiometry in the 'research' user mode ?

Yes to radiometry both in 'research' mode with selection like all other variables (only D-mode, QC flags '1', '5', or '8').

On radiometry in 'standard' mode @.***): Should this rather be R-mode or A-mode or D-mode with QC flags '1', '2', '5', or '8' (instead of R-mode only). I know there's been some DMQC on radiometry done, i.e., D-mode exist (and are better data than the R-mode data).

On CDOM: No separate treatment on the argopy side should be needed for the 'standard' mode but can be treated like all other variables (no R-mode but only A-mode or D-mode, QC flags '1', '2', '5', or '8'). The CDOM issues are to be resolved on the Argo data management side. Once solved, it may eventually transition from today's "only R-mode and better not be used" towards A-mode or D-mode and can then appear just like any other variable. Same for the 'research mode': No separate treatment of CDOM (or radiometry) needed.