Prototype next-gen fingerprinting module within emulandice2

[bobkopp]

Several modules, in their post-processing stage, use a common set of GRD fingerprints to turn global contributions into local contributions. Consider separating this out into a module that could run in a step before the integration step. This would make it easier to build in more sophisticated approaches to GRD effects.

[bobkopp]

From GMD paper:

GRD processes in current FACTS modules are currently based on a library of scaling factors (sometimes called ‘fingerprints’) applied to ice sheet, glaciers, and land water storage projections in each module’s post-processing step. Such a library approach is most appropriate for glaciers. For the larger ice sheets, however, the locus of ice mass change can significantly affect the GRD spatial pattern (Larour et al., 2017; Mitrovica et al., 2018). This variability could be incorporated into FACTS by combining more spatially resolved ice sheet emulation with a new integration module that includes an online GRD solver (e.g., Larour et al., 2020).

In addition, in some cases (for example WAIS loss, where the underlying mantle is low viscosity) viscoelastic deformation, not just elastic deformation, may be relevant.

[bobkopp]

@AlexReedy consider prototyping this in v1.2 in association with the emulandice2 module

[bobkopp]

Specifically, the emulandice2 postprocess script should be written such that (1) it uses netcdf files from the project stage, rather than pickle files, and (2) is sufficiently generic that in principle (with the addition of a few attribute tags to the netcdf files outputted in the project stage) it could be applied to the output of any ice/groundwater project script.

Once successfully working as an emulandice2 postprocess step, we can discuss the merits of turning this into its own module run in an experiment step between the sea-level and total steps.

[bobkopp]

There are going to be situations in which you want to implement fingerprinting slightly differently (different fingerprints/regions) for different modules, and others when you want a uniform approach across module. Thus the bulk of the fingerprinting code should be in the shared library, allowing it to be invoked either from postprocessing scripts or from a grd/fingerprint module.

[bobkopp]

Emulandice2_postprocess appears to work and could be the basis of developing this shared functionality.

[bobkopp]

This looks useful if we want to add fingerprint calculation capability (not just lookup): https://github.com/da380/SLReciprocityGJI/blob/main/MakeFigures.ipynb

[bobkopp]

My suggestion is toq write this such that it can be run in two modes; in 1 mode, probably the most common mode, it matches barystatic component (glacier/ice sheet/lws) global outputs with precomputed fingerprints from a data file and multiplies to produce local sea level files. In a second mode, it adds a prior (preprocessing?) task that computes those fingerprints from data files that have load patterns. The Python code linked above could be used for this task.

[victor-malagon]

Hi @bobkopp and @AlexReedy. Is emulandice2_postprocess capable of handling different fingerprints? Or does it need to be adjusted? I tried to run the glacier fingerprints with emulandice2 and although I managed to get files for the contribution from each region (19 in total) they all look the same as the global. Also The localsl file seems to have no samples at all.

I also tried to run the new GrIS fingerprints, getting the following error:

_Traceback (most recent call last):   File "emulandice_project.py", line 194, in     not(args.no_rebase),   File "emulandice_project.py", line 16, in emulandiceproject     raise Exception("Number of regions and emulator files must be the same") Exception: Number of regions and emulator files must be the same

@tamsinedwards only sent one .RData file for GrIS, but emulandice_project expects one per region as we have for the glaciers.

[tamsinedwards]

Hi Victor, thanks for this.

It seems strange the glacier fingerprints don't work, as the regional output netcdf files are equivalent to emulandice1.

I also can't imagine why it would want an emulator build file (.RData) for each region - it shouldn't be using this file at all for the regional part of predictions, it should be using the regional netcdf output files. Are you sure it means .RData when it says "emulator files", not the output netcdfs?

Perhaps it's still expecting 1 region for GIS (as in emulandice1) but is complaining because you are giving it 6?

Ah, also, remember I said that emulandice2 writes an "ALL" file as well as the regional files - so there needs to be a mechanism to tell it which one of the output files to ignore. I could stop writing the ALL files as a hack, but they would be useful for most users.

Hope that makes sense?

Thanks, Tamsin

[bobkopp]

@victor-malagon You will have to adjust the grd_fingerprint.yml file that maps regions to fingerprints. Please read the opening comment for the postprocess script.

[bobkopp]

@victor-malagon If the localsl file is empty, the postprocess script is not running, and you and @AlexReedy will have to determine whether that's because the script I prototyped for you all doesn't work or because of the install on your system. Are the fingerprints being uploaded correctly? Does the task have any associated error messages?

Fingerprints will not affect globalsl output files.

[victor-malagon]

@bobkopp Thanks for this.

For GrIS: I did uploaded the new fingerprints to modules-data and updated the grd_fingerprintmap.yml file in emulandice2/shared as follows:

GRD Fingerprint Map

This file contains the fingerprint map for GRD Placed within modules for debugging purposes Should go to modules-data when finished

AIS: AIS.ALL: fingerprint: FPRINT/fprint_wais.nc

GrIS: GIS.RGI01: fingerprint: FPRINT/fprint_gis01.nc GIS.RGI02: fingerprint: FPRINT/fprint_gis02.nc GIS.RGI03: fingerprint: FPRINT/fprint_gis03.nc GIS.RGI04: fingerprint: FPRINT/fprint_gis04.nc GIS.RGI05: fingerprint: FPRINT/fprint_gis05.nc GIS.RGI06: fingerprint: FPRINT/fprint_gis06.nc

And I got the following error:

Traceback (most recent call last): File "emulandice_project.py", line 194, in not(args.no_rebase), File "emulandice_project.py", line 16, in emulandice_project raise Exception("Number of regions and emulator files must be the same") Exception: Number of regions and emulator files must be the same

I thought, as is in glaciers pipeline.yml, I also needed to include regions in the pipeline file of GrS. So I wrote:

project: task1: python_dependencies: "xarray dask numpy scipy" executable: "python3" script: "emulandice_project.py" arguments:

• --ice_source
• GIS
• --region
• RGI01
• RGI02
• RGI03
• RGI04
• RGI05
• RGI06
• --emu_file
• emu_file/GIS_ALL_CISM_pow_exp_01_EMULATOR.RData . . .

But FACTS complained:

_Traceback (most recent call last): File "emulandice_project.py", line 194, in not(args.no_rebase), File "emulandice_project.py", line 16, in emulandiceproject raise Exception("Number of regions and emulator files must be the same") Exception: Number of regions and emulator files must be the same

So I probably shouldn't have done so. If I don't edit de pipeline file, FACTS run successfully but I only get one global GrIS file (GrIS_ALL_globalsl.nc) a no regions.

[victor-malagon]

@bobkopp as for glaciers, I'll check with @AlexReedy the emulandice_postprocess.py file.

Cheers

[bobkopp]

@victor-malagon the number of emu_file parameters passed and region parameters passed needs to be the same. See the glacier pipeline for an example.

[tamsinedwards]

Hi @victor-malagon,

There are now 11 AIS regions, so it shouldn't be AIS.ALL as a heading with a single fingerprint wais.nc (or wais/eais/pen as in your original comment I think), but these new region names: ASE, EAIS1:7, PEN, WAIS1:2.

i.e. emulandice2 now outputs 11 regional netcdfs like this:

AIS_ALL_ssp585_2300_ASE_globalsl.nc
AIS_ALL_ssp585_2300_EAIS1_globalsl.nc AIS_ALL_ssp585_2300_EAIS2_globalsl.nc AIS_ALL_ssp585_2300_EAIS3_globalsl.nc AIS_ALL_ssp585_2300_EAIS4_globalsl.nc AIS_ALL_ssp585_2300_EAIS5_globalsl.nc AIS_ALL_ssp585_2300_EAIS6_globalsl.nc AIS_ALL_ssp585_2300_EAIS7_globalsl.nc AIS_ALL_ssp585_2300_PEN_globalsl.nc AIS_ALL_ssp585_2300_WAIS1_globalsl.nc AIS_ALL_ssp585_2300_WAIS2_globalsl.nc

plus the "ALL" file.

So I imagine you need to write AIS.ASE, AIS.WAIS1 etc (not "AIS.ALL") as script headers for each of Aimee's 11 fingerprint file names? Making sure the right names match up. We can double-check the matching once the fractions are finalised though.

And the GIS regions are not called RGI01 like the glacier regions, they are called NW, SW etc so these headers should be "GIS.NW", "GIS.SW" etc. However, on writing the output filenames here I realise one is missing due to a small typo, sorry:

GIS_ALL_ssp585_2300_NW_globalsl.nc GIS_ALL_ssp585_2300_NO_globalsl.nc GIS_ALL_ssp585_2300_CW_globalsl.nc - was missing because over-written by another GIS_ALL_ssp585_2300_NE_globalsl.nc GIS_ALL_ssp585_2300_SW_globalsl.nc GIS_ALL_ssp585_2300_SE_globalsl.nc

plus the "ALL" file.

I've just fixed and pushed the change.

Tamsin

[tamsinedwards]

P.S. If my regional naming conventions are annoying, happy to move to something else (but not to RGI for ice sheets, as that stands for Randolph Glacier Inventory).

[bobkopp]

BTW @victor-malagon -- let's create a new grd_fingerprint2_data.tgz for emulandice2, and adjust pipeline files accordingly, rather than modifying the existing data file. We don't want to do something that risks changing output from other modules unintentionally.

[victor-malagon]

@bobkopp

@victor-malagon the number of emu_file parameters passed and region parameters passed needs to be the same. See the glacier pipeline for an example.

Yes, exactly, but I do not have separate emu_file's for GrI regions, as I do for glaciers, I only have one. @tamsinedwards has emulandice2 changed in a way that these emu files are created internally?

Also, @tamsinedwards I am not currently running AIS, that's why I haven't included other AIS regions in the grd_fingerprintmap file. I only have fingerprints for GrIS at the moment, that's why I'm only testing GrIS.

Creating a separate grd_fingerprint_data for emulandice2 makes sense.

[tamsinedwards]

If we really, really need emulandice to have a different build file for each region of the ice sheets, like for the glaciers, that's in theory doable. But it would be extremely inefficient: FACTS would then run emulandice2 15 duplicate times (5 GIS + 10 AIS), with the only difference being multiplication of the output by a different fraction....

Somehow the region of the emulator (GIS.ALL) needs to be able to be different to the region of the predictions (GIS.NW, ...). Even if I do emulate the 3 AIS sectors individually in future, the predicted regions will be sub-divided again.

Is that OK?

[victor-malagon]

@tamsinedwards

P.S. If my regional naming conventions are annoying, happy to move to something else (but not to RGI for ice sheets, as that stands for Randolph Glacier Inventory).

My bad, I thought RGI was just a abbreviation for region. I'll follow your conventions from now on.

I guess we don't need a build file for each region, but then we have to tailor FACTS to handle this way of considering regions. Is emulandice2 supposed to output globalsl nc files for each GrIS region? I only see one ALL nc file when I run FACTS with the new build files, so I'm wondering if we need to change something in the pipeline (or other files) to get those. Once we have those it should be easy to convert them to local with the fingerprints.

[tamsinedwards]

Can you check the code update went through OK? It makes the ALL and regional files in a single loop in R/write_data.R, so it shouldn't be able to make only "ALL" any more.

[tamsinedwards]

@bobkopp Does that mean FACTS re-runs emulandice2 to predict each ice sheet region separately when running in RSLR mode? (i.e. the massive duplication I mentioned).

Or is it able to run emulandice2 once to predict for the ice sheet, then use the separate regional output files that are generated for the RSLR part?

[bobkopp]

@victor-malagon @tamsinedwards I don't see any reason why having one region (ALL) and one emu file specified for the project step but multiple nc files passed to the postprocess step shouldn't work. You should just need to make sure the output files are correctly named in the pipeline file.

[bobkopp]

Correction: the regional files should only need to be named in the postprocess step, since the Python code will not be adding the regions together in the project step to produce the ALL file.

[victor-malagon]

Thanks for this! I think I figured out the nc file naming and pipeline structure. I am now getting regional output for Glaciers and GrIS (the whole ice sheet as one region). I'll send some diagnostic plots via email soon.

Running different GrIS or AIS regions still does not work though. I think this has to do with the region attribute in the nc files being output by emulandice2. FACTS matches the fingerprints based on the name of that attribute, and it looks like all regions have the ALL attribute. For example, for GrIS we would expect that attribute to be NW for the NW nc file, but it's ALL instead, and this happens with all nc files. So different regions (NE, NO, NW, SW, and SE) are all assigned the ALL fingerprint instead of individuals FPs. This also happens to AIS. @tamsinedwards could you correct emulandice2 so it assigns the correct attribute region for individual nc files?

Another question regarding GrIS regions. I see five regions coming form emulandice2 (NE, NO, NW, SW, and SE), but Aimee gave me six fingerprints including a CW region. Is emulandice2 missing one region or did Aimee give me an extra one?

[tamsinedwards]

Hi Victor,

I can change the attributes, no problem – sorry for this.

Should be six files: did you update emulandice2 since Monday? I fixed this bug then (one file was over-written due to a typo). I’ve pushed another change since then too: GIS now has “real” fraction values (which are actually quite close to the “dummy” ones I estimated from raw data files).

Best wishes, Tamsin

From: Víctor Malagón Santos @.> Date: Monday, 27 May 2024 at 09:06 To: radical-collaboration/facts @.> Cc: Tamsin Edwards @.>, Mention @.> Subject: Re: [radical-collaboration/facts] Prototype next-gen fingerprinting module within emulandice2 (Issue #139)

Thanks for this! I think I figured out the nc file naming and pipeline structure. I am now getting regional output for Glaciers and GrIS (the whole ice sheet as one region). I'll send some diagnostic plots via email soon.

Running different GrIS or AIS regions still does not work though. I think this has to do with the region attribute in the nc files being output by emulandice2. FACTS matches the fingerprints based on the name of that attribute, and it looks like all regions have the ALL attribute. For example, for GrIS we would expect that attribute to be NW for the NW nc file, but it's ALL instead, and this happens with all nc files. So different regions (NE, NO, NW, SW, and SE) are all assigned the ALL fingerprint instead of individuals FPs. This also happens to AIS. @tamsinedwardshttps://github.com/tamsinedwards could you correct emulandice2 so it assigns the correct attribute region for individual nc files?

Another question regarding GrIS regions. I see five regions coming form emulandice2 (NE, NO, NW, SW, and SE), but Aimee gave me six fingerprints including a CW region. Is emulandice2 missing one region or did Aimee give me an extra one?

— Reply to this email directly, view it on GitHubhttps://github.com/radical-collaboration/facts/issues/139#issuecomment-2132892450, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AK4YYNWJW6NCKH3RLYIVAZTZELSQPAVCNFSM6AAAAAAS2AKNU2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZSHA4TENBVGA. You are receiving this because you were mentioned.Message ID: @.***>

[tamsinedwards]

@victor-malagon I've changed the netcdf attributes (region, and also source) - let me know if it still doesn't work.

[victor-malagon]

@tamsinedwards I can see the amended region attribute, but I still don't see the CW region. I see six files (including an ALL file), but shouldn't I see seven files?

1. NE;
2. NO;
3. NW;
4. SW;
5. SE; 6. CW;
6. ALL

[tamsinedwards]

Hi Victor, I'm so sorry, this is my fault. I forgot to send you new build files when I fixed that error - as the bug was in the build not predict stage of emulandice2.

Will email the build file now, and the AIS one very soon after.

Tamsin

[victor-malagon]

Hi all,

Fingerprints and regional mass projections are working fine now. A few last checks and tweaks though:

@bobkopp This is what the glacier regions look like in the grd_fingerprintmap file. It was like this as default (i.e., somebody wrote this before, I am guessing it was you). I am unsure this is the correct match between regions and fingerprints for glaciers. Could you please confirm whether these are correct?

glaciers: GLA.RGI01: fingerprint: FPRINT/fprint_glac11.nc GLA.RGI02: fingerprint: FPRINT/fprint_glac12.nc GLA.RGI03: fingerprint: FPRINT/fprint_glac14.nc GLA.RGI04: fingerprint: FPRINT/fprint_glac13.nc GLA.RGI05: fingerprint: FPRINT/fprint_gis.nc GLA.RGI06: fingerprint: FPRINT/fprint_glac1.nc GLA.RGI07: fingerprint: FPRINT/fprint_glac2.nc GLA.RGI08: fingerprint: FPRINT/fprint_glac10.nc GLA.RGI09: fingerprint: FPRINT/fprint_glac3.nc GLA.RGI10: fingerprint: FPRINT/fprint_glac5.nc GLA.RGI11: fingerprint: FPRINT/fprint_glac9.nc GLA.RGI12: fingerprint: FPRINT/fprint_glac8.nc GLA.RGI13: fingerprint: FPRINT/fprint_glac6.nc GLA.RGI14: fingerprint: FPRINT/fprint_glac7.nc GLA.RGI15: fingerprint: FPRINT/fprint_glac7.nc GLA.RGI16: fingerprint: FPRINT/fprint_glac15.nc GLA.RGI17: fingerprint: FPRINT/fprint_glac16.nc GLA.RGI18: fingerprint: FPRINT/fprint_glac17.nc GLA.RGI19: fingerprint: FPRINT/fprint_wais.nc

[victor-malagon]

@tamsinedwards @AlexReedy

For the 2150 projections: I do get regional output for glaciers up to 2150 with the extrapolation that was done by @AlexReedy for the global mean deliverable, but not for Greenland (only up to 2100). I guess this has to do with the way different regions are treated for GrIS, which is different than for glaciers. GrIS nc files (NW, CW...) are produced within the R scripts in emulandice2, and these go directly to emulandice_postprocess and do not undergo the extrapolation which is done in emulandice_project. So, although I do get GrIS global mean projections up to 2150 in GrIS_ALL_globalsl.nc, that's not the case for GrIS regions (e.g., GrIS_NE_globalsl.nc) and their local estimates once multiplied by the fingerprints. How should we fix this? Should we:

1) Apply the same extrapolation to individual regions.

2) Split GrIS_ALL_globalsl.nc into different regions after it's been extrapolated in emulandice_project.

Or perhaps you have better suggestions.

[bobkopp]

Hi Victor,

If you have not touched the mapping for the glaciers, it should be correct, but I would encourage you to map the fingerprints to confirm.

---

From: Víctor Malagón Santos @.> Sent: Thursday, June 6, 2024 10:24:58 AM To: radical-collaboration/facts @.> Cc: Bob Kopp @.>; Mention @.> Subject: Re: [radical-collaboration/facts] Prototype next-gen fingerprinting module within emulandice2 (Issue #139)

Hi all,

Fingerprints and regional mass projections are working fine now. A few last checks and tweaks though:

@bobkopphttps://github.com/bobkopp This is what the glacier regions look like in the grd_fingerprintmap file. It was like this as default (i.e., somebody wrote this before, I am guessing it was you). I am unsure this is the correct match between regions and fingerprints for glaciers. Could you please confirm whether these are correct?

glaciers: GLA.RGI01: fingerprint: FPRINT/fprint_glac11.nc GLA.RGI02: fingerprint: FPRINT/fprint_glac12.nc GLA.RGI03: fingerprint: FPRINT/fprint_glac14.nc GLA.RGI04: fingerprint: FPRINT/fprint_glac13.nc GLA.RGI05: fingerprint: FPRINT/fprint_gis.nc GLA.RGI06: fingerprint: FPRINT/fprint_glac1.nc GLA.RGI07: fingerprint: FPRINT/fprint_glac2.nc GLA.RGI08: fingerprint: FPRINT/fprint_glac10.nc GLA.RGI09: fingerprint: FPRINT/fprint_glac3.nc GLA.RGI10: fingerprint: FPRINT/fprint_glac5.nc GLA.RGI11: fingerprint: FPRINT/fprint_glac9.nc GLA.RGI12: fingerprint: FPRINT/fprint_glac8.nc GLA.RGI13: fingerprint: FPRINT/fprint_glac6.nc GLA.RGI14: fingerprint: FPRINT/fprint_glac7.nc GLA.RGI15: fingerprint: FPRINT/fprint_glac7.nc GLA.RGI16: fingerprint: FPRINT/fprint_glac15.nc GLA.RGI17: fingerprint: FPRINT/fprint_glac16.nc GLA.RGI18: fingerprint: FPRINT/fprint_glac17.nc GLA.RGI19: fingerprint: FPRINT/fprint_wais.nc

— Reply to this email directly, view it on GitHubhttps://github.com/radical-collaboration/facts/issues/139#issuecomment-2152675837, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAJSFEBFRKH7ZSW3K36CMUDZGBWLVAVCNFSM6AAAAAAS2AKNU2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJSGY3TKOBTG4. You are receiving this because you were mentioned.Message ID: @.***>

[victor-malagon]

@tamsinedwards @AlexReedy

For the 2150 projections: I do get regional output for glaciers up to 2150 with the extrapolation that was done by @AlexReedy for the global mean deliverable, but not for Greenland (only up to 2100). I guess this has to do with the way different regions are treated for GrIS, which is different than for glaciers. GrIS nc files (NW, CW...) are produced within the R scripts in emulandice2, and these go directly to emulandice_postprocess and do not undergo the extrapolation which is done in emulandice_project. So, although I do get GrIS global mean projections up to 2150 in GrIS_ALL_globalsl.nc, that's not the case for GrIS regions (e.g., GrIS_NE_globalsl.nc) and their local estimates once multiplied by the fingerprints. How should we fix this? Should we:

1. Apply the same extrapolation to individual regions.
2. Split GrIS_ALL_globalsl.nc into different regions after it's been extrapolated in emulandice_project.

Or perhaps you have better suggestions.

Hi all,

I fixed this by doing option 1 in the end. I compared the sum of the global mean from each region to the global mean from GrIS_ALL and they match, so the method works ok. I am doing the extrapolation in emulandice_postprocess though, because that's where the GrIS nc_files from all regions are being loaded. This is probably not ideal, as the extrapolation should be part of _project and not _postprocess. Happy to explore other ways to do this if necessary.

Cheers