LARMIP/AIS does not randomize samples

[bobkopp]

It appears that larmip/AIS goes through sequentially, applying model 1 to the first N, model 2 to the next N, etc. It should instead be applying the models in a random order, similar to ipccar5/glaciers, so it does not create structure in the sampling:

        # Randomly pulls time-series indices from the selection pool corresponding to samps_per_model for this method
        rnd_sample_idx = np.random.choice(time_series_idx, size=samps_per_model[method_idx], replace=False)

        # Removes the selected time-series indices from the selection pool for the next loop
        time_series_idx = np.setdiff1d(time_series_idx, rnd_sample_idx)

[AlexReedy]

we want the logic to be looking like this

NUMBER OF MODELS: 16 NSAMPS: 25 NUMBER OF SAMPLES PER MODEL: [1 2 2 2 2 1 1 2 2 2 1 1 2 1 1 2] CURRENT MODEL: PISM_PIK

Number of Samples: 1 Indices of Time-Series Data used for this Model: [24] For Sample Index [24] Ocean Model Index: 17 CURRENT MODEL: PS3D_PSU Number of Samples: 2 Indices of Time-Series Data used for this Model: [18 23] For Sample Index [18] Ocean Model Index: 4 For Sample Index [23] Ocean Model Index: 13 CURRENT MODEL: CISM_NCA Number of Samples: 2 Indices of Time-Series Data used for this Model: [4 3] For Sample Index [4] Ocean Model Index: 7 For Sample Index [3] Ocean Model Index: 7 CURRENT MODEL: PISM_AWI Number of Samples: 2 Indices of Time-Series Data used for this Model: [20 7] For Sample Index [20] Ocean Model Index: 15 For Sample Index [7] Ocean Model Index: 17 CURRENT MODEL: SICO_ILTS Number of Samples: 2 Indices of Time-Series Data used for this Model: [12 15] For Sample Index [12] Ocean Model Index: 5 For Sample Index [15] Ocean Model Index: 15 CURRENT MODEL: AISM_VUB Number of Samples: 1 Indices of Time-Series Data used for this Model: [0] For Sample Index [0] Ocean Model Index: 6 CURRENT MODEL: ISSM_JPL Number of Samples: 1 Indices of Time-Series Data used for this Model: [2] For Sample Index [2] Ocean Model Index: 12 CURRENT MODEL: FETI_ULB Number of Samples: 2 Indices of Time-Series Data used for this Model: [21 22] For Sample Index [21] Ocean Model Index: 6 For Sample Index [22] Ocean Model Index: 8 CURRENT MODEL: ISSM_UCI Number of Samples: 2 Indices of Time-Series Data used for this Model: [6 1] For Sample Index [6] Ocean Model Index: 8 For Sample Index [1] Ocean Model Index: 11 CURRENT MODEL: IMAU_UU Number of Samples: 2 Indices of Time-Series Data used for this Model: [ 5 17] For Sample Index [5] Ocean Model Index: 10 For Sample Index [17] Ocean Model Index: 2 CURRENT MODEL: BISI_LBL Number of Samples: 1 Indices of Time-Series Data used for this Model: [19] For Sample Index [19] Ocean Model Index: 0 CURRENT MODEL: UA_UNN Number of Samples: 1 Indices of Time-Series Data used for this Model: [13] For Sample Index [13] Ocean Model Index: 4 CURRENT MODEL: MALI_DOE Number of Samples: 2 Indices of Time-Series Data used for this Model: [ 8 10] For Sample Index [8] Ocean Model Index: 0 For Sample Index [10] Ocean Model Index: 1 CURRENT MODEL: PISM_VUW Number of Samples: 1 Indices of Time-Series Data used for this Model: [9] For Sample Index [9] Ocean Model Index: 10 CURRENT MODEL: GRIS_LSC Number of Samples: 1 Indices of Time-Series Data used for this Model: [14] For Sample Index [14] Ocean Model Index: 15 CURRENT MODEL: PISM_DMI Number of Samples: 2 Indices of Time-Series Data used for this Model: [11 16] For Sample Index [11] Ocean Model Index: 11 For Sample Index [16] Ocean Model Index: 14

[bobkopp]

That looks write to me. Check is to make sure whatever changes leave the 17-83rd percentile range essentially unchanged but if you plot samples (eg in 2100) vs sample number it’ll look unstructured.

---

From: AHReedy @.> Sent: Wednesday, December 20, 2023 2:05:03 PM To: radical-collaboration/facts @.> Cc: Bob Kopp @.>; Author @.> Subject: Re: [radical-collaboration/facts] LARMIP/AIS does not randomize samples (Issue #294)

we want the logic to be looking like this

NUMBER OF MODELS: 16 NSAMPS: 25 NUMBER OF SAMPLES PER MODEL: [1 2 2 2 2 1 1 2 2 2 1 1 2 1 1 2] CURRENT MODEL: PISM_PIK

Number of Samples: 1 Indices of Time-Series Data used for this Model: [24] For Sample Index [24] Ocean Model Index: 17 CURRENT MODEL: PS3D_PSU Number of Samples: 2 Indices of Time-Series Data used for this Model: [18 23] For Sample Index [18] Ocean Model Index: 4 For Sample Index [23] Ocean Model Index: 13 CURRENT MODEL: CISM_NCA Number of Samples: 2 Indices of Time-Series Data used for this Model: [4 3] For Sample Index [4] Ocean Model Index: 7 For Sample Index [3] Ocean Model Index: 7 CURRENT MODEL: PISM_AWI Number of Samples: 2 Indices of Time-Series Data used for this Model: [20 7] For Sample Index [20] Ocean Model Index: 15 For Sample Index [7] Ocean Model Index: 17 CURRENT MODEL: SICO_ILTS Number of Samples: 2 Indices of Time-Series Data used for this Model: [12 15] For Sample Index [12] Ocean Model Index: 5 For Sample Index [15] Ocean Model Index: 15 CURRENT MODEL: AISM_VUB Number of Samples: 1 Indices of Time-Series Data used for this Model: [0] For Sample Index [0] Ocean Model Index: 6 CURRENT MODEL: ISSM_JPL Number of Samples: 1 Indices of Time-Series Data used for this Model: [2] For Sample Index [2] Ocean Model Index: 12 CURRENT MODEL: FETI_ULB Number of Samples: 2 Indices of Time-Series Data used for this Model: [21 22] For Sample Index [21] Ocean Model Index: 6 For Sample Index [22] Ocean Model Index: 8 CURRENT MODEL: ISSM_UCI Number of Samples: 2 Indices of Time-Series Data used for this Model: [6 1] For Sample Index [6] Ocean Model Index: 8 For Sample Index [1] Ocean Model Index: 11 CURRENT MODEL: IMAU_UU Number of Samples: 2 Indices of Time-Series Data used for this Model: [ 5 17] For Sample Index [5] Ocean Model Index: 10 For Sample Index [17] Ocean Model Index: 2 CURRENT MODEL: BISI_LBL Number of Samples: 1 Indices of Time-Series Data used for this Model: [19] For Sample Index [19] Ocean Model Index: 0 CURRENT MODEL: UA_UNN Number of Samples: 1 Indices of Time-Series Data used for this Model: [13] For Sample Index [13] Ocean Model Index: 4 CURRENT MODEL: MALI_DOE Number of Samples: 2 Indices of Time-Series Data used for this Model: [ 8 10] For Sample Index [8] Ocean Model Index: 0 For Sample Index [10] Ocean Model Index: 1 CURRENT MODEL: PISM_VUW Number of Samples: 1 Indices of Time-Series Data used for this Model: [9] For Sample Index [9] Ocean Model Index: 10 CURRENT MODEL: GRIS_LSC Number of Samples: 1 Indices of Time-Series Data used for this Model: [14] For Sample Index [14] Ocean Model Index: 15 CURRENT MODEL: PISM_DMI Number of Samples: 2 Indices of Time-Series Data used for this Model: [11 16] For Sample Index [11] Ocean Model Index: 11 For Sample Index [16] Ocean Model Index: 14

— Reply to this email directly, view it on GitHubhttps://github.com/radical-collaboration/facts/issues/294#issuecomment-1864993691, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAJSFEAAMEGXVN4NUL4W543YKMZF7AVCNFSM6AAAAABATTEIF2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRUHE4TGNRZGE. You are receiving this because you authored the thread.Message ID: @.***>

[AlexReedy]

ok excellent getting the plots and percentiles now

[AlexReedy]

For coupling.ssp245 values for the new run are on the left, and AR6 values on the right FAIR Temp GSAT [LarmipNew/AR6] 2.52 (2.15-3.01) / 2.61 (2.19-3.12) Larmip_AIS_New [LarmipNew/AR6] 0.13 (0.05-0.28) / 0.14 (0.05-0.29)

The plot doesn't look that much less structured to me than it did before

[AlexReedy]

For Coupling.SSP585 FAIR TEMP GSAT 4.24 (3.62-5.02) Lamrip AIS 0.14 (0.06-0.34)

[AlexReedy]

Also noticing in the original script and by extension the new script that this loop doesn't actually assign i to anything. Are the ocean models being assigned to specific values int he time series data as well?

for i in np.arange(samps_per_model[model_idx]):

        # Choose a random forcing from the temperature data
        temp_idx = tempcount
        tempcount = tempcount + 1
        Temp = np.array(SAT[:,temp_idx])

[bobkopp]

Check with @pkjr002 - he had plots that looked more structured, we should understand if he was doing this in a different way.

The loop over I is just incrementing the temperature index -- we don't currently guarantee a equal number of draws of ocean model, though we could and it'd be in the spirit of what we do with ice models.

[bobkopp]

Also -- X axis on the plot should be sample index number, not temperature, which I think is what you are showing.

[AlexReedy]

[AlexReedy]

New plot with changes updated in todays meeting:

[bobkopp]

Ok, looks like there is still structure there.

[pkjr002]

Attached links are outputs from a notebook (global , local) that computes presence of pattern table for FACTS 1.1.1 (data from Alex) and also includes plots.

It appears that there is a strong pattern for larmip(AIS,W/EAIS,PEN), similar to the plot above. A weaker pattern appears to exists for bamber(EAIS), FittedISMIP(GIS) and local file of tlm(2070,2090).

[bobkopp]

Thanks @pkjr002! Could one of you rerun these modules with a different random seed parameter and evaluate whether these patterns persist?

[AlexReedy]

Sure Bob I can do this todaySent from my Verizon, Samsung Galaxy smartphone -------- Original message --------From: Bob Kopp @.> Date: 1/22/24 8:07 AM (GMT-05:00) To: radical-collaboration/facts @.> Cc: AHReedy @.>, Assign @.> Subject: Re: [radical-collaboration/facts] LARMIP/AIS does not randomize samples (Issue #294) Thanks @pkjr002! Could one of you rerun these modules with a different random seed parameter and evaluate whether these patterns persist?

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were assigned.Message ID: @.***>

[AlexReedy]

@pkjr002 do yo have the plot for tlm?

[pkjr002]

@AlexReedy in the (global) notebook, very last row (it is incorrectly labeled as SD)

[AlexReedy]

This is Larmip/AIS at n=2000 with seed=5, looks less periodic than before but the structure is still there it would seem.

[AlexReedy]

This is Larmip/AIS with the randomization fixes in (sorry the above didn't include the fixes we added). Same parameters.