open-AIMS / ADRIA_matlab

Repository for the development of ADRIA: Adaptive Dynamic Reef Intervention Algorithms. ADRIA is a multi-criteria decision support tool set particularly useful for informing reef restoration and adaptation interventions.
1 stars 0 forks source link

Possible edge case in RCI? #77

Closed ConnectedSystems closed 2 years ago

ConnectedSystems commented 2 years ago

Initial trials with the RCI metric produce an entire result set (100 simulations in total) filled with 0.1

>> rci_issue_mwe  % this is the script provided below
Loading all coral types
Took 29.1309 seconds to run 100 simulations (0.29131 seconds per run)

>> all(all(all(all(RCI_test == 0.1))))

ans =

  logical

   1

% timesteps, sites, interventions, replicates
>> size(RCI_test)

ans =

    25   366    10    10

Now this could be partly due to the fact that I am making criteria weights constant (as you will see shortly), but I suspect this shouldn't be the case. I also think there may be an edge case.

In the link below, where the criteria are assessed against the 0.75 threshold, it seems nothing catches the case where the values are exactly 0.75 - all the checks are done for < or > 0.75

https://github.com/open-AIMS/ADRIA_repo/blob/369ddf78be125c6da0bb9905875d85a52aa403bc/ADRIAfunctions/Metrics/ReefConditionIndex.m#L105

@ryanheneghan could I get some help with this?

Below is a MWE which will work if you switch to the generalize-RCI-76 branch.

% Minimum working example for RCI issue (#76/#77)

rng(101)  % set random seed for reproducibility

N = 10;       % Number of scenarios
n_reps = 10;  % Number of replicate RCP scenarios

ai = ADRIA();

%% Parameter prep
% Collect details of available parameters
combined_opts = ai.parameterDetails();
sim_constants = ai.constants;

% Generate samples using simple monte carlo
% Create selection table based on lower/upper parameter bounds
sample_table = table;
for p = 1:height(combined_opts)
    a = combined_opts.lower_bound(p);
    b = combined_opts.upper_bound(p);

    selection = (b - a).*rand(N, 1) + a;

    sample_table.(combined_opts.name(p)) = selection;
end

sample_table.Guided(:) = 2;

% IPMF site selection specific options
% Disable certain aspects IPMF team are not concerned with
sample_table.Shadeyrs(:) = 0;
sample_table.Natad(:) = 0;

% JC informed values for criteria weighting
sample_table.coral_cover_high(:) = 0;
sample_table.coral_cover_low(:) = 1;
sample_table.seed_connectivity(:) = 1;
sample_table.shade_connectivity(:) = 0;
sample_table.wave_stress(:) = 0;
sample_table.heat_stress(:) = 1;
sample_table.seed_priority(:) = 0;
sample_table.shade_priority(:) = 0;
sample_table.deployed_coral_risk_tol(:) = 0;
sample_table.depth_min(:) = 5;
sample_table.depth_offset(:) = 5;

%% Load site specific data
ai.loadConnectivity('Inputs/Moore/connectivity/2015/moore_d3_2015_transfer_probability_matrix_wide.csv');
ai.loadSiteData('./Inputs/Moore/site_data/MooreReefCluster_Spatial_w4.5covers.csv', ["Acropora2026", "Goniastrea2026"]);

%% Scenario runs

tic
res = ai.run(sample_table, sampled_values=true, nreps=n_reps, collect_logs=["site_rankings"]);
Y = res.Y;  % get raw results, ignoring seed/shade logs
% ai.runToDisk(sample_table, sampled_values=true, nreps=n_reps, ...
%     file_prefix='./test', batch_size=4);
tmp = toc;

[~, ~, coral_params] = ai.splitParameterTable(sample_table);

% If saving results to disk
% Y = ai.gatherResults('./test', {@coralTaxaCover});
disp(strcat("Took ", num2str(tmp), " seconds to run ", num2str(N*n_reps), " simulations (", num2str(tmp/(N*n_reps)), " seconds per run)"))

% ReefConditionIndex

RCI_test = ReefConditionIndex(Y, @coralEvenness, @shelterVolume, @coralTaxaCover, coral_params);
ConnectedSystems commented 2 years ago

Closed thanks to fix that catches the edge case from @ryanheneghan

(see commit fcc47f2496ab919575f83ecb724dedb0d569b3fa and eb5c219df85762dd8e3f62ea864518e147d2373f)

ConnectedSystems commented 2 years ago

Thanks @ryanheneghan

ConnectedSystems commented 2 years ago

@ryanheneghan

FYI just in case its relevant to your investigations - values for juveniles seem to mostly span between 0 and 0.1, with 0.15 being fairly extreme.

This is with all values fluctuating though and the parameter bounds may be unrealistic. Might be worth seeing what the median value is if coral parameters are kept constant.

image