Simplified workflow which will break current examples

[ConnectedSystems]

While waiting on metric data structures to be finalized I've started on the long promised work to simplify the usage of ADRIA's run functions.

EDIT: Updated this post to cut down on fluff.

The proposed approach (not yet fully implemented):

% Create an ADRIA Interface object that holds all default values.
% This object is how one interacts with ADRIA through a number of convenience methods
% (some examples included here)

ai = ADRIA();  % Assign to `ai` (for ADRIA Interface) by convention

% Retrieve the table of parameter details including default values
% and their ranges (these are just parameters whose values can change)
all_params = ai.parameterDetails();

% Table of parameter bounds could also be retrieved (if needed)
% see explanation of distinction here: 
% https://github.com/open-AIMS/ADRIA_repo/blob/main/docs/src/dev_guide/parameter_interface.md
% bounds = ai.rawBounds();
% bounds = ai.sampleBounds();

% Individual parameter sets are kept as attributes of the ADRIA Interface
interv_params = ai.interventions;
criteria_weights = ai.criterias;
coral_params = ai.corals;

% ADRIA constants are not included with parameter details, but can also be
% retrieved
constants = ai.constants;

% If constants are to be overridden:
constants.con_cutoff = 0.2;
constants.max_settler_density_per_m2 = 4;
ai.constants = constants;

% Use the `all_params` variable to generate samples
% This function below doesn't exist, it's just a stand-in for whatever
% sampling method is used.
% Point is, `samples` should be a table of `N*D`,
% where `N` is number of simulations and `D` is number of parameters
sampled_values = someSamplingFunction(all_params, N=3);

% Or retrieve a parameter table of just default values (shape = 1*D).
default_params = ai.defaultParams();

% Individual parameter values can then be set directly
% Handy for optimization processes, for example.
modified_params = default_params;
modified_params.Natad = 0.2;
modified_params.Aadapt = 0.5;
modified_params.RCP = "26";

% Load connectivity data from some file
% If `cutoff` is not provided, ADRIA uses the value found in `constants`
ai.loadConnectivity('MooreTPMean.xlsx', cutoff=0.1);

% Run a single simulation with the above modified values
Y = ai.run(modified_params, sampled_values=false, nreps=8)

% Run a set of simulations with sampled values
Y = ai.run(sampled_values, sampled_values=true, nreps=8)

% Same as above, but run in batches and store results to disk (in netCDF format)
ai.runToDisk(sampled_values, sampled_values=true, nreps=8)

Y = % some function to read data in...

% Calculate specific metrics from results
some_metric = some_metric_function(Y)
another_metric = another_metric_function(Y)

% my preferred approach: collect a set of metrics by passing in an array of functions
a_struct_of_separate_metric_values = collectMetrics(Y, ...
    [@some_metric_function, @another_metric_function, @yet_another_metric])

As indicated in the code example above, the distinction between runADRIA and runScenario is abstracted away at the high-level - a single scenario is run simply by providing an input table with a single row of parameter values.

In the example above I've assumed users will never provide their own RCP/wave damage data, so the process of loading and inputting data is kept internal to ADRIA and hidden from users at this high-level.

Note as part of this change the alg_ind parameter is merged into interventionDetails(), thereby addressing #57.

Thoughts/comments/feedback appreciated.

[ConnectedSystems]

Hi @Rosejoycrocker

Please see this example which should give you an idea on how to update your own scripts

[Rosejoycrocker]

Hi @ConnectedSystems , Thanks for that, that's great. It seems like the units used for some of the parameters have changed also? (as Seed1 =15000). Are the new bounds for parameters etc now available in the default parameter table?

[ConnectedSystems]

Hi, yes all the run examples have been updated (in this branch) so please have a look through those.

Off the top of my head the full parameter detail table get be had with ai.parameterDetails()

Sorry for the brevity, I'm on my mobile as I've stepped out to grab a late lunch

[Rosejoycrocker]

Thanks @ConnectedSystems , enjoy your lunch :)

[KRNA01]

@ConnectedSystems, this is excellent - exactly what we need. I have spent the weekend thinking about how we generate a generalised structure for running different constellations of simulations. All in vain, because you were steps ahead of me - thank you.

Takuya, would you be able to show us tomorrow how you see this running, please? Also, please let us know what you need from the team to make this work.

With the business case in mind, a high priority for me (and hopefully the team) is to be able to run batches of simulations that allow us to analyse:

1. how different intervention R&D assumptions affect the performance of different metrics under optimised deployment. This is part of our task to inform R&D reprioritisation.

2. How different intervention intensities (e.g. seeding rates) and site-selection criteria combine to demonstrate optimised intervention deployments for Moore Reef and other clusters

There are more, but if we can nail these before end of Feb, I'd be happy.

[KRNA01]

Meant to extend this last reply to the whole team: @ConnectedSystems , @Rosejoycrocker, @BarbaraRobson and @veroniquelago

[Rosejoycrocker]

Sounds good @KRNA01, I've started to adapt my optimisation code to use the new ADRIA() interface

[ConnectedSystems]

Hi @Rosejoycrocker,

In case useful.

Let me know if you need some other things done.

Just in case, the interface implementation can be found here

% These map to the individual parameter detail tables.
interv_params = ai.interventions;
criteria_weights = ai.criterias;
coral_params = ai.corals;

% These get just the raw and sample bounds, respectively
% table with: name, lower_bound, upper_bound
raw_bnds = ai.raw_bounds
sample_bnds = ai.sample_bounds 

% Get the raw and sample default values as a parameter table
raw_values = ai.raw_defaults
sample_values = ai.sample_defaults

% To split a parameter table back into intervention, criteria, corals:
[interv_params, crit_params, coral_params] = ai.splitParameterTable(raw_values)

[Rosejoycrocker]

Thanks @ConnectedSystems. I'm wondering how to test my code when I've got it mostly worked out- at the moment I'm making changes on the optimisation_trials branch but this won't have the ADRIA interface functions- should I create a branch off your coral_samples branch or just wait to merge with main when your finished with developing that branch?

[ConnectedSystems]

Started writing an essay and then realised I could work something out for you real quick, give me a tick.

[Rosejoycrocker]

Thanks for that :) . I'll be away from my computer for a bit soon, but I'll be back later in the evening, so if I don't answer for a bit that is why.

[ConnectedSystems]

Please checkout coral-optimization-merge - this should have your optimization-trials code with the recent changes in coral-sampling.

As coral-sampling is a branch off 40corals, there should be no issues collapsing everything back into 40corals and then (when we're ready) main.

[Rosejoycrocker]

Thanks @ConnectedSystems, that's perfect!

[Rosejoycrocker]

Hey @ConnectedSystems, is ai.defaultParams no longer an object under the ADRIA class? I've got some code which tries to assign params = ai.defaultParams in the coral-optimisation-merge branch and its saying the object doesn't exist for the ADRIA class. Should I wait a bit for the code to get smoothed out before I have a go at integrating into the optimisation code? (or maybe I'm using the class in the wrong way ...)

[ConnectedSystems]

Hey Rose, when I first posted I hadn't actually implemented about half the code in the example snippet in the first post above - it was mainly for discussion to see if the team saw something missing or needed something else supported.

Right now I think everything is fairly stable. The only exception would be if something is missing or you want to suggest alternate names for the methods/properties.

Apart from slight adjustments to property/method names, the biggest difference between the example above and the actual implementation is that I realised using methods to access the underlying tables is a bit redundant - these you can access via the object properties instead.

I think what you're after is shown here

Have a look at the implementation as well which lists out what is actually available, and let me know if you need anything else supported.

I will earmark this Documentation Friday for writing this up.

[Rosejoycrocker]

Right, sorry, I wasn't sure how much was pseudo code, thanks for the clarification :)

[ConnectedSystems]

Just flagging that we could do something like:

ai.optimize(@some_optimization_method, @some_metric, some_additional_options)

May take a bit of work but possible.

[Rosejoycrocker]

Ok cool, I'll write the optimisation as external functions initially and then we can integrate them into ai

[ConnectedSystems]

I think we're all happy with how this got implemented at this stage so closing this issue.