Exploring uncertainty and variability

[gclawson1]

Most of the advice I've received is to explore uncertainty and variability within my results. Given that, I've started brainstorming some ways to visualize this effectively.

From Stock et al. 2018

• I'm thinking something along the lines of this could be useful. Maybe something like most impacted cells (>90th quantile?) in 1, 2, 3, or 4 scenarios (different colors represent how many scenarios agree in each cell); least impacted cells (<25th quantile) in 1, 2, 3, or 4 scenarios; this will tell us the places where each scenario agrees on high or low impact areas

Showing standard deviation

• I have maps of standard deviation for each feed scenario, raw material, taxon group etc, so I could refer to these in some supplementary material

• something similar to this where I add the mean or median absolute deviation per latitude

Showing distribution of values

• I already have something like this, which gives a decent idea of distribution of values

[cottrellr]

Cool. I'm not sure the scenario is what you want to play with uncertainty wise. Most of the uncertainty comes from 1) the species distributions 2) the distribution of production activity (where the fishing and farming occurs) 3) the sensitivity of the species to the production ctivity 4) the variability in the assumed impacts within a cell among the species affected. The FCR and feed scenarios assume the same thing for each of these and so don't to my mind address the uncertainty, they manufacture change to ask the question we want. The standard deviation maps address number 4 above. If you looped through different extinction exponents and different sensitivity values for each species, each iteration could tell you something about whether the distribution still looks the same for X% of those iterations. This would address number 3. I assume computer power is a limiting factor at some point though. Nonetheless, you could present the same metric as Stock et al - the top quartile of impacted species and the least impacted quartile (you could choose an extinction threshold for the quartile) and then how the distribution of the species experiencing this extinction risk changes as you alter parameters?

I think points 1 and 2 are a data limitation and something we need to live with.

[gclawson1]

Yeah, the computation power is definitely a limitation here... running with different extinction coefficients and sensitivity values could get dicey with the amount of files, scenarios produced... So I would need to be careful about incorporating different values for these things.

Regarding the extinction coefficient, there is a ton of research that suggests that z = 0.25 is the accepted value for terrestrial species (Eyres et al. 2023; Duran et al. 2020; Pimm & Askins 1995; Ney-Nifle & Mangel 2000), so the pushback I could foresee with assumption is for marine species, but not so much for terrestrial species. I'm gonna take a look at the literature and try to see if there is any equivalent value for marine species, or if we could possibly justify using the same value for marine species...

[gclawson1]

Went ahead and took a look at high impact vs low impact agreement among all scenarios as a first pass

What I've done here is read in each scenario

• allocation types: mass, economic, ge
• diet types: plant or fish
• fcr types: regular or efficient

And for each global mean raster, calculated the cells that are in the top 95th quantile and in the bottom 5th quantile. Then from there I've summed across all rasters (12 of them) to see how many scenarios agree on high impact or low impact cells.

High impact:

• Cells with a 12 means that all scenarios have this cell as in the 95th quantile of impacts or above
• Cells with a 1 means that only one scenario has this cell as in the 95th quantile

Low impact:

• Same thing here, but with 5th quantile

You'll notice the area in the ocean off of the west coast of South America; this is likely due to trimmings catch happening there. In the economic allocations, trimmings impacts are highly discounted, which is why we see ~3-4 scenarios having low impacts there (the economic scenarios)

[gclawson1]

And here is where the models disagree on high impact vs low impact:

• a value of 1 means that this cell was classified as both high impact and low impact in at least 2 different scenarios
• a value of 0 means that it is either high impact or low impact, but not both

[cottrellr]

Fascinating stuff! The allocation strategies makes sense as a sensitivity - not sure about the FCR or diets as that is part of the results which in a way are their own sensitivity tests. The meaningful sensitivity analysis here should play around with assumptions underpinning the results (aside from two form we are deliberately changing). Re the extinction risk exponent, I think that makes sense to check whether there is such broader agreement in the ocean-based literature as there is on land. And if there is broad agreement then all good. I would say the species sensitivity is what could be played around with then? Rather than looping through heaps of values you could do it for a couple and compare e.g., what does assigning marginal impact a 0.25 or 0.75 do instead of a 0.5 for sensitivity? You don't necessarily have to do it for all species either - could do it for a handful (making sure to include, small and large habitats and a range of taxa).....

[cottrellr]

And here is where the models disagree on high impact vs low impact:

• a value of 1 means that this cell was classified as both high impact and low impact in at least 2 different scenarios
• a value of 0 means that it is either high impact or low impact, but not both

Cool, based on what you said earlier I'm guessing this all allocation rather than FCR? Using the diet doesn't make sense to me - we have manufactured them to be different and we're not presenting results in one map - it's always broken out by diet (and by efficiency). What I think needs identifying is, within a given map for e.g., marine diet, BAU efficiency, where is the uncertainty. The allocation does this nicely - yo assume economic but you could assume mass or energy. Then there the other aspects - see my point above on species' sensitivity.

[gclawson1]

Oh yeah, you're right. The main differences will be between the allocation types. So technically I would need to produce these for each diet and fcr scenario. e.g., I would have 3 categories (high or low impact in each allocation scenario) in the plots, rather than 12

Regarding the extinction coefficient, so far I haven't been able to really find anything on marine research. Seems like most people will use the IUCN categories to classify risk of extinction or not. I haven't been able to find any work that is calculating an extinction risk or persistence score like we are for marine species

[cottrellr]

Well, I would say at least one for each diet (as the cells are in physically different places because of differences in ingredients). You just need to show that there are parameters in your analyses that change the result presented. Goes to Chris’ point, these maps (across everything we do) are model output rather than fact. And we use multiple data sources here with cumulative error. So we need to show where the error comes from.

Re the extinction risk – it may be easier not to do the probability of persistence/extinction risk. What if you did proportion of impact habitat instead (i.e. the step before extinction risk)? One level fewer in uncertainty….

---

Richard S. Cottrell Research Fellow in Aquaculture Sustainability Institute for Marine and Antarctic Studies College of Sciences and Engineering University of Tasmania

Theme Co-Lead, Sustainable Futures and Planetary Health Centre for Marine Socioecology University of Tasmania

Size Ecology Labhttps://www.sizeecology.org/ | Centre for Marine Socioecologyhttps://marinesocioecology.org/themes/sustainable-futures-and-planetary-health/ Google Scholarhttps://scholar.google.com/citations?user=X1a9t90AAAAJ&hl=en&authuser=1 | ORCIDhttps://orcid.org/my-orcid?orcid=0000-0002-6499-7503 | @RichCottrell22https://twitter.com/RichCottrell22

From: Gage Clawson @.> Date: Wednesday, 27 March 2024 at 10:53 AM To: Sustainable-Aquafeeds-Project/feed_biodiv_impact_mapping @.> Cc: Richard Cottrell @.>, Comment @.> Subject: Re: [Sustainable-Aquafeeds-Project/feed_biodiv_impact_mapping] Exploring uncertainty and variability (Issue #16)

Oh yeah, you're right. The main differences will be between the allocation types. So technically I would need to produce these for each diet and fcr scenario. e.g., I would have 3 categories (high or low impact in each allocation scenario) in the plots, rather than 12

Regarding the extinction coefficient, so far I haven't been able to really find anything on marine research. Seems like most people will use the IUCN categories to classify risk of extinction or not. I haven't been able to find any work that is calculating an extinction risk or persistence score like we are for marine species

— Reply to this email directly, view it on GitHubhttps://github.com/Sustainable-Aquafeeds-Project/feed_biodiv_impact_mapping/issues/16#issuecomment-2021726634, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJK3YJB2QO6BMJOPN5FHK3TY2IJ75AVCNFSM6AAAAABFGHC246VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRRG4ZDMNRTGQ. You are receiving this because you commented.Message ID: @.***>

This email is confidential, and is for the intended recipient only. Access, disclosure, copying, distribution, or reliance on any of it by anyone outside the intended recipient organisation is prohibited and may be a criminal offence. Please delete if obtained in error and email confirmation to the sender. The views expressed in this email are not necessarily the views of the University of Tasmania, unless clearly intended otherwise.

[gclawson1]

I do have those proportions calculated, those values are reallllly small for the most part. Which I guess we have the same problem with extinction risk as well.

But you're right, the proportion of each habitat cell impacted is way easier to understand than extinction risk (and removes the extinction coefficient uncertainty). The only concerns I have with this are:

• Might make it a bit harder to visualize
• Does excluding the extinction coefficient assume linear impacts? The reason for calculating the extinction risk or persistence score is to assume a non-linear relationship between AOH loss and population persistence. In other words, the extinction risk metric captures cumulative effects, and allows us to derive implications on species populations from habitat loss. Would we lose that cumulative effect if we exclude the extinction coefficient (z)?
  • However, maybe losing the extinction risk wouldn't necessarily hinder our ability make claims about cumulative effects. Dave Williams work only looks at percentage of habitat lost. They linked their results to IUCN categories to derivate implications on species. Again, the problem here is that the majority of the aquamaps species are not IUCN listed, so we would have a massive Data Deficient category (which is common in these assessments anyways).

For context: The average proportion of habitat impacted in plant-dominant, BAU fcr, economic allocation is 0.00001099836, meaning 0.001% of each cell impacted, on average.

Average proportion impacted in each cell for that scenario

Zooming in on "high impact" (~2% of habitat impacted on average, around Norway I think)

[cottrellr]

Yep, I agree the concerns are valid.

To me though, the counter point is given the size of these extinction risks - linear or non-linear - it basically makes no difference (it definitely would if we were talking 10 or 20% of a habitat). Even assuming cumulative effects, these effects are vanishingly small. Further, because we are interested in the comparison between diets, we will focus on relative differences between the two and so for most of the paper (with perhaps the exception of the initial maps) this relative difference rather than the absolute difference is what is important.

When looking all livestock in later papers - perhaps this is when we can start making more absolute comparisons?

Ultimately, whatever we use will have critics on either side. The conservationists may claim that prob.persistence/ extinction risk is more meaningful because of cumulative effects and industry will be asking - "how do you reliably translate proportion of habitat to extinction risk - how reliable is that exponent?". We also have the challenge with assuming the exponent holds constant across land and sea as you show. How has Casey approached this in the ocean?

[gclawson1]

Yeah, I'm leaning towards using the proportion of habitat impacted, given that the values are so small regardless of metric. We can always just put the extinction risk stuff in a supplement.

Casey has always used species ranges, and intersected to find the amount or proportion of ranges impacted. He links it to IUCN categories too.

Also he calculates a "cumulative impact metric" "in a given location (i.e., grid cell) as the product of stressor intensity 𝑠𝑗 and vulnerability of that species to that stressor 𝑣𝑖𝑗", and then he sums impacts across all stressors to get cumulative impact. So he doesn't really incorporate a km2 estimate or anything like that. Its all rescaled 0-1 (based on the stressors, which are between 0-1)

[gclawson1]

Based on what we chatted about yesterday, I started exploring if the mean scales with the standard deviation, here are some results:

blue is fitted line, black is abline:

In this plot, each point represents a cell. So it appears that the trend is generally that the mean scales with the standard deviation, particularly for higher values. There is a lot more variation with small values.

I went ahead and calculated the coefficient of variation for this, and here is that:

For comparison, here are the standard deviations:

You'll notice that the areas with high CVs are pretty different from that of the sds, particularly on land. The Northern sea and ocean areas around Europe match pretty well between the CV and SD maps.

It'll be helpful to have both the sd and CV in the supplementary material, and I can cite both within the main manuscript.

[gclawson1]

Here's a look at the proportion of spp in each cell which are impacted, also something useful for the supplementary: