search

OHI-Science / ohicore

Ocean Health Index - R library of core functions
http://ohi-science.org/ohicore
GNU General Public License v2.0
19 stars 19 forks source link

SPP: extract IUCN extinction status and spatial downloads #37

Closed bbest closed 10 years ago

bbest commented 10 years ago
bbest commented 10 years ago

adding initial NCEAS-SpeciesDiversity_v2014 with IUCN status extraction in https://github.com/OHI-Science/ohiprep/pull/17

bbest commented 10 years ago

Hi Jamie,

The problem before with plotting the species probabilities was due to the merge() function reordering the rows, so replacing the raster data was not properly aligned. Here's a little script plot_sp_probability.R to accurately associate the AquaMaps probability cells of a given species. Particularly:

# substitute values of raster cid with spp_cells' probability
r_sp = subs(r, sp_cells[,c('cid','probability')], by='cid', which='probability', subsWithNA=T)

BB

plot-sp-probability_Eubalaena-glacialis.png

bbest commented 10 years ago

updated script in progress with https://github.com/bbest/ohiprep/commit/d384516d084c6cbabfc6de7e3f9d75f720aa66d3

bbest commented 10 years ago

Hi @bshalpern,

Good point. I'll inspect these differences and produce a comparison report. In the past, there seemed to be slight differences between the web and API results, so will probably email IUCN to confirm an authoritative answer.

BB

On Mon, Apr 21, 2014 at 8:42 PM, Ben Halpern halpern@bren.ucsb.edu For IUCN data, odd that 15 species less in 2014 -- are you sure that's right?

On 4/21/2014 2:45 PM, Ben Best wrote: For the SPP 2014 (#37), I previously reported a massive boon in IUCN species assessed. In fact they now seem to use multiple records to describe synonyms such that there's in fact very little difference in species assessed:

2013 (IUCN v.2013.1):
  CR   DD   EN   EX   LC   NT   VU  Sum 
 148 2178  231   24 4564  544  676 8365 

2014(IUCN v.2013.2):
  CR   DD   EN   EX   LC   NT   VU  Sum 
 147 2175  228   24 4557  541  678 8350

Finished with the initial ingestion of 2014 data. Next, incorporating any new IUCN spatial downloads and sending species list request to AquaMaps.

bbest commented 10 years ago

Hi @bshalpern, @jafflerbach, @katlongo, @Melsteroni,

Here's an update on the IUCN species differences between last year and this year, which amounts to very little, except some confusing differences in species identifiers seeming to change for subpopulations. Ben, I previously inverted my results (2014 shows 15 new categories assessed vs 2013, not vice versa).

I need to move onto OHI toolbox development, so will be deferring further development until later (including contacting AquaMaps, and obtaining any new IUCN spatial downloads; subject to BH reprioritizing).

Thanks, BB

PS Below is pasted from the R markdown, rendered as markdown and data extracted from IUCN website by ingest_iucn.R.

SPP: IUCN Species 2014 vs 2013 Report

Versions:

extinction categories

Totals compared between years.

category freq_2013 freq_2014 freq_dif
CR 147 148 1
DD 2175 2178 3
EN 228 231 3
EX 24 24 0
LC 4557 4564 7
NT 541 544 3
VU 678 676 -2
Total 8350 8365 15

species

species new to 2014 by species id (sid)

Looking first at a summary of species counts by order and modified.year new to 2014 without matching field sid in 2013.

class order modified.year count
AVES ANSERIFORMES 2012 77
AVES ANSERIFORMES 2013 11
AVES APODIFORMES 2012 6
AVES CAPRIMULGIFORMES 2012 5
AVES CAPRIMULGIFORMES 2013 1
AVES CHARADRIIFORMES 2012 245
AVES CHARADRIIFORMES 2013 15
AVES CICONIIFORMES 2012 44
AVES CICONIIFORMES 2013 4
AVES CORACIIFORMES 2012 29
AVES CORACIIFORMES 2013 1
AVES CUCULIFORMES 2012 1
AVES FALCONIFORMES 2012 19
AVES FALCONIFORMES 2013 9
AVES GALLIFORMES 2012 5
AVES GALLIFORMES 2013 1
AVES GAVIIFORMES 2012 5
AVES GRUIFORMES 2012 31
AVES GRUIFORMES 2013 1
AVES PASSERIFORMES 2012 97
AVES PASSERIFORMES 2013 4
AVES PELECANIFORMES 2012 56
AVES PELECANIFORMES 2013 6
AVES PHOENICOPTERIFORMES 2012 4
AVES PICIFORMES 2012 2
AVES PODICIPEDIFORMES 2012 8
AVES PODICIPEDIFORMES 2013 1
AVES PROCELLARIIFORMES 2012 105
AVES PROCELLARIIFORMES 2013 26
AVES PSITTACIFORMES 2012 1
AVES SPHENISCIFORMES 2012 15
AVES SPHENISCIFORMES 2013 3
AVES STRIGIFORMES 2012 2
CHONDRICHTHYES CARCHARHINIFORMES 2007 1
GASTROPODA LITTORINIMORPHA 2013 5
GASTROPODA SORBEOCONCHA 2013 1
HOLOTHUROIDEA ASPIDOCHIROTIDA 2013 2
MAGNOLIOPSIDA SANTALALES 2013 1
REPTILIA TESTUDINES 2013 6
Total NA NA 856

species new to 2014 by scientific name (sciname)

But the existence of a new species id (sid) seems to most likely be attribution to a newly identified subpopulation, since when looking for unmatched scientific names (sciname) present in 2014 and missing in 2013, there are the only a few new species to 2014.

sid sciname class order family modified.year category
22735273 Himantopus mexicanus AVES CHARADRIIFORMES RECURVIROSTRIDAE 2013 LC
19711507 Holothuria tuberculata HOLOTHUROIDEA ASPIDOCHIROTIDA HOLOTHURIIDAE 2013 DD
200714 Holothuria inornata HOLOTHUROIDEA ASPIDOCHIROTIDA HOLOTHURIIDAE 2013 DD
198892 Potamopyrgus estuarinus GASTROPODA LITTORINIMORPHA HYDROBIIDAE 2013 LC
198845 Halopyrgus pupoides GASTROPODA LITTORINIMORPHA HYDROBIIDAE 2013 LC
179657 Agelanthus kayseri MAGNOLIOPSIDA SANTALALES LORANTHACEAE 2013 LC
198843 Halopyrgus pagodulus GASTROPODA LITTORINIMORPHA HYDROBIIDAE 2013 LC
155075 Remora albescens ACTINOPTERYGII PERCIFORMES ECHENEIDAE 2010 LC
155980 Potamopyrgus antipodarum GASTROPODA LITTORINIMORPHA HYDROBIIDAE 2013 LC
198893 Potamopyrgus kaitunuparaoa GASTROPODA LITTORINIMORPHA HYDROBIIDAE 2013 VU
189217 Melanopsis trifasciata GASTROPODA SORBEOCONCHA MELANOPSIDAE 2013 LC
19798 Parasaccogaster melanomycter ACTINOPTERYGII OPHIDIIFORMES BYTHITIDAE 1996 VU
5325 Emblemariopsis tayrona ACTINOPTERYGII PERCIFORMES CHAENOPSIDAE 1996 VU

species missing in 2014 but present in 2013, by scientific name (sciname)

Conversely, here are the species present in 2013 but missing from 2014 by matching field sciname.

sid sciname class order family modified.year category
155075 Remorina albescens ACTINOPTERYGII PERCIFORMES ECHENEIDAE 2010 LC
19798 Saccogaster melanomycter ACTINOPTERYGII OPHIDIIFORMES BYTHITIDAE 1996 VU
18176 Pristis perotteti CHONDRICHTHYES RAJIFORMES PRISTIDAE 2007 CR
5325 Coralliozetus tayrona ACTINOPTERYGII PERCIFORMES CHAENOPSIDAE 1996 VU
18174 Pristis microdon CHONDRICHTHYES RAJIFORMES PRISTIDAE 2006 CR

Of those 2013 species missing in 2014, here are the same species ids (field sid) still found in 2014. So the sciname changed.

sid sciname class order family modified.year category
155075 Remorina albescens ACTINOPTERYGII PERCIFORMES ECHENEIDAE 2010 LC
19798 Saccogaster melanomycter ACTINOPTERYGII OPHIDIIFORMES BYTHITIDAE 1996 VU
18176 Pristis perotteti CHONDRICHTHYES RAJIFORMES PRISTIDAE 2007 CR
5325 Coralliozetus tayrona ACTINOPTERYGII PERCIFORMES CHAENOPSIDAE 1996 VU
18174 Pristis microdon CHONDRICHTHYES RAJIFORMES PRISTIDAE 2006 CR

species with extinction changed from 2013 to 2014, by species id (sid)

Species with extinction category changed by matching sid.

sid sciname sciname_2013 category category_2013 criteria criteria_2013 modified.year modified.year_2013 class order family
6494 Dermochelys coriacea Dermochelys coriacea VU CR A2bd A1abd 2013 2000 REPTILIA TESTUDINES DERMOCHELYIDAE

species with extinction changed from 2013 to 2014, by scientific name (sciname)

Number of rows: 7100

First 6 rows...

sid sid_2013 sciname category category_2013 criteria criteria_2013 modified.year modified.year_2013 class order family
44559 234 Acipenser transmontanus VU LC A2abc NA 2004 2004 ACTINOPTERYGII ACIPENSERIFORMES ACIPENSERIDAE
2468 2467 Balaena mysticetus NT LC NA NA 1996 2012 MAMMALIA CETARTIODACTYLA BALAENIDAE
2469 2467 Balaena mysticetus EN LC D NA 2012 2012 MAMMALIA CETARTIODACTYLA BALAENIDAE
2472 2467 Balaena mysticetus CR LC D NA 2012 2012 MAMMALIA CETARTIODACTYLA BALAENIDAE
2467 2468 Balaena mysticetus LC NT NA NA 2012 1996 MAMMALIA CETARTIODACTYLA BALAENIDAE
2469 2468 Balaena mysticetus EN NT D NA 2012 1996 MAMMALIA CETARTIODACTYLA BALAENIDAE

So scientific name (sciname) can repeat for a given sid, indicating a subpopulation, in which case matching by sciname alone is insufficient.

bshalpern commented 10 years ago

Just confirming that Mel will be able to take over redoing the global species analyses, for SPP and ICO status calculations. Katie/Jamie, please take lead on contacting Aquamaps for any missing pieces (and so coordinate with Mel on this).

thanks, Ben

On 4/23/2014 1:40 PM, Ben Best wrote:

Hi @bshalpern https://github.com/bshalpern, @jafflerbach https://github.com/jafflerbach, @katlongo https://github.com/katlongo, @Melsteroni https://github.com/Melsteroni,

Here's an update on the species differences between last year and this year, which amounts to very little, except some confusing differences in species identifiers seeming to change for subpopulations. Ben, I previously inverted my results (2014 shows 15 new categories assessed vs 2013, not vice versa).

I need to move onto OHI toolbox development, so will be deferring further development until later (including contacting AquaMaps, and obtaining any new IUCN spatial downloads http://www.iucnredlist.org/technical-documents/spatial-data; subject to BH reprioritizing).

Thanks, BB

PS Below is pasted from the R markdown https://github.com/OHI-Science/ohiprep/blob/d384516d084c6cbabfc6de7e3f9d75f720aa66d3/Global/NCEAS-SpeciesDiversity_v2014/report_2014vs2013.Rmd, rendered as markdown https://github.com/OHI-Science/ohiprep/blob/d384516d084c6cbabfc6de7e3f9d75f720aa66d3/Global/NCEAS-SpeciesDiversity_v2014/report_2014vs2013.md and data extracted from IUCN website by ingest_iucn.R https://github.com/OHI-Science/ohiprep/blob/d384516d084c6cbabfc6de7e3f9d75f720aa66d3/Global/NCEAS-SpeciesDiversity_v2014/ingest_iucn.R.

SPP: IUCN Species 2014 vs 2013 Report

Versions:

  • 2013: /2013.1/ (as of Jul 1, 2013)

  • 2014: /2013.2/ (as of Apr 20, 2014)

    extinction categories

Totals compared between years.

category freq_2013 freq_2014 freq_dif CR 147 148 1 DD 2175 2178 3 EN 228 231 3 EX 24 24 0 LC 4557 4564 7 NT 541 544 3 VU 678 676 -2 Total 8350 8365 15

species

  species new to 2014 by species id (sid)

Looking first at a summary of species counts by |order| and |modified.year| new to 2014 without matching field |sid| in 2013.

class order modified.year count AVES ANSERIFORMES 2012 77 AVES ANSERIFORMES 2013 11 AVES APODIFORMES 2012 6 AVES CAPRIMULGIFORMES 2012 5 AVES CAPRIMULGIFORMES 2013 1 AVES CHARADRIIFORMES 2012 245 AVES CHARADRIIFORMES 2013 15 AVES CICONIIFORMES 2012 44 AVES CICONIIFORMES 2013 4 AVES CORACIIFORMES 2012 29 AVES CORACIIFORMES 2013 1 AVES CUCULIFORMES 2012 1 AVES FALCONIFORMES 2012 19 AVES FALCONIFORMES 2013 9 AVES GALLIFORMES 2012 5 AVES GALLIFORMES 2013 1 AVES GAVIIFORMES 2012 5 AVES GRUIFORMES 2012 31 AVES GRUIFORMES 2013 1 AVES PASSERIFORMES 2012 97 AVES PASSERIFORMES 2013 4 AVES PELECANIFORMES 2012 56 AVES PELECANIFORMES 2013 6 AVES PHOENICOPTERIFORMES 2012 4 AVES PICIFORMES 2012 2 AVES PODICIPEDIFORMES 2012 8 AVES PODICIPEDIFORMES 2013 1 AVES PROCELLARIIFORMES 2012 105 AVES PROCELLARIIFORMES 2013 26 AVES PSITTACIFORMES 2012 1 AVES SPHENISCIFORMES 2012 15 AVES SPHENISCIFORMES 2013 3 AVES STRIGIFORMES 2012 2 CHONDRICHTHYES CARCHARHINIFORMES 2007 1 GASTROPODA LITTORINIMORPHA 2013 5 GASTROPODA SORBEOCONCHA 2013 1 HOLOTHUROIDEA ASPIDOCHIROTIDA 2013 2 MAGNOLIOPSIDA SANTALALES 2013 1 REPTILIA TESTUDINES 2013 6 Total NA NA 856

  species new to 2014 by scientific name (sciname)

But the existence of a new species id (|sid|) seems to most likely be attribution to a newly identified subpopulation, since when looking for unmatched scientific names (|sciname|) present in 2014 and missing in 2013, there are the only a few new species to 2014.

sid sciname class order family modified.year category 22735273 Himantopus mexicanus AVES CHARADRIIFORMES RECURVIROSTRIDAE 2013 LC 19711507 Holothuria tuberculata HOLOTHUROIDEA ASPIDOCHIROTIDA HOLOTHURIIDAE 2013 DD 200714 Holothuria inornata HOLOTHUROIDEA ASPIDOCHIROTIDA HOLOTHURIIDAE 2013 DD 198892 Potamopyrgus estuarinus GASTROPODA LITTORINIMORPHA HYDROBIIDAE 2013 LC 198845 Halopyrgus pupoides GASTROPODA LITTORINIMORPHA HYDROBIIDAE 2013 LC 179657 Agelanthus kayseri MAGNOLIOPSIDA SANTALALES LORANTHACEAE 2013 LC 198843 Halopyrgus pagodulus GASTROPODA LITTORINIMORPHA HYDROBIIDAE 2013 LC 155075 Remora albescens ACTINOPTERYGII PERCIFORMES ECHENEIDAE 2010 LC 155980 Potamopyrgus antipodarum GASTROPODA LITTORINIMORPHA HYDROBIIDAE 2013 LC 198893 Potamopyrgus kaitunuparaoa GASTROPODA LITTORINIMORPHA HYDROBIIDAE 2013 VU 189217 Melanopsis trifasciata GASTROPODA SORBEOCONCHA MELANOPSIDAE 2013 LC 19798 Parasaccogaster melanomycter ACTINOPTERYGII OPHIDIIFORMES BYTHITIDAE 1996 VU 5325 Emblemariopsis tayrona ACTINOPTERYGII PERCIFORMES CHAENOPSIDAE 1996 VU

  species missing in 2014 but present in 2013, by scientific name
  (sciname)

Conversely, here are the species present in 2013 but missing from 2014 by matching field |sciname|.

sid sciname class order family modified.year category 155075 Remorina albescens ACTINOPTERYGII PERCIFORMES ECHENEIDAE 2010 LC 19798 Saccogaster melanomycter ACTINOPTERYGII OPHIDIIFORMES BYTHITIDAE 1996 VU 18176 Pristis perotteti CHONDRICHTHYES RAJIFORMES PRISTIDAE 2007 CR 5325 Coralliozetus tayrona ACTINOPTERYGII PERCIFORMES CHAENOPSIDAE 1996 VU 18174 Pristis microdon CHONDRICHTHYES RAJIFORMES PRISTIDAE 2006 CR

Of those 2013 species missing in 2014, here are the same species ids (field |sid|) still found in 2014. So the |sciname| changed.

sid sciname class order family modified.year category 155075 Remorina albescens ACTINOPTERYGII PERCIFORMES ECHENEIDAE 2010 LC 19798 Saccogaster melanomycter ACTINOPTERYGII OPHIDIIFORMES BYTHITIDAE 1996 VU 18176 Pristis perotteti CHONDRICHTHYES RAJIFORMES PRISTIDAE 2007 CR 5325 Coralliozetus tayrona ACTINOPTERYGII PERCIFORMES CHAENOPSIDAE 1996 VU 18174 Pristis microdon CHONDRICHTHYES RAJIFORMES PRISTIDAE 2006 CR

  species with extinction changed from 2013 to 2014, by species id
  (sid)

Species with extinction category changed by matching |sid|.

sid sciname sciname_2013 category category_2013 criteria criteria_2013 modified.year modified.year_2013 class order family 6494 Dermochelys coriacea Dermochelys coriacea VU CR A2bd A1abd 2013 2000 REPTILIA TESTUDINES DERMOCHELYIDAE

  species with extinction changed from 2013 to 2014, by scientific
  name (sciname)

Number of rows: 7100

First 6 rows...

sid sid_2013 sciname category category_2013 criteria criteria_2013 modified.year modified.year_2013 class order family 44559 234 Acipenser transmontanus VU LC A2abc NA 2004 2004 ACTINOPTERYGII ACIPENSERIFORMES ACIPENSERIDAE 2468 2467 Balaena mysticetus NT LC NA NA 1996 2012 MAMMALIA CETARTIODACTYLA BALAENIDAE 2469 2467 Balaena mysticetus EN LC D NA 2012 2012 MAMMALIA CETARTIODACTYLA BALAENIDAE 2472 2467 Balaena mysticetus CR LC D NA 2012 2012 MAMMALIA CETARTIODACTYLA BALAENIDAE 2467 2468 Balaena mysticetus LC NT NA NA 2012 1996 MAMMALIA CETARTIODACTYLA BALAENIDAE 2469 2468 Balaena mysticetus EN NT D NA 2012 1996 MAMMALIA CETARTIODACTYLA BALAENIDAE

So scientific name (|sciname|) can repeat for a given |sid|, indicating a subpopulation, in which case matching by |sciname| alone is insufficient.

— Reply to this email directly or view it on GitHub https://github.com/OHI-Science/ohicore/issues/37#issuecomment-41211748.

Benjamin S. Halpern Professor, Bren School of Environmental Science and Management UCSB, Santa Barbara, CA 93106 (ph) 805.893.2862 halpern@bren.ucsb.edu

Chair in Marine Conservation, Imperial College London Grand Challenges in Ecosystems and the Environment http://www3.imperial.ac.uk/ecosystemsandenvironment

Director, Center for Marine Assessment and Planning (CMAP) http://cmap.msi.ucsb.edu

Senior Fellow, UN Envir. Prog.- World Conserv. Monitor. Cent. (UNEP-WCMC)

Research Biologist, Nat'l. Center for Ecol. Anal. & Synth. (NCEAS) 735 State St., Suite 300, Santa Barbara, CA 93101 phone: 805.892.2523; fax: 805.892.2510