Expand the list of species

[scott97]

Maintaining a list of species in our database would involve a lot of work. Potentially we could use inaturalist's API to fetch a list of species and their images. To get the most popular species we can make a query like so:

GET https://api.inaturalist.org/v1/taxa?is_active=true&taxon_id=<taxon_id>&rank=species&order=desc&order_by=observations_count

In this query the taxon ID is used to filter to only show its descendants. Combined with rank=species it can find all the species within that category. We will need to make multiple queries and merge the results to get a satisfactory list of marine life. This is because marine animals do not fit neatly into one category.

Vertibrates

Within the vertibrates, I think we should include all within:

• [x] Jawless Fishes Superclass Agnatha
• [x] Ray-finned Fishes Class Actinopterygii because this is so big, I am splitting into multiple imports
  • [x] Tarpons, Eels, and Allies Superorder Elopomorpha
  • [x] Carps, Characins, Catfishes and Allies Superorder Ostariophysi
  • [x] Bonytongues and Mooneyes Superorder Osteoglossomorpha
  • [x] Butterflyfishes, Angelfishes, and Allies Order Acanthuriformes
  • [x] Sturgeons and Paddlefishes Order Acipenseriformes
  • [x] Oceanic Basses Order Acropomatiformes
  • [x] Slickheads and Tubeshoulders Order Alepocephaliformes
  • [x] Bowfins Order Amiiformes
  • [x] Leaffishes, Snakeheads, and Gouramis Order Anabantiformes
  • [x] Marine Smelts Order Argentiniformes
  • [x] Jellynose Fishes Order Ateleopodiformes
  • [x] Silversides Order Atheriniformes
  • [x] Grinners, Lizardfishes, and Allies Order Aulopiformes
  • [x] Toadfishes Order Batrachoidiformes
  • [x] Needlefishes, Halfbeaks, and Allies Order Beloniformes
  • [x] Sawbellies and Squirrelfishes Order Beryciformes
  • [x] Blennies, Damselfishes and Allies Order Blenniiformes
  • [x] Jacks, Flounders, Barracudas, and Allies Order Carangiformes
  • [x] Freshwater Sunfishes, Grunters, and Allies Order Centrarchiformes
  • [x] Cichlids and Allies Order Cichliformes
  • [x] Herrings Order Clupeiformes
  • [x] Rivulines, Killifishes, and Livebearers Order Cyprinodontiformes
  • [x] Cods and Hakes Order Gadiformes
  • [x] Galaxiids Order Galaxiiformes
  • [x] Gobies, Cardinalfishes and Allies Order Gobiiformes
  • [x] Wrasses and Allies Order Labriformes
  • [x] Velifers, Opahs, and Ribbonfishes Order Lampriformes
  • [x] Salamanderfishes Order Lepidogalaxiiformes
  • [x] Gars Order Lepisosteiformes
  • [x] Anglerfishes Order Lophiiformes
  • [x] Mullets and Glassfishes Order Mugiliformes
  • [x] Lanternfishes Order Myctophiformes
  • [x] Cusk-Eels Order Ophidiiformes
  • [x] Smelts and Allies Order Osmeriformes
  • [x] Perch-like Fishes Order Perciformes
  • [x] Trout-perches, Pirate Perches, and Cavefishes Order Percopsiformes
  • [x] Beardfishes Order Polymixiiformes
  • [x] Bichirs Order Polypteriformes
  • [x] Salmons, Pikes and Mudminnows Order Salmoniformes
  • [x] Mackerels, Tunas, Butterfishes, and Allies Order Scombriformes
  • [x] Dragonfishes and Allies Order Stomiiformes
  • [x] Swamp Eels Order Synbranchiformes
  • [x] Pipefishes and Allies Order Syngnathiformes
  • [x] Puffers and Filefishes Order Tetraodontiformes
  • [x] Lanterneyes, Fangtooths and Allies Order Trachichthyiformes
  • [x] Dories Order Zeiformes
• [x] Lobe-finned Fishes Class Sarcopterygii
• [x] Chimaeras Class Holocephali
• [x] Elasmobranchs Class Elasmobranchii

Within Birds Class Aves

• [x] - Penguins Order Sphenisciformes

Within Mammals Class Mammalia

• [x] - Cetaceans Infraorder Cetacea

Within Reptiles Class Reptilia

• [x] - Crocodilians Order Crocodylia

• [x] - Sea Turtles Superfamily Chelonioidea

• sea snakes would require more research

Within Amphibians more research needed

Invertibrates

more research needed

Species API and caching

Since there are many species, and a bit of work merging the results of multiple requests, it would be best to cache the info we need in our datastore, omitting the data that we don't need.

Sources:

• https://api.inaturalist.org/v1/docs/#!/Taxa/get_taxa

[scott97]

I propose we add a new API to import species from inaturalist. This API would be for Admins only and could be used periodically to keep our species list up to date.

POST /api/v1/species/import-inaturalist?descendants_of=Infraorder Cetacea,Order Sphenisciformes,Class Holocephali etc

This API will use the provided name and rank(s) to query the inaturalist API for species and insert these species into the datastore.

Updating already imported species

In our datastore, we will store the inaturalist taxon id if a species is imported. When that species is imported again, we can search by this id and can update that species instead of creating a new one.

Preview API

POST /api/v1/species/import-inaturalist?descendants_of=Infraorder Cetacea&preview=true

This API would provide information about what will be inserted and updated but not perform the action.

[scott97]

https://www.inaturalist.org/pages/api+recommended+practices

Key points:

• Authentication not required for what we are doing
• Requests are to be limited to 10,000 items per page
• Requests are to be limited to 1 per second
• We should set the user agent with the HTTP header User-Agent: openfish