James-Lee-998 / IUCN_webscraper

An online IUCN webscraper designed for custom outputs to track red listed species
0 stars 0 forks source link

Is that possible to get conservation actions and threats from IUCN by scrapping? #2

Open karolazvdo opened 1 year ago

karolazvdo commented 1 year ago

Hi, a time ago I left here an issue about empty lists from the results output, that you indeed help me to handle. However, trying to run that code again, the filter part to filter rows from empty list into a dataframe does not works. And returns to me some species with empty results.

This was the code:

`df = df %>% mutate(iucn_pull= map(species, rl_search, key=apikey))%>% mutate(filter_nulls = map(iucn_pull, pluck, "result")) %>% filter(filter_nulls != 'NULL')

api_clean <- df %>% mutate(category = map_chr(iucn_pull, pluck, "result", "category")) %>% mutate(trend = map_chr(iucn_pull, pluck, "result", "population_trend")) %>% dplyr::select(species, category, trend)`

Besides that, I was looking for an answer on the IUCN redlist website, and I saw that they give data about conservation actions and threats for the species. Is your webscrape code could get this information too??

I'll give some species name that worked and didn't work here.

df = c("Crocidura allex", "Acomys selousi", "Afronycteris nanus", "Capra hircus", "Acinonyx jubatus hecki", Damaliscus korrigum", "Dasymys rufulus")

James-Lee-998 commented 1 year ago

Hi @karolazvdo,

I was looking into the problem and the below section of code does filter results that are NULL.

mutate(filter_nulls = map(iucn_pull, pluck, "result")) %>% filter(filter_nulls != 'NULL')

It assumes that NULL values means that there is no record on the IUCN red list (or no access via their API). Since the API endpoint here is 'rl_search' and a generic one it's my way of filtering out any species that don't show up on the IUCN redlist. list of species.csv As a sanity check and since I don't know the full range of species you're working I've shared a csv file with all the IUCN redlist species.

James-Lee-998 commented 1 year ago

As for the second part of your request, I made a quick python script to test this small list and produced some csvs for each field ('conservation actions' and 'threats'). Some didn't return any information which will conservation_measures.csv threats.csv be shown as missing data in the respective row.