Update data

[ibidyouadu]

The script previously used to update code (not in this repo but in the Ecological_Dashboard repo) is pretty simple:

fish.surveys <- data.table::fread("https://query.data.world/s/beo5xpafq24c45b4a6bdzvbgxa3c5l", ... )

The problem is that that link is dead. And looking at the fish surveys dataset, it is not obvious what the query to produce fish.surveys was. Sure, a lot of the columns are there in separate files and it will take a few joins, but there's still some columns missing, namely the subnational and local regions. Digging around through the other Rare datasets, it looks like this missing info may be completely available in various datasets from the global footprint datasets. Look into these

[ibidyouadu]

A nearly complete dataset is found in the dive surveys project. The only columns missing are level1_name and level2_name. To get these, we can map the information from global footprint data.

The idea is to take an ma_name from the nearly complete dataset and find a row in the footprint data with that ma_name. Then grab the level1_name and level2_name of that row. This assumes that there is a one-to-one mapping between ma_name and level1_name-level2_name pairs. As it turns out, there is not. Here is the result of checking for ma_names with different level1_name-level2_name pairs:

   country                      level1_name       level2_name          ma_name                  
   <fct>                        <fct>             <fct>                <fct>                    
 1 Brazil                       Pará              Augusto Corrêa       RESEX Marinha de Araí-Pe…
 2 Brazil                       Pará              Augusto Corrêa       RESEX Marinha Mestre Luc…
 3 Brazil                       Pará              Maracanã             RESEX Maracanã           
 4 Brazil                       Pará              Maracanã             RESEX Marinha Mestre Luc…
 5 Brazil                       Pará              Marapanim            RESEX Marinha Mestre Luc…
 6 Brazil                       Pará              Marapanim            RESEX Marinha de Araí-Pe…
 7 Brazil                       Pará              Marapanim            RESEX Maracanã           
 8 Honduras                     Islas de la Bahía Roatán               Roatan                   
 9 Honduras                     Islas de la Bahía Jose Santos Guardio… Roatan                   
10 Federated States of Microne… Pohnpei           Nett                 Kolonia - Nett           
11 Federated States of Microne… Pohnpei           Kolonia              Kolonia - Nett    

The managed access areas from Brazil and FSM are no problem since the dashboard currently does not show data from there. But the newest data is made up entirely of Roatan data. So then we cannot map simply ma_name to level1_name/level2_name; we need another distinguishing column besides ma_name. Looking at the footprint data, there doesn't seem to be any. Thus the current problem is that the newest data will have NA level1_name and level2_name.

Either dig around the data.world datasets more or simply remove filtering by subnational/local gov't.

[ibidyouadu]

Included 2021 IDN fish data with d812799