inbo / alien-species-portal

Portal for alien and invasive species indicators
MIT License
0 stars 0 forks source link

Management page Ondatra zibethicus #50

Closed EmmaCartuyvels1 closed 8 months ago

EmmaCartuyvels1 commented 11 months ago

Muskrat management data (catches made) can be found in datasets published on GBIF with the following dataset keys "ddd51fa5-97ce-48ff-9a58-a09d7e76b103", "b7ee2a4d-8e10-410f-a951-a7f032678ffe", "95b0e787-8508-4247-9e48-18b45fc7d12e", "3634aee3-41d5-4aa2-8cb5-875859f62a3a", "69351197-880d-4100-8e69-e80babf3fdd7".

However some data is still missing:

Ideally this data will be published on GBIF as well, this process is underway: https://github.com/inbo/mica-occurrences

IMPORTANT: these datasets contain both catches and observations and only catches should be considered for analyses. It is unclear if this distinction can be made based on the GBIF export.

Analyses using these data can be found here: https://github.com/inbo/muskusratten/blob/master/Scripts/AnaGBIF.R

This code visualises the catches over the years, indicating each province in a different colour:

## Evolutie vangsten
catches_year <- d %>%
  filter(stateProvince %in% c("West Flanders", "East Flanders", 
                              "Flemish Brabant", "Antwerp", "Limburg")) %>%
  filter(year < 2023) %>% 
  group_by(year, stateProvince) %>%
  summarise(catches = sum(individualCount, na.rm = TRUE))

ggplot(catches_year, 
       aes(x = year, y = catches, fill = stateProvince)) +
  geom_bar(position = "stack", 
           stat = "identity") +
  facet_zoom(xy = year > 2008, 
             horizontal = FALSE, 
             zoom.size = 1.5,
             zoom.data = ifelse(year>2008, NA, FALSE)) +
  labs(x = "Jaar", y = "Vangsten") +
  scale_fill_discrete(name = "Provincie", labels = c("Antwerpen",
                                                     "Oost-Vlaanderen", 
                                                     "Vlaams-Brabant", 
                                                     "Limburg", 
                                                     "West-Vlaanderen"))

This code shows catches made per muncipality for a given year:

catches_gem <- d %>%
  group_by(year, gemeente, .drop=FALSE) %>%
  summarise(catches = sum(individualCount, na.rm = TRUE)) %>% 
  st_drop_geometry()

VL_Gem <- gem %>% 
  filter(LanguageSt %in% c(1, 5)) %>% 
  select(gemeente = NameDut) %>% 
  mutate(year = 2022,
         catches = 0) %>% 
  st_drop_geometry()

catches_gem_2022 <- catches_gem %>% 
  filter(year == 2022) %>%
  ungroup() %>% 
  add_row(VL_Gem) %>% 
  group_by(year, gemeente) %>% 
  summarise(catches = sum(catches, na.rm = TRUE))

data <- gem %>% 
  left_join(catches_gem_2022, 
            by = c("Gemeente" = "gemeente"))

pal <- colorBin(palette = rev(heat.colors(11)), 
                bins = c(-Inf,0.0001,5,10,25,50,100,+Inf),
                na.color="#cecece")

leaflet(data, options = leafletOptions(minZoom = 7, maxZoom = 10)) %>%
  addTiles() %>% 
  addLegend(title = paste("Gevangen muskusratten 2022"),
            pal = pal, values = ~catches, 
            position = "bottomleft") %>%
  addPolygons(color = ~ pal(catches), 
              stroke = FALSE, smoothFactor = 0.2, fillOpacity = 0.85,
              popup=sprintf("<strong>%s</strong><br>%s",
                            data$Gemeente, data$catches)) %>%
  addPolylines(weight = 1.5, color = "white") %>%
  setView(4.2813167, 50.76, zoom = 8) %>% 
  addPolygons(data = provincies, group = "Provincies", fill = NA,
              color = "forestgreen", weight = 2)

This code shows the evalution of catches per border area:

Wal <- c("Voeren", "Riemst", "Mesen", "Ieper", "Zonnebeke", "Wervik", "Menen", 
         "Kortrijk", "Spierre-Helkijn", "Avelgem", "Kluisbergen", "Ronse", 
         "Maarkedal", "Brakel", "Geraardsbergen", "Bever", "Herne", "Pepingen", 
         "Halle", "Beersel", "Sint-Genesius-Rode", "Hoeilaart", "Overijse", 
         "Huldenberg", "Oud-Heverlee", "Bierbeek", "Boutersem", "Hoegaarden", 
         "Tienen", "Landen", "Gingelom", "Heers", "Tongeren", "Herstappe")
Fran <- c("De Panne", "Veurne", "Alveringem", "Poperinge", "Heuvelland")
Ned <- c("Lanaken", "Maasmechelen", "Dilsen-Stokkem", "Maaseik", "Kinrooi", 
         "Bree", "Bocholt", "Hamont-Achel", "Neerpelt", "Lommel", "Mol", 
         "Arendonk", "Ravels", "Turnhout", "Baarle-Hertog", "Merksplas", 
         "Hoogstraten", "Wuustwezel", "Kalmthout", "Essen", "Kapellen", 
         "Stabroek", "Antwerpen", "Beveren", "Sint-Gillis-Waas", "Stekene", 
         "Moerbeke", "Wachtebeke","Zelzate", "Assenede", "Sint-Laureins", 
         "Maldegem", "Damme", "Knokke-Heist")

stap1 <- catches_gem %>%  
  mutate(Grens =  ifelse(gemeente %in% Wal, "Wal", "Nee")) %>%
  mutate(Grens =  ifelse(gemeente %in% Fran, "Fran", Grens)) %>%
  mutate(Grens =  ifelse(gemeente %in% Ned, "Ned", Grens))

stap1 %>% 
  filter(year > 2009 & year < 2023) %>% 
  group_by(year, Grens) %>% 
  summarise(catches = sum(catches, na.rm = TRUE)) %>% 
  ggplot(aes(x = year, y = catches)) +
  geom_point(aes(color = Grens)) +
  geom_line(aes(color = Grens)) +
  scale_x_continuous(breaks = 2010:2022) +
  labs(x = "Jaar", y = "Vangsten") +
  scale_color_discrete(name = "Gemeente grenst aan:", labels = c("Frankrijk",
                                                     "Nederland", 
                                                     "Geen grensgemeente", 
                                                     "Wallonië"))

This code visualises the change in catches compared to the previous year per muncipality:

VL_Gem_ext <- expand.grid(VL_Gem$gemeente, c(1991:2022)) %>% 
  mutate(catches = 0) %>% 
  rename(gemeente = Var1,
         year = Var2)

catches_gem_full <- catches_gem %>%
  ungroup() %>% 
  add_row(VL_Gem_ext) %>% 
  group_by(year, gemeente) %>% 
  summarise(catches = sum(catches, na.rm = TRUE)) %>% 
  ungroup() %>% 
  arrange(gemeente, year) %>% 
  mutate(prev_catch = lag(catches)) %>% 
  mutate(change = catches - prev_catch)

data <- catches_gem_full %>%
  filter(year == 2022) %>% 
  left_join(gem, 
            by = c("gemeente" = "Gemeente"))

data <- st_as_sf(data)

pal <- colorBin(palette = "PiYG", 
                bins = c(-Inf,-100,-50,-25,-10,-5,0,5,10,25,50,100,+Inf),
                na.color="#cecece",
                reverse = TRUE)

leaflet(data, options = leafletOptions(minZoom = 7, maxZoom = 10)) %>%
  addTiles() %>% 
  addLegend(title = paste("Toename / afname aantal vangsten"),
            pal = pal, values = ~change, 
            position = "bottomleft") %>%
  addPolygons(color = ~ pal(change), 
              stroke = FALSE, smoothFactor = 0.2, fillOpacity = 0.85,
              popup=sprintf("<strong>%s</strong><br>%s",
                            data$gemeente, data$change)) %>%
  addPolylines(weight = 1.5, color = "white") %>%
  setView(4.2813167, 50.76, zoom = 8) %>% 
  addPolygons(data = provincies, group = "Provincies", fill = NA,
              color = "forestgreen", weight = 2)
timadriaens commented 11 months ago

it would be great to have these management results graphs and maps on the dashboard!

SanderDevisscher commented 11 months ago

@mvarewyck I'll write a script to extract the neccessary data from gbif and provide you with some cleaned datasets to use.

SanderDevisscher commented 10 months ago

@mvarewyck like I see it these 2 visualisations are similar to those we allready have:

While the other 2 are new.

EmmaCartuyvels1 commented 10 months ago

"This code visualises the change in catches compared to the previous year per muncipality:" I do think that this gives an instant idea of which areas are (becoming) problem areas and will be easier to interpret by field managers than having to compare the different years by themselves.

mvarewyck commented 9 months ago
  • "This code shows the evalution of catches per border area:" I see some options here, @mvarewyck can you indicate feasability/time cost:
    • Is it possible to add the border regions as municipality scale options. Ifso should I add them to the data ?

In the app I would add an extra option 'border municipalities' for Region level. In the data I would prefer to have an extra column 'isBorderMunicipality' with value TRUE/FALSE, besides the 'municipality' column. (1) Preselect all border regions and hide the other municipalities in the map (gray?). To have this implemented, I would count 1 day. (2) If we don't need to hide the other municipalities but just preselect the border regions and showing these in the 'Regions' input field, we can mostly re-use the code of 'municipalities' option, and I would count 0.5 day. Screenshot from 2023-09-28 12-13-00

  • or we could consider this as a separate graph

Same cost as above. Less efficient coding-wise because we copy-paste code.

  • "This code visualises the change in catches compared to the previous year per muncipality:"
    • @mvarewyck how much time would it cost to add a map indicating change when we provide a data frame with province, niscode & change as columns ?

I would count 0.5 day.

mvarewyck commented 9 months ago

"This code shows the evalution of catches per border area:" I see some options here, @mvarewyck can you indicate feasability/time cost:

  • Is it possible to add the border regions as municipality scale options. Ifso should I add them to the data ?

We drop this plot

"This code visualises the change in catches compared to the previous year per muncipality:"

  • @mvarewyck how much time would it cost to add a map indicating change when we provide a data frame with province, niscode & change as columns ?

We keep this plot. Also include it for the other management pages if relevant. Include the unit 'Difference with previous year' as an option besides absolute count

EmmaCartuyvels1 commented 9 months ago

@mvarewyck here is some sample data muskrat_sample.csv

mvarewyck commented 9 months ago

@EmmaCartuyvels1 Thanks. Some remarks on the current data format

I've done this manually for now, but good to keep in mind with future data updates.

mvarewyck commented 9 months ago

I've created a draft version of the mgt page similar to the one for Lithobates catesbeianus.

Screenshot from 2023-10-16 15-05-26 Screenshot from 2023-10-16 15-05-50 Screenshot from 2023-10-16 15-11-14

@EmmaCartuyvels1 @SanderDevisscher Some questions/remarks

Screenshot from 2023-10-16 15-00-57

TODO by me

EmmaCartuyvels1 commented 9 months ago
  • There are a lot of observations with NA for individualCount, for all years before 2021. Should we (a) exclude the rows with NA values then the barplot starts at 2021. Or (b) set the value to 0 then the barplot starts at 2016 with 0 counts until 2021. (Currently (a) is implemented)

If we are only using the GBIF data there will be some NA's that aren't zeros (eg Limburg, Antwerp and Flemish Brabant will be 0 but for other provinces it might also be a case of missing data). However, there should be data going back to 1991.

EmmaCartuyvels1 commented 9 months ago
  • We don't have 'cpue' unit choice as we don't have information on n_fuiken.

Some of the raw data does have this info, but I don't think it's published to GBIF. If you want to include this you should ask @LienReyserhove or @DimEvil.

EmmaCartuyvels1 commented 9 months ago
  • For Lithobates catesbeianus we have a nice overlap between cube occurrence and captures: whenever there's a red square observation the polygon is colored. For Ondatra zibethicus this is not the case. Is this to be expected?

I'm assuming the cubes come from waarnemingen? Then it would be expected since muskrats are not often active during the day and trappers don't record them in waarnemingen.

SanderDevisscher commented 9 months ago
  • Provinces are named in English, preferably in Dutch, like for the other datasets.
  • Province is often missing, while exact coordinates are known. Can it be completed or is there a reason for being missing?
  • Variable (column) 'Gewest' is missing to display the results per gewest.

I'm working on this, @EmmaCartuyvels1 used different shapes for provinces/communes. Once these changes are done I'll upload the new files (see https://github.com/inbo/aspbo/pull/22).

SanderDevisscher commented 9 months ago
  • For Lithobates catesbeianus we have a nice overlap between cube occurrence and captures: whenever there's a red square observation the polygon is colored. For Ondatra zibethicus this is not the case. Is this to be expected?

I'm assuming the cubes come from waarnemingen? Then it would be expected since muskrats are not often active during the day and trappers don't record them in waarnemingen.

The cubes are a gbif product (using a Trias workflow, more info) the current cube is outdated (2022-03-09) and given the fact a lot of (historical & recent) muskrat management data (often the only observation source) have been uploaded to gbif more recently (after 04-2022, see https://github.com/riparias/vmm-rattenapp-occurrences) one would expect a mismatch. This will, hopefully, be fixed with an updated cube (see https://github.com/inbo/aspbo/issues/18).

SanderDevisscher commented 9 months ago

@mvarewyck which columns do you need to create the graphs/maps ? I think some of the columns currently in the dataset are not needed persé to create the graphs/maps.

mvarewyck commented 9 months ago

@mvarewyck which columns do you need to create the graphs/maps ? I think some of the columns currently in the dataset are not needed persé to create the graphs/maps.

Indeed, many columns are redundant. I kept these:

Screenshot from 2023-10-16 21-23-13

Optional: gbifID is good for tracking issues with the data. decimalLatitude and decimalLongitude currently not used in the maps, but can be used to complete missing values on the location.

mvarewyck commented 9 months ago

Examples of plotting the difference with previous year

Ondatra zibethicus in 2022 per gemeente

Ondatra zibethicus_2022_management

Lithobates catesbeianus in 2018 per province

Lithobates catesbeianus_2018_management

Lithobates catesbeianus trend plot per gemeente

Screenshot from 2023-10-17 12-00-17

Questions/Remarks

Alternative color palettes:

029-r-color-palettes-rcolorbrewer-palettes-1

EmmaCartuyvels1 commented 9 months ago
  • The 'Yellow-Orange-Brown' color palette is not useful for differences. I changed it to 'Red-Orange-Green', but this conflicts with 'Occurrence' in red (see plot above).

I do like the 'Red-Orange-Green' since red = bad = more animals than last year and the reverse for green. Can we change the occurences squares to blue or this not an option?

mvarewyck commented 9 months ago
  • The 'Yellow-Orange-Brown' color palette is not useful for differences. I changed it to 'Red-Orange-Green', but this conflicts with 'Occurrence' in red (see plot above).

I do like the 'Red-Orange-Green' since red = bad = more animals than last year and the reverse for green. Can we change the occurences squares to blue or this not an option?

It was decided to have red squares here in contrast with the background map. We can change to any color, example 'blue' below.

Screenshot from 2023-10-17 13-59-02

Fhuysent commented 8 months ago

For me, blue is OK!