Virtual Museum data updates

[helendallas]

@dimasciput Please can you check that the harvesting of VM occurrence records fro Anurans and Odonates (adults) is happening automatically every 1 month.

The last records were in early 2022.

Similarly for Odonates (adult) - last records were in 2021.

[dimasciput]

Currently fetching odonata and frog data. Will check the result next week.

[helendallas]

Thanks @dimasciput Please can you confirm if this has been set to be automated once a month or not?

[dimasciput]

@helendallas I have made sure that the script, which imports 'frog' and 'odonata', runs on the 1st of every month

[helendallas]

A few issues:

1) I don't think this has worked as there are less records now, and no 2022 ones for Odonates, and no 2023 for frogs.

There are 2023 records on VM for both - please can you check again. Here are the records in the last 4 weeks on the VM website

OdonataMap https://vmus.adu.org.za/vm_view_db.php

FrogMap https://vmus.adu.org.za/vm_view_db.php

2) VM data were also added to invertebrates - this is incorrect. However, please see ticket #3624 . Can we please discuss this as an option and action is feasible.

So Odonata (adults) would be removed, and the VM data added to Invertebrates as a separate data source (same way as Anurans).

[dimasciput]

@helendallas ok thanks, let me try to re harvest the data and store the logs, to see what went wrong.

[dimasciput]

@helendallas I have fixed this issue for frog data in testing https://testing.healthyrivers.kartoza.com/map/#site-detail/taxon=&search=&siteId=&collector=&category=&yearFrom=&yearTo=&months=&boundary=&userBoundary=&referenceCategory=&spatialFilter=&reference=&endemic=&conservationStatus=[]&modules=36&validated=&sourceCollection=[%22fbis%22,%22gbif%22,%22virtual_museum%22]&module=occurrence&ecologicalCategory=&rank=&siteIdOpen=&orderBy=name&polygon=&dst=&ecosystemType=River,Wetland,Open%20waterbody,Unspecified

Will deploy to production

[helendallas]

Still not fixed for odonates - in testing I used Biodiversity = Invertebrates, and Data Source = Virtual Museum.

Records returned only until 2021.

[dimasciput]

@helendallas I'm reharvesting the odonates data in production now, after that I will merge the odonates and invertebrates

[dimasciput]

Hi @helendallas, the issue with the data not being updated is due to their server frequently going down during the harvesting process. We can do something on our end, but updating the script might take some time. However, it would be more effective if their server could maintain stability consistently.

[helendallas]

Okay thanks @dimasciput I will email Rene and see what the problem is.

[helendallas]

Helen has emailed Rene Navarro (rene.navarro@uct.ac.za) to check on stability of the server.

[helendallas]

Followed on in ticket #3872 , so closing this ticket