inbo / alien-species-portal

Portal for alien and invasive species indicators

MIT License

0 stars 0 forks source link

[Enhancement] Active hotbeds reworks #103

Open SanderDevisscher opened 2 weeks ago

SanderDevisscher commented 2 weeks ago

Feature title

Species infromation > Management > Active hotbeds

Description

The active hotbeds map should display a hot map of recent (last 2 weeks) observations independent of management status of nearby nests. This should occur at a high update frequency (daily) to allow for rapid response.

(Optional) Needed changes in backoffice

https://github.com/inbo/aspbo/issues/173

(Optional) Suggestion how to implement change

Might be naive but I think this can be achieved by doing a daily pull of the waarnemingen.be API. We have the necessary credentials and 3 example scripts to get access.

the example scripts: waarnemingenAPI.zip

Alternatively, with some minor changes to these scripts, we can create a GDPR proof version of the data in a googlesheet. Which do you prefer?

mvarewyck commented 1 week ago

Might be naive but I think this can be achieved by doing a daily pull of the waarnemingen.be API. We have the necessary credentials and 3 example scripts to get access.

the example scripts: waarnemingenAPI.zip

Alternatively, with some minor changes to these scripts, we can create a GDPR proof version of the data in a googlesheet. Which do you prefer?

@SanderDevisscher Yes, based on the scripts you shared, you can do a daily API pull. I see no need to write the data to a googlesheet, I would just add the data to the S3 bucket with a script from aspbo.

SanderDevisscher commented 1 week ago

@mvarewyck best to put it on the PRD bucket directly then ?

mvarewyck commented 1 week ago

@mvarewyck best to put it on the PRD bucket directly then ?

Yes, if you don't want to do a daily PR from UAT to PRD. Then in the data processing script I would

add some checks on the data structure (e.g. no. of rows >0, column names) before the data is pushed to the bucket.
when the checks fail and there is no update of the data, make sure we get a notification (automatically with github actions?)

SanderDevisscher commented 1 week ago

@mvarewyck 0 rows is a real possibility in the winter when they are not active. The app should display a disclaimer when this the case.

@TheJenne18 can you confirm the PRD - bucket exists and can be writen to by github actions from the aspbo repo ?