Automate pipeline - Githubissues

dali-lab / pine-beetle-backend

Visualizing data and predicting outbreaks of Southern Pine Beetles across twenty US states. Built in the DALI Lab.

https://spbpredict.com

0 stars 0 forks source link

Automate pipeline #9

Closed AnujVarma closed 4 years ago

AnujVarma commented 4 years ago

The pipeline of converting the survey123 data into summarized database data is a little buggy- Working on automating the process

amanda-bak commented 4 years ago

Create a way for pipeline to automatically pull data from Survey123 and store it in our database, as Survey123 is updated throughout the trapping season.

amanda-bak commented 4 years ago

Option 1: Our server updates in real time as Survey123 updates

Can't find documentation for this

Option 2: Our server runs a weekly check to pull data from Survey 123

When data is pulled, we check what has changed and save that to our database
This can be automated (i.e. does every week on Sunday at 5pm)
Process of checking what data has changed might be a little tricky