GeospatialCentroid / Colorado-EnviroScreen

Code base to support the development of a environmental health screening tool for the state of Colorado.
1 stars 0 forks source link

how to determine if new data has been added #16

Open dcarver1 opened 3 years ago

dcarver1 commented 3 years ago

Need to refine this questions but here the main idea.

How does an analysis determine if they need to rerun all function and push an updated resource to the public site. Two means -1. test for updates on all individual sources

I only want to apply option one for datasets that are updated manually. When you do that just rerun the process and update data on the application

If the data processing is fast enough, I'd suggest running the data processing code once every month and using something described here https://stackoverflow.com/questions/43081791/in-r-find-whether-two-files-differ/53333988 answer by stephanmg to determine if the new output file is different from the previous, if so the new file is save if not the previous file remains and now updates are needed.