search

OCHA-DAP / ds-raster-pipelines

1 stars 0 forks source link

FloodScan - dealing with update interruptions? #46

Open zackarno opened 2 weeks ago

zackarno commented 2 weeks ago

You may have already dealt with this, but just wanted to make sure as it's something I had to account for already in the dev pipeline i set up here

Basically, it's not uncommon for there to be delay in updating, or perhaps the zip just doesn't appear on a particular day. You can see that reflected in the failed runs of the GHA link above. While I probably could have handled those failed runs more gracefully what I did was everytime a new cog is added, it checks for any date gaps. If there are any they get filled with the latest unzipped cogs. This way it keep a continuous time-series with no gaps even if certain days are missed.

isatotun commented 1 week ago

I am working on a solution for this to be handled more automatically but for now we can just run for the missing dates when we get a new file eventually.