search

agrc / forklift

:tractor::package::sparkles: Slinging data all over the place :tractor::package::sparkles:
MIT License
28 stars 3 forks source link

Possible Lost Updates #369

Open stdavis opened 5 months ago

stdavis commented 5 months ago

We have a design flaw that makes it easy to miss updates that are pushing during ship.

For example, if you have a pallet that fails during ship and doesn't push the updates, the next lift will run and if the data hasn't changed since the previous lift, it will never push the updated version of the data.

steveoh commented 5 months ago

Does this only affect pallets that explicitly push their data somewhere?

I want to say that forklift's original design was to leave the updated data in staging so the next push included the updated data to the destination servers.

stdavis commented 5 months ago

It does leave the data in the hashed folder. But it won't ever get pushed to production until there is another change to the FGDB in hashed. And for other pallets that do something during ship, like AGOL, it doesn't happen until there is another change to the specific dataset.