Open ashmrtn opened 1 year ago
also related to
as it requires knowing when kopia actually needs data
thinking about this more, it also has implications for how backup details for these items are handled. A plan for that should be pinned down prior to starting work on this
Related also to #3007
This (along with #3007) will help with scenarios such as:
Now that we added support for sourcing details entries from assist bases, the requirements for enabling kopia assisted incrementals have become a bit easier to meet. Currently the requirements are:
Since we merge assist base details, we need only know the mod time and item name when first telling kopia about the item. We can put off materializing the ItemInfo until we know kopia read the file data and it wasn't cached
However, this leaves us with a couple new questions (numbered for ease of reference):
For the first case, we can just return an error when opening the lazy reader. This will be caught by kopia and handled similar to an error fetching OneDrive/SharePoint data
For the second case, we need to be a little more careful (and this needs validated through manual testing). We should probably do three things:
This should be sufficient because (in theory, needs validation in practice) the next backup should get a delta result saying the item was deleted. That delta result will cause us to exclude the item in kopia. The details entry for the (now excluded) item shouldn't exist anyway so the item will disappear completely from the new backup
Although having to check for sentinel errors in the kopia code is a bit messy, it does allow us to avoid having details entries that don't have data to back them
The error check can also be leveraged down the road to make sure the details are properly initialized when we return them (i.e. return an error if details was requested but the item data was never read)
If we go this route, the below help setup the error return value from the Info call
re-opening this since we want to disable the feature and get more test coverage on it
un-assigning for the moment since there's additional work that needs to be done to get this enabled
Kopia-assisted incrementals allows us to skip uploading items that kopia already has to kopia again. However, the logic for GraphConnector still fetches all item data. This data fetch eats into the number of requests that graph API allows us
Instead of always fetching item data, we should only fetch the data if kopia requests it