Closed pcolmer closed 4 years ago
This is a change from the previous version I was using and I'm curious if this is deliberate or a bug. The current approach does mean that synchronizing takes more time than it used to and I wasn't aware of any consistency issues when I was using the previous version.
This is in response to fixing https://github.com/abraunegg/onedrive/issues/658
When using sync_list
this discards 'changes' and information about directories & files as they are stored on OneDrive. If you then logically move a folder from a directory that is forced out of scope by 'sync_list' into scope by moving it into a directory that is synced by 'sync_list' the OneDrive API will only send 1 change for that move - the change that represents the top level folder which was moved - not any change record for any children.
So what happens is - the new folder gets created locally, but no content in that folder is ever downloaded. This is because OneDrive 'thinks' the folder state is the same, and that the 'logical' move is what occured on the local file system as all the files were previously present.
I spent a lot of time looking into this (5-6 local different versions all trying to force the OneDrive API to send all those changes) and in the end this really is a OneDrive API issue (not a bug) that it does not send all the changes for when a folder is moved - and I say not a bug because if not using 'sync_list' then we get this change / move correctly, handling it, and move the folder from A-> B with all the data replicating OneDrive.
So the only way around this - is that when using 'sync_list' force a full scan. When using 'sync_list' as well, it should not do a double 'full scan' which is needless & wasteful as well.
Unfortunately there is zero way around this at this point in time, without totally re-writing the client & how the actual OneDrive state is being stored, including a flag per DB entry as to if this is in scope / out of scope is probably the solution, so that when something 'bit flips' to be in-scope, it can be checked locally, and if it does not exist, go download it. This sort of approach would bring back the prior faster execution & negate having to go and check for changes.
Apologies for the impact this is causing.
This happens even if there are no changes to be processed.
This has given me an idea to enhance the above fix a little - if using 'sync_list' and 0 changes waiting, don't do the full check (pointless). Let me work on a PR and provide that for testing.
@pcolmer Please can you test the following PR as an enhancement to reduce the needless scanning:
git clone https://github.com/abraunegg/onedrive.git
cd onedrive
git fetch origin pull/691/head:pr691
git checkout pr691
./configure; make clean; make;
@pcolmer Any chance in testing this PR yet?
I'm sorry I haven't yet had a chance to test this - I missed your PR comment as I was busy over the weekend. I'll find some time today.
Hi @abraunegg
I'm currently getting an error when I try to sync but please note that the error occurs on the current code as well as this PR.
I will email the full debug log to you.
Regards
Philip
@pcolmer Looked at the log file, will need to treat that as a separate issue. Marking this issue as fixed as evidenced in the debug log provided.
Closing issue as PR #691 merged into master
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Bug Report Details
Describe the bug Until recently, I'd been using the modified version that fixed https://github.com/abraunegg/onedrive/issues/603. I have recently been hitting some problems with the
onedrive
client reporting errors about missing expiration data so thought I'd update to the latest version.Now, however, whenever I run
onedrive --synchronize
, it seems to be processing all of the item states, e.g.:This happens even if there are no changes to be processed. This is a change from the previous version I was using and I'm curious if this is deliberate or a bug. The current approach does mean that synchronizing takes more time than it used to and I wasn't aware of any consistency issues when I was using the previous version.
Application and Operating System Details:
To Reproduce Steps to reproduce the behavior if not causing an application crash:
onedrive
so that it has a sync list.onedrive --synchronize
- watch how it processes a lot of items.Application Log Output:
I've only provided a partial output - it is just lots of the same over and over again.
Bug Report Checklist