Request: auto refresh node cache on upload conflict

[animetosho]

I have a script which automatically uploads to ACD via the stream option. Before it does so, it always checks whether the file already exists, before uploading.

When the upload fails, the script will attempt to re-upload it later. Unfortunately, if the upload "failed" due to an ACDCLI crash, ACD failure or otherwise, retries may also fail as ACD may have the file, but ACDCLI's local DB cache does not, and the following error occurs:

16-06-18 03:48:32.354 [ERROR] [acd_cli] - Error uploading stream. RequestError: 409, {"logref":"0bffa061-34f6-11e6-bb2c-5dab154d5eec","message":"Node with the name zzzzzz already exists under parentId xxxxxxxxxxxxxxxxxxxxxx conflicting NodeId: yyyyyyyyyyyyyyyyyyyyyy","code":"NAME_ALREADY_EXISTS","info":{"nodeId":"yyyyyyyyyyyyyyyyyyyyyy"}}

To the uploading script, it sees it as a failure (non-zero return code) and will try to re-submit the upload at some later point.

It would be nice if ACDCLI could detect this error and automatically cache the details for the referenced node ID, since it's obviously out of sync at this point.

Thank you for the great tool!

[vb0]

This is actually getting critical since the cache seems to get out of sync (at least with a couple files) at each interrupted sync (which happens more and more often recently - I suspect because of https://github.com/yadayada/acd_cli/issues/416 ).

[jerblack]

I'm getting the same thing, but it's not possible for it to be a cache out of sync issue since I am manually deleting your database before I sync and it still happens. To be clear, these are the steps I take to see this error:

1. Remove every file except oauth_data from .cache/acd_cli
2. Run the following

acd_cli sync
acd_cli create backup/LnLqMsdfgsdXwd,A5Xvktui,thU/6OYEdX29Ywrbtertr,md5kcCf/fzk1mHvyic-cLjceNqwertbebdpz1G4-Jx0,2p,Jl52w-qZ,

Then I get this error:

16-10-28 13:26:12.225 [WARNING] [acd_cli] - Node "fzk1mHvyic-cLjceNqwertbebdpz1G4-Jx0,2p,Jl52w-qZ,," already exists. 
RequestError: 409, {
"code":"NAME_ALREADY_EXISTS",
"logref":"c479ca84-9d4c-11e6-a6f0-ddfbb612694b",
"message":"Node with the name fzk1mHvyic-cLjceNqwertbebdpz1G4-Jx0,2p,Jl52w-qZ, already exists under parentId LasdfabtQneH-YE5tMrqOA conflicting NodeId: CE4HS-9oS7qtmysnJ0Ezs-g5hA",
"info":{"nodeId":"CE4HS-9oSimt4Ezs-g5hA"}}

When I attempt to acd_cli ls the same folder, I get

16-10-28 13:36:34.758 [CRITICAL] [acd_cli] - Could not resolve path "/backup/LnLqMfkN9Xwd,A5XFCSsSihU/6OYEdX29YEjHuGV8,md5kcCf/fzk1mHvyic-cLjceNqYzCP1dpz1G4-Jx0,2p,Jl52w-qZ,".

This folder is not visible from an acd_cli mount on another machine, but it is visible from the web interface. From the web interface, If I move the files out of that folder, delete and recreate the folder with the same same, and then attempt acd_cli create against that new folder (yes, I know it already exists) then the failure is a much more reasonable 16-10-28 13:26:12.800 [WARNING] [acd_cli] - Folder "fzk1mHvyic-cLjceNqwertbebdpz1G4-Jx0,2p,Jl52w-qZ," already exists.

[shadycuz]

Do we have a fix for this? I see it mentioned several places here on github

xxxx@xxxxx:~/.scripts$ bash uploadsync.sh

Starting script Fri Apr 7 11:21:47 EDT 2017 Unmounting Drive Drive Unmounted Caution Sync Syncing... Done. Starting upload command 17-04-07 11:28:28.933 [ERROR] [acd_cli] - Uploading "J0AuyTbFnr1,MXaZQuvuEVJ6r9n PJlReYO7vQT9a15F1hCrFTtOhrAkIB6u,jEREZBdwhRx28saB5iQQYqo9xHXT" failed. RequestEr ror: 1000, HTTPSConnectionPool(host='content-na.drive.amazonaws.com', port=443): Read timed out. (read timeout=60). 17-04-07 11:28:32.193 [ERROR] [acd_cli] - Uploading "KW0yb-7c5r6x8sCSM3uoLmHT8gk HoflOQMXn,WIouBqAmDXtHFTT7JuYQIFMGa5HMRqO0pOd-Gx4EU88n4akDKjj" failed. RequestEr ror: 1000, HTTPSConnectionPool(host='content-na.drive.amazonaws.com', port=443): Read timed out. (read timeout=60). 17-04-07 11:29:03.809 [ERROR] [acd_cli] - Uploading "J0AuyTbFnr1,MXaZQuvuEVJ6r9n PJlReYO7vQT9a15F1hCrFTtOhrAkIB6u,jEREZBdwhRx28saB5iQQYqo9xHXT" failed. Name coll ision with non-cached file. If you want to overwrite, please sync and try again 17-04-07 11:29:03.819 [WARNING] [acdcli.api.backoff_req] - Waiting 7.196986s bec ause of error(s). 17-04-07 11:30:58.071 [ERROR] [acd_cli] - Uploading "NlnoVAa4m51iUQkuz0cQGvOVhmh 2sqQoE1BWkyOCSRWylGSmdHPQF4Bkm6jtqeugSyFMPwU3rDtNtt1SDaH68BJW" failed. RequestEr ror: 1000, HTTPSConnectionPool(host='content-na.drive.amazonaws.com', port=443): Read timed out. (read timeout=60). 17-04-07 11:31:43.017 [ERROR] [acd_cli] - Uploading "NlnoVAa4m51iUQkuz0cQGvOVhmh 2sqQoE1BWkyOCSRWylGSmdHPQF4Bkm6jtqeugSyFMPwU3rDtNtt1SDaH68BJW" failed. Name coll ision with non-cached file. If you want to overwrite, please sync and try again 17-04-07 11:31:43.035 [WARNING] [acdcli.api.backoff_req] - Waiting 20.930934s be cause of error(s). 17-04-07 11:38:22.413 [ERROR] [acd_cli] - Uploading "KW0yb-7c5r6x8sCSM3uoLmHT8gk HoflOQMXn,WIouBqAmDXtHFTT7JuYQIFMGa5HMRqO0pOd-Gx4EU88n4akDKjj" failed. Name coll ision with non-cached file. If you want to overwrite, please sync and try again 17-04-07 11:42:09.892 [ERROR] [acd_cli] - Uploading "Xc1NauzAK5qWNPCPeRpnJdWqqd5 nmk,g4KiG-cxEA7,mhWphKp1yLyGLtZyj0IQReQekyrtFRGE6Asygrzfd3nJFxAftt,tAPiEi73OCyeG gn-" failed. RequestError: 1000, HTTPSConnectionPool(host='content-na.drive.amaz onaws.com', port=443): Read timed out. (read timeout=60). [#########################] 100.0% of 16GiB 38/38 0.0B/s 0s 3 file(s) failed. Syncing New Files Syncing... Done. Remounting ACD

xxxx@xxxxx:~/.scripts$ bash uploadsync.sh

Starting script Fri Apr 7 11:48:33 EDT 2017 Starting Delete Files CMD Unmounting Drive Drive Unmounted Caution Sync Syncing... Done. Starting upload command [#########################] 100.0% of 16GiB 38/38 5.7GB/s 0s Syncing New Files Syncing... Done. Remounting ACD

Thhe files were created on my ACD but they are blank...

If I delete them. They still don't reupload?

Thanks, Levi