Open gh123man opened 9 years ago
I have the same issue.
@gh123man
Thanks for your efforts so far on this. @xybu has limited availability at the moment, and at this point you probably have more experience with the live api than myself. I was going to assign you the issue, but it seems I don't have rights to do so. I'd appreciate your thoughts on what functionality we should be testing before pulling your change into future branch? thanks!
@kutsy
Would you be able to provide testing feedback of the solution that @gh123man has proposed?
Would you be able to provide testing feedback of the solution that @gh123man has proposed?
I rebuilded using this patch gh123man@6b369b7 and all my files from OneDrive were renamed from TestFile.png
to TestFile (andromeda, 1).png
, TestFile (andromeda, 2).png
and so on.
Where "andromeda" - its my local computer name.
After few hours of uploading I always have the same exception for workers one-by-one:
Exception in thread worker74:
Traceback (most recent call last):
File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.4/dist-packages/onedrive_d-1.1.0.dev0-py3.4.egg/onedrive_d/od_worker_thread.py", line 446, in run
self.upload_file(task)
File "/usr/local/lib/python3.4/dist-packages/onedrive_d-1.1.0.dev0-py3.4.egg/onedrive_d/od_worker_thread.py", line 375, in upload_file
local_path=task['local_path'])
File "/usr/local/lib/python3.4/dist-packages/onedrive_d-1.1.0.dev0-py3.4.egg/onedrive_d/od_onedrive_api.py", line 579, in put
raise OneDriveAPIException(ret)
onedrive_d.od_onedrive_api.OneDriveAPIException: An error occurred while performing the action. Try again later. (server_internal_error)
Log became to this:
[2015-07-23 20:29:22,342] INFO: worker74: network connection error.
[2015-07-23 20:29:22,342] INFO: worker74: put to sleep due to networking error.
[2015-07-23 20:29:22,513] DEBUG: thread_mgr: able to realize "onedrive.com:80".
[2015-07-23 20:29:22,513] INFO: worker74: waken up by ThreadManager.
For fast workarond I've increase workers count to 256 and restart onedrive-d daemon every 8-10 hours.
@kutsy
Would you be able to provide testing feedback of the solution that @gh123man has proposed?
I rebuilded using this patch gh123man@6b369b7 and all my files from OneDrive were renamed from >TestFile.png to TestFile (andromeda, 1).png, TestFile (andromeda, 2).png and so on. Where "andromeda" - its my local computer name.
This happens when you erase your local database and thus onedrive-d has a hard time resolving duplicates. It should not be a side affect of my changes.
@ewann this issue has to do with the API used for breaking files into chunks (I think this is the API, but I am looking at this quickly: https://msdn.microsoft.com/en-us/library/dn858877.aspx). It happens I believe on the very last call to finalize the upload. So you should be testing that particular functionality. I will reply back with more details if/when I get a chance to revisit this. I have not looked into this issue for a while.
As a note, the last call will be likely to fail if the access token expires. The server throws error despite that you renew the token. The new onedrive API seems to not have such issue.
Sent from my iPhone
On Jul 28, 2015, at 2:24 PM, Brian Floersch notifications@github.com wrote:
@ewann this issue has to do with the API used for breaking files into chunks (I think this is the API, but I am looking at this quickly: https://msdn.microsoft.com/en-us/library/dn858877.aspx). It happens I believe on the very last call to finalize the upload. So you should be testing that particular functionality. I will reply back with more details if/when I get a chance to revisit this. I have not looked into this issue for a while.
— Reply to this email directly or view it on GitHub.
@Dr-Xperience
This will get fixed with introduction of the new OneDrive API.
I have not been able to dig too deep into this yet, but all the worker theads are crashing after finishing uploading some large files (2.1gb).
This is the exception being thrown:
As far as I can tell, the parameter:
file.8db3599e7fa97.8db3599e7fa97!96571
is somehow malformed. Each of the first two id's are 3 characters short? (basing this theory off of API documentation/reference).UPDATE
I am not super familiar with the live API, but it did appear to be a malformed request ID. The format
file.<file_id>.<file_id>
does not seem to work. Whereas the formatfile.<folder_id>.<file_id>
does seem to work. The commit here: https://github.com/gh123man/onedrive-d/commit/6b369b73756aa684629180eafcd44d72bfbbeefc fixes the bug. If you are okay with this, you can assign the bug to me and Ill submit a pull request. If there is a better or more correct way to fix this, please let me know.