search

abraunegg / onedrive

OneDrive Client for Linux
https://abraunegg.github.io
GNU General Public License v3.0
10.17k stars 865 forks source link

Conversion overflow on sync #1367

Closed ShaunFell closed 3 years ago

ShaunFell commented 3 years ago

Bug Report Details

When synchronizing or monitoring, program fails to sync with error std.conv.ConvOverflowException@std/conv.d(544): Conversion positive overflow

Application and Operating System Details:

curl 7.75.0 (x86_64-pc-linux-gnu) libcurl/7.75.0 OpenSSL/1.1.1j zlib/1.2.11 zstd/1.4.9 libidn2/2.3.0 libpsl/0.21.1 (+libidn2/2.3.0) libssh2/1.9.0 nghttp2/1.43.0
Release-Date: 2021-02-03
Protocols: dict file ftp ftps gopher gophers http https imap imaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: alt-svc AsynchDNS GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets zstd

Dual boot system with linux on m.2 and windows on 2.5 in SSD. Onedrive folder is in the Windows volume. Folder is not shared. Onedrive accoutn is shared between my desktop and laptop. Desktop config is shown above. laptop config is similar.

debug log:

[DEBUG] homePath: HOME environment variable set
[DEBUG] homePath: /home/shaunf
[DEBUG] configDirBase: WARNING - no XDG_CONFIG_HOME environment variable set
[DEBUG] configDirBase: /home/shaunf/.config
[DEBUG] configDirName: Configuring application to use default config path
Using 'user' Config Dir: /home/shaunf/.config/onedrive
Using 'system' Config Dir: /etc/onedrive
[DEBUG] refreshTokenFilePath = /home/shaunf/.config/onedrive/refresh_token
[DEBUG] deltaLinkFilePath = /home/shaunf/.config/onedrive/delta_link
[DEBUG] databaseFilePath = /home/shaunf/.config/onedrive/items.sqlite3
[DEBUG] databaseFilePathDryRun = /home/shaunf/.config/onedrive/items-dryrun.sqlite3
[DEBUG] uploadStateFilePath = /home/shaunf/.config/onedrive/resume_upload
[DEBUG] userConfigFilePath = /home/shaunf/.config/onedrive/config
[DEBUG] syncListFilePath = /home/shaunf/.config/onedrive/sync_list
[DEBUG] systemConfigFilePath = /etc/onedrive/config
[DEBUG] businessSharedFolderFilePath = /home/shaunf/.config/onedrive/business_shared_folders
Configuration file successfully loaded
[DEBUG] sync_dir: Getting syncDir from config value sync_dir
[DEBUG] syncDir: /mnt/Windows/Users/drums/OneDrive
[DEBUG] Testing network to ensure network connectivity to Microsoft OneDrive Service
[DEBUG] Attempting to contact online service
[DEBUG] Shutting down HTTP engine as sucessfully reached OneDrive Online Service
Initializing the OneDrive API ...
Configuring Global Azure AD Endpoints
[DEBUG] Configured authUrl:          https://login.microsoftonline.com/common/oauth2/v2.0/authorize
[DEBUG] Configured redirectUrl:      https://login.microsoftonline.com/common/oauth2/nativeclient
[DEBUG] Configured tokenUrl:         https://login.microsoftonline.com/common/oauth2/v2.0/token
[DEBUG] Configured driveUrl:         https://graph.microsoft.com/v1.0/me/drive
[DEBUG] Configured driveByIdUrl:     https://graph.microsoft.com/v1.0/drives/
[DEBUG] Configured sharedWithMeUrl:  https://graph.microsoft.com/v1.0/me/drive/sharedWithMe
[DEBUG] Configured itemByIdUrl:      https://graph.microsoft.com/v1.0/me/drive/items/
[DEBUG] Configured itemByPathUrl:    https://graph.microsoft.com/v1.0/me/drive/root:/
[DEBUG] Configured siteSearchUrl:    https://graph.microsoft.com/v1.0/sites?search
[DEBUG] Configured siteDriveUrl:     https://graph.microsoft.com/v1.0/sites/
[DEBUG] Downgrading all HTTP operations to HTTP/1.1 by default
[DEBUG] clientId    = d50ca740-c83f-4d1b-b616-12c519384f0c
[DEBUG] companyName = abraunegg
[DEBUG] appTitle    = OneDrive Client for Linux
Opening the item database ...
[DEBUG] Using database file: /home/shaunf/.config/onedrive/items.sqlite3
[DEBUG] Configuring default new folder permissions as: 700
[DEBUG] Configuring default new file permissions as: 600
All operations will be performed in: /mnt/Windows/Users/drums/OneDrive
[DEBUG] Configuring skip_dir ...
[DEBUG] skip_dir: 
[DEBUG] Configuring skip_dir_strict_match ...
[DEBUG] skip_dir_strict_match: false
[DEBUG] Configuring skip_dotfiles ...
[DEBUG] skip_dotfiles: true
[DEBUG] Configuring skip_file ...
[DEBUG] skip_file: ~*|.~*|*.tmp|*.ini|
[DEBUG] Setting file permissions for: /home/shaunf/.config/onedrive/refresh_token
[DEBUG] Request URL = https://graph.microsoft.com/v1.0/me/drive
[DEBUG] Request URL = https://graph.microsoft.com/v1.0/me/drive/root
[DEBUG] OneDrive Account Details:      {"@odata.context":"https:\/\/graph.microsoft.com\/v1.0\/$metadata#drives\/$entity","driveType":"personal","id":"fbe031eb7598ec71","owner":{"user":{"displayName":"Shaun Fell","id":"fbe031eb7598ec71"}},"quota":{"deleted":1242090197,"remaining":1849959515,"state":"normal","storagePlanInformation":{"upgradeAvailable":true},"total":5368709120,"used":3518749605}}
[DEBUG] OneDrive Account Root Details: {"@odata.context":"https:\/\/graph.microsoft.com\/v1.0\/$metadata#users('shaundbfell%40gmail.com')\/drive\/root\/$entity","cTag":"adDpGQkUwMzFFQjc1OThFQzcxITExMy42Mzc1MjM1MTYwMTQ2MzAwMDA","createdBy":{"application":{"displayName":"OneDrive website","id":"44048800"},"user":{"displayName":"Shaun Fell","id":"fbe031eb7598ec71"}},"createdDateTime":"2019-04-16T01:20:14.313Z","eTag":"aRkJFMDMxRUI3NTk4RUM3MSExMTMuMA","fileSystemInfo":{"createdDateTime":"2019-04-16T01:20:14.313Z","lastModifiedDateTime":"2019-04-16T01:20:14.313Z"},"folder":{"childCount":5,"view":{"sortBy":"name","sortOrder":"ascending","viewType":"thumbnails"}},"id":"FBE031EB7598EC71!113","lastModifiedBy":{"application":{"displayName":"OneDrive website","id":"44048800"},"user":{"displayName":"Shaun Fell","id":"fbe031eb7598ec71"}},"lastModifiedDateTime":"2021-03-26T10:33:21.463Z","name":"root","parentReference":{"driveId":"fbe031eb7598ec71","driveType":"personal"},"root":{},"size":3518749605,"webUrl":"https:\/\/onedrive.live.com\/?cid=fbe031eb7598ec71"}
Application version: v2.4.10
Account Type: personal
Default Drive ID: fbe031eb7598ec71
Default Root ID: FBE031EB7598EC71!113
Remaining Free Space: 1849959515
Fetching details for OneDrive Root
[DEBUG] Request URL = https://graph.microsoft.com/v1.0/me/drive/root
[DEBUG] lastModifiedDateTime (OneDrive item): 2019-Apr-16 01:20:14.313Z
OneDrive Root exists in the database
Trying to restore the upload session ...
[DEBUG] Request URL = https://df.api.onedrive.com/rup/fbe031eb7598ec71/eyJSZXNvdXJjZUlEIjoiRkJFMDMxRUI3NTk4RUM3MSExODgwMCIsIlJlbGF0aW9uc2hpcE5hbWUiOiJTY3JhdGNoLm5iIn0/4mYExKZBhn2_306U_RCsKOHRNNnSLg50ikpoz_zgLXtgOXp5Q-cub0GHyUAWpgHCIFjpRAHe_CkoU0GZ3mg1ep1SnEEeWdYUwabFZn7fCz1e0/eyJuYW1lIjoiU2NyYXRjaC5uYiIsImZpbGVTeXN0ZW1JbmZvIjp7Ikxhc3RNb2RpZmllZERhdGVUaW1lIjoiMjAyMS0wMy0yM1QyMjozNDozNyswMDowMCJ9LCJAbmFtZS5jb25mbGljdEJlaGF2aW9yIjoicmVwbGFjZSJ9/4wTjaE7VYdqL6lGBRqDXV_IC4Jd1w_B4xXe7wJLQItX3gLJrUhlITGU4dzi-P5RVeguXSjIFw-4LWUYyxivk8-L5KDs6o69p9dwjgAz9PGRA5y2zvYxcFPeOexkEUBGZFl0FgsytBBOSoKRae8KtU8oQeh76E5Cayy79vPYy3E75Ve4OdkKZi36iUR7Yc3XcImjk_8bbqMDj1kpUq-agOGVmdLZ1jtjzkwgS65gZOvQSGK0mgEwPaGFr2JCNdVrrhId39vGtgEkxYs7q0FExNxVJ1FwTE8M9HNhQ13MqepSvlJFBn2xPCq12c87cFMIhQxlLN05ymYi764JDb2RS3UtkLH-4r5r5Pylkj0H9_kftImZozIhX-C0sJQupLKtxyO63AWQj2j_y3W_zNiL5Si1EHO_2Ju4msnb7BSSLTx6XMtZ7Y1M0ZtJmq8doFO_yfReZPPXfKni43ad8emVV-24BB-lFIDPyry4vDaydKCgfnPcKTvMVlYcxfxfau6rdPSWCqEz7Pdmt03dcOQ8ioPdLBFpm2KG9VqlxLDyliMiD8iWSkpN_YPsOQ83DfpbWklDfARA7ePbI9PvZjgHxfEiA
Continuing the upload session ...
Uploading   0% |                                        |   ETA   --:--:--:     
[DEBUG] Fragment: 1 of 1
Uploading 100% |oooooooooooooooooooooooooooooooooooooooo| DONE IN 00:00:00 std.conv.ConvOverflowException@std/conv.d(544): Conversion positive overflow
----------------
??:? pure @safe bool std.exception.enforce!(bool).enforce(bool, lazy object.Throwable) [0x55798faa2d9d]
??:? pure @safe long std.conv.toImpl!(long, ulong).toImpl(ulong) [0x55798fc12d44]
??:? pure @safe long std.conv.to!(long).to!(ulong).to(ulong) [0x55798fc12d04]
??:? @property void std.net.curl.HTTP.contentLength(ulong) [0x55798fbebe00]
??:? std.json.JSONValue onedrive.OneDriveApi.uploadFragment(const(char)[], immutable(char)[], long, long, long) [0x55798fb774a1]
??:? std.json.JSONValue upload.UploadSession.upload() [0x55798fb9beea]
??:? void sync.SyncEngine.init() [0x55798fb81708]
??:? bool main.initSyncEngine(sync.SyncEngine) [0x55798fb70918]
??:? _Dmain [0x55798fb6e672]
std.net.curl.CurlException@std/net/curl.d(4402): A libcurl function was given a bad argument on handle null
----------------
??:? pure @safe void std.exception.bailOut!(std.net.curl.CurlException).bailOut(immutable(char)[], ulong, scope const(char)[]) [0x55798fc16e06]
??:? pure @safe bool std.exception.enforce!(std.net.curl.CurlException).enforce!(bool).enforce(bool, lazy const(char)[], immutable(char)[], ulong) [0x55798fc16d77]
??:? void std.net.curl.Curl._check(int) [0x55798fbec4ff]
??:? void std.net.curl.Curl.clear(etc.c.curl.CurlOption) [0x55798fbec8ae]
??:? void std.net.curl.HTTP.clearRequestHeaders() [0x55798fbeb5d3]
??:? void onedrive.OneDriveApi.shutdown() [0x55798fb74dcc]
??:? _Dmain [0x55798fb7029a]

might be related to https://github.com/abraunegg/onedrive/issues/1365 ? Didnt notice onedrive was failing to sync for a couple days. Only noticed when a file I edited on laptop wasnt syncing with the file on my desktop.

abraunegg commented 3 years ago

@Shaun-F Another weird issue ... but not related to #1365

Due to the time, will look at this in the morning - sorry.

abraunegg commented 3 years ago

@Shaun-F I am unable to reproduce this on Manjaro building from AUR

If you are able to 100% reproduce this, I will need to develop a special PR to gather more details to help diagnose this issue to develop a fix. Please can you advise if this is still an issue / problem for you.

ShaunFell commented 3 years ago

@abraunegg Thanks for looking into it. strange you werent able to reproduce. Could be an issue with my specific setup. Last time I checked, it was still an issue. I will get back to you in the morning and will try debugging on my end to hopefully find the bug

abraunegg commented 3 years ago

@Shaun-F

Thanks for looking into it. strange you werent able to reproduce. Could be an issue with my specific setup.

This type of issue potentially could be a compiler change - which is why I cant reproduce, but if you can identify what compiler was used when you built the client (DMD or LDC) and what version, this might help track down the change + assist with being able to reproduce.

In terms of the actual issue, I have made a code tweak in the below PR, but again as I cannot reproduce this, I cannot say if this is the fix or not ...

When you do your testing, please use the following PR #1368:

git clone https://github.com/abraunegg/onedrive.git
cd onedrive
git fetch origin pull/1368/head:pr1368
git checkout pr1368
./configure; make clean; make;

To run the PR, you need to run the client from the PR build directory:

./onedrive <any options needed>

When running the PR, your version should be: onedrive v2.4.10-24-gf77e012 or greater.

ShaunFell commented 3 years ago

Just checked this morning and miraculously the bug disappeared and I am now able to sync without this error. Not sure what changed between yesterday and today, I dont think I updated any libraries. If the error pops up again, I will try your pull request and update this issue. Thanks for the help anyways!

github-actions[bot] commented 3 years ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.