search

abraunegg / onedrive

OneDrive Client for Linux
https://abraunegg.github.io
GNU General Public License v3.0
10.16k stars 864 forks source link

Bug: Incorrect backup target #2593

Closed JC-comp closed 10 months ago

JC-comp commented 10 months ago

Describe the bug

When moving files online, two problems occur:

  1. Hashes are missing from the API response (Not a bug actually, but there is potential for improvement) This is likely the cause of triggering an out-of-sync issue. If so, it can be resolved by querying the item by ID to obtain the hash value.
  2. The backup seems to be choosing the wrong target to preserve.

Operating System Details

Distributor ID: Ubuntu
Description:    Ubuntu 22.04.3 LTS
Release:        22.04
Codename:       jammy

Client Installation Method

From Source

OneDrive Account Type

Business | Office365

What is your OneDrive Application Version

v2.5.0-alpha-5 GitHub version: v2.4.25-36-gcda8317

What is your OneDrive Application Configuration

Reading configuration file: /home/alan/.config/onedrive/config
Invalid value for key in config file - using default value: monitor_fullscan_frequency
Configuration file successfully loaded
onedrive version                             = v2.5.0-alpha-5 GitHub version: v2.4.25-36-gcda8317
Config path                                  = /home/JC/.config/onedrive
Config file found in config path             = true
Config option 'drive_id'                     =
Config option 'sync_dir'                     = ~/OneDrive
Config option 'enable_logging'               = false
Config option 'log_dir'                      = /var/log/onedrive
Config option 'disable_notifications'        = false
Config option 'skip_dir'                     =
Config option 'skip_dir_strict_match'        = false
Config option 'skip_file'                    = ~*|.~*|*.tmp|*.swp|*.partial
Config option 'skip_dotfiles'                = false
Config option 'skip_symlinks'                = false
Config option 'monitor_interval'             = 300
Config option 'monitor_log_frequency'        = 12
Config option 'monitor_fullscan_frequency'   = 12
Config option 'read_only_auth_scope'         = false
Config option 'dry_run'                      = false
Config option 'upload_only'                  = false
Config option 'download_only'                = false
Config option 'local_first'                  = false
Config option 'check_nosync'                 = false
Config option 'check_nomount'                = false
Config option 'resync'                       = false
Config option 'resync_auth'                  = false
Config option 'cleanup_local_files'          = false
Config option 'classify_as_big_delete'       = 1000
Config option 'disable_upload_validation'    = false
Config option 'disable_download_validation'  = false
Config option 'bypass_data_preservation'     = false
Config option 'no_remote_delete'             = false
Config option 'remove_source_files'          = false
Config option 'sync_dir_permissions'         = 700
Config option 'sync_file_permissions'        = 600
Config option 'space_reservation'            = 52428800
Config option 'application_id'               = d50ca740-c83f-4d1b-b616-12c519384f0c
Config option 'azure_ad_endpoint'            =
Config option 'azure_tenant_id'              =
Config option 'user_agent'                   = ISV|abraunegg|OneDrive Client for Linux/v2.5.0-alpha-5
Config option 'force_http_11'                = false
Config option 'debug_https'                  = false
Config option 'rate_limit'                   = 0
Config option 'operation_timeout'            = 3600
Config option 'dns_timeout'                  = 60
Config option 'connect_timeout'              = 30
Config option 'data_timeout'                 = 240
Config option 'ip_protocol_version'          = 0

Selective sync 'sync_list' configured        = false

Config option 'sync_business_shared_items'   = false
Selective Business Shared Items configured   = false

Config option 'webhook_enabled'              = false

What is your 'curl' version

curl 7.81.0 (x86_64-pc-linux-gnu) libcurl/7.81.0 OpenSSL/3.0.2 zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 libidn2/2.3.2 libpsl/0.21.0 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.43.0 librtmp/2.3 OpenLDAP/2.5.16
Release-Date: 2022-01-05
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets zstd

Where is your 'sync_dir' located

Local

What are all your system 'mount points'

Skipped

What are all your local file system partition types

Skipped

How do you use 'onedrive'

Skipped

Steps to reproduce the behaviour

// move file to some other folder online $ onedrive --sync

Complete Verbose Log Output

DEBUG: Processing OneDrive JSON item 2 of 2 as part of JSON Item Batch 1 of 1
DEBUG: Attempting to calculate local filesystem path for af2b5e52bea96776 and AF2B5E52BEA96776!s6de0d0cd368943e4ba5bc3e591d937b6
DEBUG: New Item calculated full path is: a/mondate.txt
DEBUG: The item we are syncing is a file
DEBUG: skip_file item to check: /a/mondate.txt
DEBUG: skip_file evaluation for: /a/mondate.txt
DEBUG: Result: false
DEBUG: Flagging object as a file
DEBUG: OneDrive change is an update to an existing local item
DEBUG: Attempting to calculate local filesystem path for af2b5e52bea96776 and AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639
DEBUG: Moving ./mondate.txt to a/mondate.txt
DEBUG: The local item is out-of-sync with OneDrive, renaming to preserve existing file and prevent local data loss: a/mondate.txt -> a/mondate-55db26f69037.txt
DEBUG: Renaming of local file failed for a/mondate.txt: a/mondate-55db26f69037.txt: No such file or directory
DEBUG:
DEBUG: ERROR: The local file system returned an error with the following message:
DEBUG:   Error Message:    a/mondate.txt: No such file or directory
DEBUG:   Calling Function: applyPotentiallyChangedItem()

Screenshots

No response

Other Log Information or Details

No response

Additional context

No response

abraunegg commented 10 months ago

@JC-comp

Hashes are missing from the API response

Hashes should 100% be in the API response for a file object. That is the API standard, and if this is not happening is an API bug and needs to be flagged as such. Evidence please, and then raise your issue here: https://github.com/OneDrive/onedrive-api-docs

move file to some other folder online

FYI - local moves, when using '--sync' should result in an online delete at old location + re-upload to new location.

Please read: https://github.com/abraunegg/onedrive/blob/onedrive-v2.5.0-alpha-5/docs/known-issues.md#renaming-or-moving-files-in-standalone-mode-causes-online-deletion-and-re-upload-to-occur

You need to provide more details and redacted/cutdown log out put please ... the logging you have provided is not helpful.

JC-comp commented 10 months ago

Steps to reproduce the behaviour

  1. On local machine, performe a full sync with onedrive --sync
  2. Open Onedrive web interface, drag and move a file (mondate.txt in this case) to some other folder (a in this case)
  3. Back to local machine and run onedrive --sync
  4. Hit The local item is out-of-sync with OneDrive, renaming to preserve existing file and prevent local data loss (Probably caused by missing hash value)
  5. Encounter Renaming of local file failed for a/mondate.txt: a/mondate-55db26f69037.txt: No such file or directory

Note: Complete logs are provided at the end of the comment.

Hashes should 100% be in the API response for a file object.

According to Hashes resource type

Note: In some cases hash values may not be available. If this is the case, the hash values on an item will be updated after the item is downloaded.

In my case, the hashes are not included when files are moved online.

Full logs

DEBUG: Application started
DEBUG: Process ID: 7FB90ABC5A60
DEBUG: User UID:   1000
DEBUG: User GID:   1000
DEBUG: User Name:  alan
DEBUG: Application Version: v2.5.0-alpha-5 GitHub version: v2.4.25-36-gcda8317
DEBUG: Passed in 'cliArgs': ["./work/onedrive/onedrive", "--sync"]
DEBUG: Note: --confdir and --verbose are not listed in 'cliArgs' array
DEBUG: Passed in --confdir if present:
DEBUG: Passed in --verbose count if present: 3
DEBUG: runtime_environment: HOME environment variable detected, expansion of '~' should be possible
DEBUG: runtime_environment: Calculated defaultHomePath: /home/alan
DEBUG: configDirBase: WARNING - no XDG_CONFIG_HOME environment variable set
DEBUG: configDirBase: /home/alan/.config
DEBUG: configDirName: Configuring application to use calculated config path
DEBUG: refreshTokenFilePath =   /home/alan/.config/onedrive/refresh_token
DEBUG: deltaLinkFilePath =      /home/alan/.config/onedrive/delta_link
DEBUG: databaseFilePath =       /home/alan/.config/onedrive/items.sqlite3
DEBUG: databaseFilePathDryRun = /home/alan/.config/onedrive/items-dryrun.sqlite3
DEBUG: uploadSessionFilePath =  /home/alan/.config/onedrive/session_upload
DEBUG: userConfigFilePath =     /home/alan/.config/onedrive/config
DEBUG: syncListFilePath =       /home/alan/.config/onedrive/sync_list
DEBUG: systemConfigFilePath =   /etc/onedrive/config
DEBUG: configBackupFile =       /home/alan/.config/onedrive/.config.backup
DEBUG: configHashFile =         /home/alan/.config/onedrive/.config.hash
DEBUG: syncListHashFile =       /home/alan/.config/onedrive/.sync_list.hash
DEBUG: businessSharedItemsFilePath = /home/alan/.config/onedrive/business_shared_items
DEBUG: businessSharedItemsHashFile = /home/alan/.config/onedrive/.business_shared_items.hash
DEBUG: Reading configuration file: /home/alan/.config/onedrive/config
DEBUG: Invalid value for key in config file - using default value: monitor_fullscan_frequency
DEBUG: Configuration file successfully loaded
DEBUG: Using 'user' configuration path for application state data: /home/alan/.config/onedrive
DEBUG: Configuring Client Side Filtering (Selective Sync)
DEBUG: Configuring skip_dir ...
DEBUG: skip_dir:
DEBUG: Selective Sync Directory Mask: Regex!char("^$", "i")
DEBUG: Configuring skip_dir_strict_match ...
DEBUG: skip_dir_strict_match: false
DEBUG: Configuring skip_dotfiles ...
DEBUG: skip_dotfiles: false
DEBUG: Configuring skip_file ...
DEBUG: skip_file: ~*|.~*|*.tmp|*.swp|*.partial
DEBUG: Selective Sync File Mask: Regex!char("^~.*$|^\\.~.*$|^.*\\.tmp$|^.*\\.swp$|^.*\\.partial$", "i")
DEBUG: sync_dir: Setting runtimeSyncDirectory from config value 'sync_dir'
DEBUG: sync_dir: A '~' was found in the configured 'sync_dir', automatically expanding as SHELL and USER environment variable is set
DEBUG: sync_dir: runtimeSyncDirectory set to: /home/alan/OneDrive
DEBUG: Configuring default new folder permissions as: 700
DEBUG: Configuring default new file permissions as: 600
DEBUG: Creating a backup of the applicable config file
DEBUG: Updating applicable config file hash
DEBUG: Using IPv4 and IPv6 (if configured) for all network operations
DEBUG: Testing network to ensure network connectivity to Microsoft OneDrive Service
DEBUG: Attempting to contact Microsoft OneDrive Login Service
DEBUG: Shutting down HTTP engine as successfully reached OneDrive Login Service
DEBUG: Checking Application Version ...
DEBUG: applicationVersion:       2.4.25
DEBUG: latestVersion:            2.4.25
DEBUG: publishedDate:            2023-Jun-21 05:30:55Z
DEBUG: currentTime:              2024-Jan-16 20:55:26Z
DEBUG: releaseGracePeriod:       2023-Jul-21 05:30:55Z
DEBUG: Attempting to initialise the OneDrive API ...
DEBUG: Configuring Global Azure AD Endpoints
DEBUG: Authorised State: true
DEBUG: The OneDrive API was initialised successfully
DEBUG: Configured clientId          d50ca740-c83f-4d1b-b616-12c519384f0c
DEBUG: Configured userAgent         ISV|abraunegg|OneDrive Client for Linux/v2.5.0-alpha-5
DEBUG: Configured authScope:        &scope=Files.ReadWrite%20Files.ReadWrite.All%20Sites.ReadWrite.All%20offline_access&response_type=code&prompt=login&redirect_uri=
DEBUG: Configured authUrl:          https://login.microsoftonline.com/common/oauth2/v2.0/authorize
DEBUG: Configured redirectUrl:      https://login.microsoftonline.com/common/oauth2/nativeclient
DEBUG: Configured tokenUrl:         https://login.microsoftonline.com/common/oauth2/v2.0/token
DEBUG: Configured driveUrl:         https://graph.microsoft.com/v1.0/me/drive
DEBUG: Configured driveByIdUrl:     https://graph.microsoft.com/v1.0/drives/
DEBUG: Configured sharedWithMeUrl:  https://graph.microsoft.com/v1.0/me/drive/sharedWithMe
DEBUG: Configured itemByIdUrl:      https://graph.microsoft.com/v1.0/me/drive/items/
DEBUG: Configured itemByPathUrl:    https://graph.microsoft.com/v1.0/me/drive/root:/
DEBUG: Configured siteSearchUrl:    https://graph.microsoft.com/v1.0/sites?search
DEBUG: Configured siteDriveUrl:     https://graph.microsoft.com/v1.0/sites/
DEBUG: Opening the item database ...
DEBUG: Threadsafe database value: 1
DEBUG: Using Curl defaults for HTTP operational protocol version (potentially HTTP/2)
DEBUG: Read token from appConfig
DEBUG: Authorised State: true
DEBUG: Getting Account Default Drive Details
DEBUG: Microsoft OneDrive Access Token has EXPIRED. Must generate a new Microsoft OneDrive Access Token
DEBUG: Need to generate a new access token for Microsoft OneDrive
DEBUG: Updating appConfig.refreshToken with updated refreshToken
DEBUG: Updating refreshToken on disk
DEBUG: Setting file permissions for: /home/alan/.config/onedrive/refresh_token
DEBUG: Request URL = https://graph.microsoft.com/v1.0/me/drive
DEBUG: OneDrive Account Default Drive Details:      {"@odata.context":"https:\/\/graph.microsoft.com\/v1.0\/$metadata#drives\/$entity","createdBy":{"user":{"displayName":"SharePoint App"}},"createdDateTime":"2023-08-08T12:15:25Z","description":"","driveType":"personal","id":"af2b5e52bea96776","lastModifiedBy":{"user":{"displayName":"","email":"@hotmail.com"}},"lastModifiedDateTime":"2024-01-16T20:55:22Z","name":"OneDrive","owner":{"user":{"displayName":"","email":"@hotmail.com"}},"quota":{"deleted":0,"remaining":5364164150,"state":"normal","storagePlanInformation":{"upgradeAvailable":true},"total":5368709120,"used":4544970},"webUrl":"https:\/\/my.microsoftpersonalcontent.com\/personal\/af2b5e52bea96776\/Documents"}
DEBUG: appConfig.accountType        = personal
DEBUG: appConfig.defaultDriveId     = af2b5e52bea96776
DEBUG: appConfig.remainingFreeSpace = 5364164150
DEBUG: appConfig.quotaAvailable     = 
DEBUG: appConfig.quotaRestricted    =
DEBUG: Getting Account Default Root Details
DEBUG: Existing Microsoft OneDrive Access Token Expires: 2024-Jan-16 21:55:27.1920982
DEBUG: Request URL = https://graph.microsoft.com/v1.0/me/drive/root
DEBUG: OneDrive Account Default Root Details:       {"@odata.context":"https:\/\/graph.microsoft.com\/v1.0\/$metadata#users('%40hotmail.com')\/drive\/root\/$entity","createdBy":{"user":{"displayName":"","email":"@hotmail.com"}},"createdDateTime":"2023-08-08T12:15:25Z","eTag":"\"{65822DD9-9D16-4FF3-A91E-EA3F05CED3B7},14\"","fileSystemInfo":{"createdDateTime":"2023-08-08T12:15:25Z","lastModifiedDateTime":"2024-01-16T20:55:22Z"},"folder":{"childCount":1,"view":{"sortBy":"name","sortOrder":"ascending","viewType":"thumbnails"}},"id":"AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639","lastModifiedDateTime":"2024-01-16T20:55:22Z","name":"root","parentReference":{"driveId":"af2b5e52bea96776","driveType":"personal"},"root":{},"size":3,"webUrl":"https:\/\/onedrive.live.com?cid=af2b5e52bea96776&id=01NRMAN7F6Y2GOVW7725BZO354PWSELRRZ"}
DEBUG: appConfig.defaultRootId      = AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639
DEBUG: saveItem - creating DB item from this JSON: {"@odata.context":"https:\/\/graph.microsoft.com\/v1.0\/$metadata#users('%40hotmail.com')\/drive\/root\/$entity","createdBy":{"user":{"displayName":"","email":"@hotmail.com"}},"createdDateTime":"2023-08-08T12:15:25Z","eTag":"\"{65822DD9-9D16-4FF3-A91E-EA3F05CED3B7},14\"","fileSystemInfo":{"createdDateTime":"2023-08-08T12:15:25Z","lastModifiedDateTime":"2024-01-16T20:55:22Z"},"folder":{"childCount":1,"view":{"sortBy":"name","sortOrder":"ascending","viewType":"thumbnails"}},"id":"AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639","lastModifiedDateTime":"2024-01-16T20:55:22Z","name":"root","parentReference":{"driveId":"af2b5e52bea96776","driveType":"personal"},"root":{},"size":3,"webUrl":"https:\/\/onedrive.live.com?cid=af2b5e52bea96776&id=01NRMAN7F6Y2GOVW7725BZO354PWSELRRZ"}
DEBUG: Flagging object as a directory
DEBUG: Updating DB Item object with correct values as this is a 'root' object
DEBUG: ROOT JSON Item HAS parentReference .... setting item.driveId = jsonItem['parentReference']['driveId'].str
DEBUG: Adding to database: Item("af2b5e52bea96776", "AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639", "root", "", dir, "\"{65822DD9-9D16-4FF3-A91E-EA3F05CED3B7},14\"", "", 2024-Jan-16 20:55:22Z, "", "", "", "", "", "Y", "")
DEBUG: Application Version:  v2.5.0-alpha-5 GitHub version: v2.4.25-36-gcda8317
DEBUG: Account Type:         personal
DEBUG: Default Drive ID:     af2b5e52bea96776
DEBUG: Default Root ID:      AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639
DEBUG: Remaining Free Space: 5.00 GB (5364164150 bytes)
DEBUG: Sync Engine Initialised with new Onedrive API instance
DEBUG: No valid Microsoft OneDrive webhook subscription to delete
DEBUG: All application operations will be performed in the configured local 'sync_dir' directory: /home/alan/OneDrive
DEBUG: Perform a Full Scan True-Up:
DEBUG: driveIdToQuery was empty, setting to appConfig.defaultDriveId
DEBUG: driveIdToQuery: af2b5e52bea96776
DEBUG: itemIdToQuery was empty, setting to appConfig.defaultRootId
DEBUG: itemIdToQuery: AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639
DEBUG: DeltaLink Query (driveId): af2b5e52bea96776
DEBUG: DeltaLink Query (id):      AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639
DEBUG: Using database stored deltaLink
DEBUG: Fetching /delta response from the OneDrive API for Drive ID: af2b5e52bea96776
DEBUG: Read token from appConfig
DEBUG: Authorised State: true
DEBUG: ------------------------------------------------------------------
DEBUG: selectedDriveId:   af2b5e52bea96776
DEBUG: selectedItemId:    AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639
DEBUG: providedDeltaLink: https://graph.microsoft.com/v1.0/drives/af2b5e52bea96776/items/AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639/delta?token=NDslMjM0OyUyMzE7Mzs1ZTg4M2Q0YS1mYWVmLTQ4YWQtOTA1Ny0yNWJmNTUwNzQ4ZDI7NjM4NDEwMzUzMTkzNzcwMDAwOzMxNTQ3NzE2MDslMjM7JTIzOyUyMzA7JTIz
DEBUG: ------------------------------------------------------------------
DEBUG: Existing Microsoft OneDrive Access Token Expires: 2024-Jan-16 21:55:27.1920982
DEBUG: Request URL = https://graph.microsoft.com/v1.0/drives/af2b5e52bea96776/items/AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639/delta?token=NDslMjM0OyUyMzE7Mzs1ZTg4M2Q0YS1mYWVmLTQ4YWQtOTA1Ny0yNWJmNTUwNzQ4ZDI7NjM4NDEwMzUzMTkzNzcwMDAwOzMxNTQ3NzE2MDslMjM7JTIzOyUyMzA7JTIz
DEBUG: Processing API Response Bundle: 1 - Quantity of 'changes|items' in this bundle to process: 3
DEBUG: ------------------------------------------------------------------
DEBUG: Processing OneDrive Item 1 of 3 from API Response Bundle 1
DEBUG: Raw JSON OneDrive Item: {"@odata.type":"#microsoft.graph.driveItem","createdDateTime":"2023-08-08T12:15:25Z","fileSystemInfo":{"createdDateTime":"2023-08-08T12:15:25Z","lastModifiedDateTime":"2024-01-16T20:55:22Z"},"folder":{"childCount":1,"view":{"sortBy":"name","sortOrder":"ascending","viewType":"thumbnails"}},"id":"AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639","lastModifiedDateTime":"2024-01-16T20:55:22Z","name":"root","parentReference":{"driveId":"af2b5e52bea96776","driveType":"personal"},"root":{},"size":3,"webUrl":"https:\/\/onedrive.live.com?cid=af2b5e52bea96776&id=01NRMAN7F6Y2GOVW7725BZO354PWSELRRZ"}
DEBUG: This item is not a OneDrive deletion change
DEBUG: defaultRootId                                        = AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639
DEBUG: 'search id'                                          = AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639
DEBUG: id == defaultRootId                                  = true
DEBUG: isItemRoot(onedriveJSONItem)                         = true
DEBUG: onedriveJSONItem['name'].str == 'root'               = true
DEBUG: itemHasParentReferenceId                             = false
DEBUG: JSON item will flagged as a 'root' item
DEBUG: objectParentDriveId                                  = af2b5e52bea96776
DEBUG: handleItemAsRootObject                               = true
DEBUG: itemHasParentReferenceId                             = false
DEBUG: itemIsDeletedOnline                                  = false
DEBUG: Handling change immediately as 'root item', or has no parent reference id or is a deleted item
DEBUG: Handing JSON object as OneDrive 'root' object
DEBUG: ------------------------------------------------------------------
DEBUG: Processing OneDrive Item 2 of 3 from API Response Bundle 1
DEBUG: Raw JSON OneDrive Item: {"@odata.type":"#microsoft.graph.driveItem","cTag":"\"c:{6DE0D0CD-3689-43E4-BA5B-C3E591D937B6},0\"","createdBy":{"application":{"displayName":"00000003-0000-0000-c000-000000000000"},"user":{"displayName":"","email":"@hotmail.com"}},"createdDateTime":"2024-01-09T13:34:24Z","eTag":"\"{6DE0D0CD-3689-43E4-BA5B-C3E591D937B6},1\"","fileSystemInfo":{"createdDateTime":"2024-01-09T13:34:24Z","lastModifiedDateTime":"2024-01-09T13:34:24Z"},"folder":{"childCount":3,"view":{}},"id":"AF2B5E52BEA96776!s6de0d0cd368943e4ba5bc3e591d937b6","lastModifiedBy":{"application":{"displayName":"00000003-0000-0000-c000-000000000000"},"user":{"displayName":"","email":"@hotmail.com"}},"lastModifiedDateTime":"2024-01-09T13:34:24Z","name":"a","parentReference":{"driveId":"af2b5e52bea96776","driveType":"personal","id":"AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639","path":"\/drives\/af2b5e52bea96776\/root:","siteId":"ad57710d-fb91-4ff1-9f7e-565b7bd3e764"},"size":3,"webUrl":"https:\/\/onedrive.live.com?cid=af2b5e52bea96776&id=01NRMAN7GN2DQG3CJW4RB3UW6D4WI5SN5W"}
DEBUG: This item is not a OneDrive deletion change
DEBUG: defaultRootId                                        = AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639
DEBUG: 'search id'                                          = AF2B5E52BEA96776!s6de0d0cd368943e4ba5bc3e591d937b6
DEBUG: id == defaultRootId                                  = false
DEBUG: isItemRoot(onedriveJSONItem)                         = false
DEBUG: onedriveJSONItem['name'].str == 'root'               = false
DEBUG: itemHasParentReferenceId                             = true
DEBUG: Adding this Raw JSON OneDrive Item to jsonItemsToProcess array for further processing
DEBUG: ------------------------------------------------------------------
DEBUG: Processing OneDrive Item 3 of 3 from API Response Bundle 1
DEBUG: Raw JSON OneDrive Item: {"@odata.type":"#microsoft.graph.driveItem","cTag":"\"c:{8B00ACB4-D015-420B-8BB4-885D46844459},2\"","createdBy":{"application":{"displayName":"i:0i.t|ms.sp.ext|00000000-0000-0000-0000-0000481710a4@9188040d-6c67-4c5b-b112-36a304b66dad"},"user":{"displayName":"","email":"@hotmail.com"}},"createdDateTime":"2024-01-16T11:13:47Z","eTag":"\"{8B00ACB4-D015-420B-8BB4-885D46844459},10\"","file":{"hashes":{"quickXorHash":"79jFLwAAAAAAAAAAAwAAAAAAAAA="},"mimeType":"text\/plain"},"fileSystemInfo":{"createdDateTime":"2024-01-16T11:13:47Z","lastModifiedDateTime":"2024-01-16T11:13:49Z"},"id":"AF2B5E52BEA96776!s8b00acb4d015420b8bb4885d46844459","lastModifiedBy":{"application":{"displayName":"i:0i.t|ms.sp.ext|00000000-0000-0000-0000-0000481710a4@9188040d-6c67-4c5b-b112-36a304b66dad"},"user":{"displayName":"","email":"@hotmail.com"}},"lastModifiedDateTime":"2024-01-16T11:13:49Z","name":"mondate.txt","parentReference":{"driveId":"af2b5e52bea96776","driveType":"personal","id":"AF2B5E52BEA96776!s6de0d0cd368943e4ba5bc3e591d937b6","path":"\/drives\/af2b5e52bea96776\/root:\/a","siteId":"ad57710d-fb91-4ff1-9f7e-565b7bd3e764"},"size":3,"webUrl":"https:\/\/onedrive.live.com?cid=af2b5e52bea96776&id=01NRMAN7FUVQAIWFOQBNBIXNEILVDIIRCZ"}
DEBUG: This item is not a OneDrive deletion change
DEBUG: defaultRootId                                        = AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639
DEBUG: 'search id'                                          = AF2B5E52BEA96776!s8b00acb4d015420b8bb4885d46844459
DEBUG: id == defaultRootId                                  = false
DEBUG: isItemRoot(onedriveJSONItem)                         = false
DEBUG: onedriveJSONItem['name'].str == 'root'               = false
DEBUG: itemHasParentReferenceId                             = true
DEBUG: Attempting to calculate local filesystem path for af2b5e52bea96776 and AF2B5E52BEA96776!s6de0d0cd368943e4ba5bc3e591d937b6
DEBUG: New Item calculated full path is: a/mondate.txt
DEBUG: skip_file item to check (full calculated path): /a/mondate.txt
DEBUG: skip_file evaluation for: /a/mondate.txt
DEBUG: Result: false
DEBUG: Adding this Raw JSON OneDrive Item to jsonItemsToProcess array for further processing
DEBUG: Setting next currentDeltaLink to (@odata.deltaLink): https://graph.microsoft.com/v1.0/drives/af2b5e52bea96776/items/AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639/delta?token=NDslMjM0OyUyMzE7Mzs1ZTg4M2Q0YS1mYWVmLTQ4YWQtOTA1Ny0yNWJmNTUwNzQ4ZDI7NjM4NDEwMzUzMjgzMjMwMDAwOzMxNTQ3NzE4NjslMjM7JTIzOyUyMzA7JTIz
DEBUG: ------------------------------------------------------------------
DEBUG: No valid Microsoft OneDrive webhook subscription to delete
DEBUG: Finished processing /delta JSON response from the OneDrive API
DEBUG: Number of JSON Objects received from OneDrive API:                 3
DEBUG: Number of JSON Objects already processed (root and deleted items): 1
DEBUG: Number of JSON items to process is: 2
DEBUG: Processing 2 applicable changes and items received from Microsoft OneDrive DEBUG:
DEBUG: Processing OneDrive JSON item batch [1/1] to ensure consistent local state
DEBUG: ------------------------------------------------------------------
DEBUG: Processing OneDrive JSON item 1 of 2 as part of JSON Item Batch 1 of 1
DEBUG: Raw JSON OneDrive Item: {"@odata.type":"#microsoft.graph.driveItem","cTag":"\"c:{6DE0D0CD-3689-43E4-BA5B-C3E591D937B6},0\"","createdBy":{"application":{"displayName":"00000003-0000-0000-c000-000000000000"},"user":{"displayName":"","email":"@hotmail.com"}},"createdDateTime":"2024-01-09T13:34:24Z","eTag":"\"{6DE0D0CD-3689-43E4-BA5B-C3E591D937B6},1\"","fileSystemInfo":{"createdDateTime":"2024-01-09T13:34:24Z","lastModifiedDateTime":"2024-01-09T13:34:24Z"},"folder":{"childCount":3,"view":{}},"id":"AF2B5E52BEA96776!s6de0d0cd368943e4ba5bc3e591d937b6","lastModifiedBy":{"application":{"displayName":"00000003-0000-0000-c000-000000000000"},"user":{"displayName":"","email":"@hotmail.com"}},"lastModifiedDateTime":"2024-01-09T13:34:24Z","name":"a","parentReference":{"driveId":"af2b5e52bea96776","driveType":"personal","id":"AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639","path":"\/drives\/af2b5e52bea96776\/root:","siteId":"ad57710d-fb91-4ff1-9f7e-565b7bd3e764"},"size":3,"webUrl":"https:\/\/onedrive.live.com?cid=af2b5e52bea96776&id=01NRMAN7GN2DQG3CJW4RB3UW6D4WI5SN5W"}
DEBUG: Attempting to calculate local filesystem path for af2b5e52bea96776 and AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639
DEBUG: New Item calculated full path is: ./a
DEBUG: The item we are syncing is a folder
DEBUG: Flagging object as a directory
DEBUG: OneDrive change is an update to an existing local item
DEBUG: Attempting to calculate local filesystem path for af2b5e52bea96776 and AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639
DEBUG: ------------------------------------------------------------------
DEBUG: Processing OneDrive JSON item 2 of 2 as part of JSON Item Batch 1 of 1
DEBUG: Raw JSON OneDrive Item: {"@odata.type":"#microsoft.graph.driveItem","cTag":"\"c:{8B00ACB4-D015-420B-8BB4-885D46844459},2\"","createdBy":{"application":{"displayName":"i:0i.t|ms.sp.ext|00000000-0000-0000-0000-0000481710a4@9188040d-6c67-4c5b-b112-36a304b66dad"},"user":{"displayName":"","email":"@hotmail.com"}},"createdDateTime":"2024-01-16T11:13:47Z","eTag":"\"{8B00ACB4-D015-420B-8BB4-885D46844459},10\"","file":{"hashes":{"quickXorHash":"79jFLwAAAAAAAAAAAwAAAAAAAAA="},"mimeType":"text\/plain"},"fileSystemInfo":{"createdDateTime":"2024-01-16T11:13:47Z","lastModifiedDateTime":"2024-01-16T11:13:49Z"},"id":"AF2B5E52BEA96776!s8b00acb4d015420b8bb4885d46844459","lastModifiedBy":{"application":{"displayName":"i:0i.t|ms.sp.ext|00000000-0000-0000-0000-0000481710a4@9188040d-6c67-4c5b-b112-36a304b66dad"},"user":{"displayName":"","email":"@hotmail.com"}},"lastModifiedDateTime":"2024-01-16T11:13:49Z","name":"mondate.txt","parentReference":{"driveId":"af2b5e52bea96776","driveType":"personal","id":"AF2B5E52BEA96776!s6de0d0cd368943e4ba5bc3e591d937b6","path":"\/drives\/af2b5e52bea96776\/root:\/a","siteId":"ad57710d-fb91-4ff1-9f7e-565b7bd3e764"},"size":3,"webUrl":"https:\/\/onedrive.live.com?cid=af2b5e52bea96776&id=01NRMAN7FUVQAIWFOQBNBIXNEILVDIIRCZ"}
DEBUG: Attempting to calculate local filesystem path for af2b5e52bea96776 and AF2B5E52BEA96776!s6de0d0cd368943e4ba5bc3e591d937b6
DEBUG: New Item calculated full path is: a/mondate.txt
DEBUG: The item we are syncing is a file
DEBUG: skip_file item to check: /a/mondate.txt
DEBUG: skip_file evaluation for: /a/mondate.txt
DEBUG: Result: false
DEBUG: Flagging object as a file
DEBUG: OneDrive change is an update to an existing local item
DEBUG: Attempting to calculate local filesystem path for af2b5e52bea96776 and AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639
DEBUG: Moving ./mondate.txt to a/mondate.txt
DEBUG: The local item is out-of-sync with OneDrive, renaming to preserve existing file and prevent local data loss: a/mondate.txt -> a/mondate-55db26f69037.txt
DEBUG: Renaming of local file failed for a/mondate.txt: a/mondate-55db26f69037.txt: No such file or directory
DEBUG:
DEBUG: ERROR: The local file system returned an error with the following message:
DEBUG:   Error Message:    a/mondate.txt: No such file or directory
DEBUG:   Calling Function: applyPotentiallyChangedItem()

DEBUG: ------------------------------------------------------------------
DEBUG: Number of JSON items to process is: 0
DEBUG: Number of JSON items processed was: 2
DEBUG: Updating completed deltaLink in DB to: https://graph.microsoft.com/v1.0/drives/af2b5e52bea96776/items/AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639/delta?token=NDslMjM0OyUyMzE7Mzs1ZTg4M2Q0YS1mYWVmLTQ4YWQtOTA1Ny0yNWJmNTUwNzQ4ZDI7NjM4NDEwMzUzMjgzMjMwMDAwOzMxNTQ3NzE4NjslMjM7JTIzOyUyMzA7JTIz
DEBUG: Performing a database consistency and integrity check on locally stored data ...
DEBUG: Processing DB entries for this Drive ID: af2b5e52bea96776
DEBUG: Selecting DB items via itemDB.selectByDriveId(driveId)
DEBUG: Database items to process for this driveId: 1
DEBUG: Attempting to calculate local filesystem path for af2b5e52bea96776 and AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639
DEBUG: Processing ~/OneDrive
DEBUG: The directory has not changed
DEBUG: Attempting to calculate local filesystem path for af2b5e52bea96776 and AF2B5E52BEA96776!s6de0d0cd368943e4ba5bc3e591d937b6
DEBUG: Processing a
DEBUG: The directory has not changed
DEBUG: Attempting to calculate local filesystem path for af2b5e52bea96776 and AF2B5E52BEA96776!s11696409b045472cb85d9fc334f5fd69
DEBUG: Processing a/b
DEBUG: The directory has not changed
DEBUG: Attempting to calculate local filesystem path for af2b5e52bea96776 and AF2B5E52BEA96776!s361bef779e714e77a0b3500d3c529009
DEBUG: Processing a/b/c
DEBUG: The directory has not changed
DEBUG: Attempting to calculate local filesystem path for af2b5e52bea96776 and AF2B5E52BEA96776!s9e665501a2bb4f6daad66f49147b50ef
DEBUG: Processing a/b/c/d
DEBUG: The directory has not changed
DEBUG: Attempting to calculate local filesystem path for af2b5e52bea96776 and AF2B5E52BEA96776!s13a5ebf120554c4087d7a771d70520ab
DEBUG: Processing a/e
DEBUG: The directory has not changed
DEBUG: Attempting to calculate local filesystem path for af2b5e52bea96776 and AF2B5E52BEA96776!sf7a647e79aa44396be9d0c915c8a9515
DEBUG: Processing a/e/f
DEBUG: The directory has not changed
DEBUG: Attempting to calculate local filesystem path for af2b5e52bea96776 and AF2B5E52BEA96776!sc13d441293754a49903142e3148c3f7c
DEBUG: Processing a/e/f/g
DEBUG: The directory has not changed
DEBUG: Attempting to calculate local filesystem path for af2b5e52bea96776 and AF2B5E52BEA96776!s8b00acb4d015420b8bb4885d46844459
DEBUG: Processing mondate.txt
DEBUG: The file has not changed
DEBUG: Scanning the local file system '~/OneDrive' for new data to upload ...
DEBUG: Starting Filesystem Walk:     2024-Jan-16 20:55:29.1677986
DEBUG: ------------------------------------------------------------------
DEBUG: ------------------------------------------------------------------
DEBUG: ------------------------------------------------------------------
DEBUG: ------------------------------------------------------------------
DEBUG: ------------------------------------------------------------------
DEBUG: ------------------------------------------------------------------
DEBUG: ------------------------------------------------------------------
DEBUG: ------------------------------------------------------------------
DEBUG: ------------------------------------------------------------------
DEBUG: Finished Filesystem Walk:     2024-Jan-16 20:55:29.1696665
DEBUG: Elapsed Time Filesystem Walk: 1 ms, 867 μs, and 9 hnsecs
DEBUG: Attempting to perform a database vacuum to merge any temporary data
DEBUG: Database vacuum is complete
DEBUG: Performing a last examination of the most recent online data within Microsoft OneDrive to complete the reconciliation process
DEBUG: Perform a Full Scan True-Up:
DEBUG: driveIdToQuery was empty, setting to appConfig.defaultDriveId
DEBUG: driveIdToQuery: af2b5e52bea96776
DEBUG: itemIdToQuery was empty, setting to appConfig.defaultRootId
DEBUG: itemIdToQuery: AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639
DEBUG: DeltaLink Query (driveId): af2b5e52bea96776
DEBUG: DeltaLink Query (id):      AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639
DEBUG: Using database stored deltaLink
DEBUG: Fetching /delta response from the OneDrive API for Drive ID: af2b5e52bea96776
DEBUG: Read token from appConfig
DEBUG: Authorised State: true
DEBUG: ------------------------------------------------------------------
DEBUG: selectedDriveId:   af2b5e52bea96776
DEBUG: selectedItemId:    AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639
DEBUG: providedDeltaLink: https://graph.microsoft.com/v1.0/drives/af2b5e52bea96776/items/AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639/delta?token=NDslMjM0OyUyMzE7Mzs1ZTg4M2Q0YS1mYWVmLTQ4YWQtOTA1Ny0yNWJmNTUwNzQ4ZDI7NjM4NDEwMzUzMjgzMjMwMDAwOzMxNTQ3NzE4NjslMjM7JTIzOyUyMzA7JTIz
DEBUG: ------------------------------------------------------------------
DEBUG: Existing Microsoft OneDrive Access Token Expires: 2024-Jan-16 21:55:27.1920982
DEBUG: Request URL = https://graph.microsoft.com/v1.0/drives/af2b5e52bea96776/items/AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639/delta?token=NDslMjM0OyUyMzE7Mzs1ZTg4M2Q0YS1mYWVmLTQ4YWQtOTA1Ny0yNWJmNTUwNzQ4ZDI7NjM4NDEwMzUzMjgzMjMwMDAwOzMxNTQ3NzE4NjslMjM7JTIzOyUyMzA7JTIz
DEBUG: Processing API Response Bundle: 1 - Quantity of 'changes|items' in this bundle to process: 0
DEBUG: Setting next currentDeltaLink to (@odata.deltaLink): https://graph.microsoft.com/v1.0/drives/af2b5e52bea96776/items/AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639/delta?token=NDslMjM0OyUyMzE7Mzs1ZTg4M2Q0YS1mYWVmLTQ4YWQtOTA1Ny0yNWJmNTUwNzQ4ZDI7NjM4NDEwMzUzMjkyOTcwMDAwOzMxNTQ3NzE4NzslMjM7JTIzOyUyMzA7JTIz
DEBUG: ------------------------------------------------------------------
DEBUG: No valid Microsoft OneDrive webhook subscription to delete
DEBUG: Finished processing /delta JSON response from the OneDrive API
DEBUG: Number of JSON Objects received from OneDrive API:                 0
DEBUG: Number of JSON Objects already processed (root and deleted items): 0
DEBUG: Number of JSON items to process is: 0
DEBUG: No additional changes or items that can be applied were discovered while processing the data received from Microsoft OneDrive
DEBUG: Updating completed deltaLink in DB to: https://graph.microsoft.com/v1.0/drives/af2b5e52bea96776/items/AF2B5E52BEA96776!sea8cc6beffdb43d7976fbc7da445c639/delta?token=NDslMjM0OyUyMzE7Mzs1ZTg4M2Q0YS1mYWVmLTQ4YWQtOTA1Ny0yNWJmNTUwNzQ4ZDI7NjM4NDEwMzUzMjkyOTcwMDAwOzMxNTQ3NzE4NzslMjM7JTIzOyUyMzA7JTIz
DEBUG:
DEBUG: Sync with Microsoft OneDrive is complete
DEBUG: Exit scope was called
DEBUG: Running performStandardExitProcess due to: exitScope
DEBUG: Shutdown OneDrive API instance
DEBUG: No valid Microsoft OneDrive webhook subscription to delete
DEBUG: Shutdown Sync Engine instance
DEBUG: Shutdown Client Side Filtering instance
DEBUG: Shutdown Application Configuration instance
DEBUG: Shutdown Database instance
DEBUG: Attempting to perform a database vacuum to merge any temporary data
DEBUG: Database vacuum is complete
DEBUG: Application exit
abraunegg commented 10 months ago

@JC-comp

On local machine, performe a full sync with onedrive --sync Open Onedrive web interface, drag and move a file (mondate.txt in this case) to some other folder (a in this case)

Does the target folder 'a' exist before you perform the full sync, or is this created before you move the file to 'a' ?

JC-comp commented 10 months ago

Does the target folder 'a' exist before you perform the full sync, or is this created before you move the file to 'a'

Created before performing the first full sync, the only operation after the first sync is moving the file to that folder

abraunegg commented 10 months ago

@JC-comp So the issue stems from an over-aggressive 'find|replace' operation done sometime between 'alpha-3' and 'alpha-4' which is now corrected.

As per below, the file is moved to the target folder without issue.

Reading configuration file: /home/alex/.config/onedrive/config
Configuration file successfully loaded
Using 'user' configuration path for application state data: /home/alex/.config/onedrive

ERROR: Unable to create /var/log/onedrive
ERROR: Please manually create '/var/log/onedrive' and set appropriate permissions to allow write access for your user to this location.
ERROR: The requested client activity log will instead be located in your users home directory

Using the following path to store the runtime application log: /home/alex
Using IPv4 and IPv6 (if configured) for all network operations
Checking Application Version ...
Attempting to initialise the OneDrive API ...
Configuring Global Azure AD Endpoints
The OneDrive API was initialised successfully
Opening the item database ...
Application Version:  v2.5.0-alpha-5 GitHub version: v2.4.25-36-gcda8317
Account Type:         personal
Default Drive ID:     66d53be8a5056eca
Default Root ID:      66D53BE8A5056ECA!101
Remaining Free Space: 4.92 GB (5280888779 bytes)
Sync Engine Initialised with new Onedrive API instance
All application operations will be performed in the configured local 'sync_dir' directory: /home/alex/OneDrive
Fetching /delta response from the OneDrive API for Drive ID: 66d53be8a5056eca
Processing API Response Bundle: 1 - Quantity of 'changes|items' in this bundle to process: 4
Finished processing /delta JSON response from the OneDrive API
Processing 3 applicable changes and items received from Microsoft OneDrive 
Processing OneDrive JSON item batch [1/1] to ensure consistent local state
Moving Source_Folder/dummy_doc.docx to Target_Folder/dummy_doc.docx
Performing a database consistency and integrity check on locally stored data ... 
Processing DB entries for this Drive ID: 66d53be8a5056eca
Processing ~/OneDrive
The directory has not changed
Processing Target_Folder
The directory has not changed
Processing Target_Folder/dummy_doc.docx
The file has not changed
Processing Source_Folder
The directory has not changed
Scanning the local file system '~/OneDrive' for new data to upload ...
Skipping item - excluded by sync_list config: ./random_25k_files
Skipping item - .file or .folder: ./.folder
Performing a last examination of the most recent online data within Microsoft OneDrive to complete the reconciliation process
Fetching /delta response from the OneDrive API for Drive ID: 66d53be8a5056eca
Processing API Response Bundle: 1 - Quantity of 'changes|items' in this bundle to process: 0
Finished processing /delta JSON response from the OneDrive API
No additional changes or items that can be applied were discovered while processing the data received from Microsoft OneDrive

Sync with Microsoft OneDrive is complete

Please rebuild your 'alpha-5' client to: 2.5.0-alpha-5 GitHub version: v2.4.25-39-ge8406b7 and retest.

JC-comp commented 10 months ago

Thank you for the update. I've tested the changes and the issues have been resolved.

abraunegg commented 9 months ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.