Bug: skip_dir path may be deleted from remote drive

dalemartin commented 5 days ago

Describe the bug

A remote SharePoint folder may be deleted even if a skip_dir pattern matches it.

Operating System Details

Using a custom Linux kernel based on 6.6.51.

$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 12 (bookworm)
Release:    12
Codename:   bookworm

Client Installation Method

From 3rd Party Source (PPA, OpenSuSE Build Service etc)

OneDrive Account Type

Business | Office365

What is your OneDrive Application Version

onedrive v2.5.2-1+np1+1.1

What is your OneDrive Application Configuration

$ onedrive --confdir="~/.config/onedrive/accounts/SharePoint_SDI_Documents" --display-config
Reading configuration file: /home/dmartin/.config/onedrive/accounts/SharePoint_SDI_Documents/config
Configuration file successfully loaded
WARNING: D-Bus message bus daemon is not available; GUI notifications are disabled
Application version                          = onedrive v2.5.2-1+np1+1.1
Compiled with                                = LDC 2100
User Application Config path                 = /home/dmartin/.config/onedrive/accounts/SharePoint_SDI_Documents
System Application Config path               = /etc/onedrive
Applicable Application 'config' location     = /home/dmartin/.config/onedrive/accounts/SharePoint_SDI_Documents/config
Configuration file found in config location  = true - using 'config' file values to override application defaults
Applicable 'sync_list' location              = /home/dmartin/.config/onedrive/accounts/SharePoint_SDI_Documents/sync_list
Applicable 'items.sqlite3' location          = /home/dmartin/.config/onedrive/accounts/SharePoint_SDI_Documents/items.sqlite3
Config option 'drive_id'                     = ********
Config option 'sync_dir'                     = ~/SharePoint/SDI_Documents
Config option 'enable_logging'               = false
Config option 'log_dir'                      = /var/log/onedrive
Config option 'disable_notifications'        = false
Config option 'skip_dir'                     = /foo/bar*
Config option 'skip_dir_strict_match'        = false
Config option 'skip_file'                    = ~*|.~*|*.tmp|*.swp|*.partial
Config option 'skip_dotfiles'                = false
Config option 'skip_symlinks'                = false
Config option 'monitor_interval'             = 300
Config option 'monitor_log_frequency'        = 12
Config option 'monitor_fullscan_frequency'   = 12
Config option 'read_only_auth_scope'         = false
Config option 'dry_run'                      = false
Config option 'upload_only'                  = false
Config option 'download_only'                = false
Config option 'local_first'                  = false
Config option 'check_nosync'                 = false
Config option 'check_nomount'                = false
Config option 'resync'                       = false
Config option 'resync_auth'                  = false
Config option 'cleanup_local_files'          = false
Config option 'classify_as_big_delete'       = 1000
Config option 'disable_upload_validation'    = false
Config option 'disable_download_validation'  = false
Config option 'bypass_data_preservation'     = false
Config option 'no_remote_delete'             = false
Config option 'remove_source_files'          = false
Config option 'sync_dir_permissions'         = 700
Config option 'sync_file_permissions'        = 600
Config option 'space_reservation'            = 52428800
Config option 'application_id'               = d50ca740-c83f-4d1b-b616-12c519384f0c
Config option 'azure_ad_endpoint'            = 
Config option 'azure_tenant_id'              = 
Config option 'user_agent'                   = ISV|abraunegg|OneDrive Client for Linux/v2.5.2-1+np1+1.1
Config option 'force_http_11'                = false
Config option 'debug_https'                  = false
Config option 'rate_limit'                   = 0
Config option 'operation_timeout'            = 3600
Config option 'dns_timeout'                  = 60
Config option 'connect_timeout'              = 10
Config option 'data_timeout'                 = 60
Config option 'ip_protocol_version'          = 0
Config option 'threads'                      = 8
Environment var 'XDG_RUNTIME_DIR'            = true
Environment var 'DBUS_SESSION_BUS_ADDRESS'   = true
Config option 'notify_file_actions'          = false

Selective sync 'sync_list' configured        = true
sync_list config option 'sync_root_files'    = false
sync_list contents:
/foo

Config option 'sync_business_shared_items'   = false

Config option 'webhook_enabled'              = false

What is your 'curl' version

$ curl --version
curl 7.88.1 (x86_64-pc-linux-gnu) libcurl/7.88.1 OpenSSL/3.0.14 zlib/1.2.13 brotli/1.0.9 zstd/1.5.4 libidn2/2.3.3 libpsl/0.21.2 (+libidn2/2.3.3) libssh2/1.10.0 nghttp2/1.52.0 librtmp/2.3 OpenLDAP/2.5.13
Release-Date: 2023-02-20, security patched: 7.88.1-10+deb12u7
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

Where is your 'sync_dir' located

Local

What are all your system 'mount points'

$ mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=16212332k,nr_inodes=4053083,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,noexec,relatime,size=3252812k,mode=755)
rootpool/ROOT/falcon on / type zfs (rw,relatime,xattr,posixacl,casesensitive)
rootpool/ROOT/falcon/etc on /etc type zfs (rw,relatime,xattr,posixacl,casesensitive)
rootpool/ROOT/falcon/home on /home type zfs (rw,relatime,xattr,posixacl,casesensitive)
rootpool/ROOT/falcon/tmp on /tmp type zfs (rw,relatime,xattr,posixacl,casesensitive)
rootpool/ROOT/falcon/usr on /usr type zfs (rw,relatime,xattr,posixacl,casesensitive)
rootpool/ROOT/falcon/var on /var type zfs (rw,relatime,xattr,posixacl,casesensitive)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
efivarfs on /sys/firmware/efi/efivars type efivarfs (rw,nosuid,nodev,noexec,relatime)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2963)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime)
tracefs on /sys/kernel/tracing type tracefs (rw,nosuid,nodev,noexec,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
ramfs on /run/credentials/systemd-sysusers.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
ramfs on /run/credentials/systemd-tmpfiles-setup-dev.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
ramfs on /run/credentials/systemd-sysctl.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
/dev/nvme0n1p3 on /boot type ext4 (rw,relatime)
/dev/nvme0n1p1 on /boot/efi type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro)
rootpool/qemu on /qemu type zfs (rw,relatime,xattr,posixacl,casesensitive)
rootpool/redis_data on /redis_data type zfs (rw,relatime,xattr,posixacl,casesensitive)
rootpool/registry_data on /registry_data type zfs (rw,relatime,xattr,posixacl,casesensitive)
rootpool/docker on /var/lib/docker type zfs (rw,relatime,xattr,posixacl,casesensitive)
rootpool/qemu/images on /qemu/images type zfs (rw,relatime,xattr,posixacl,casesensitive)
rootpool/qemu/isos on /qemu/isos type zfs (rw,relatime,xattr,posixacl,casesensitive)
rootpool/qemu/images/sdi_builder on /qemu/images/sdi_builder type zfs (rw,relatime,xattr,posixacl,casesensitive)
ramfs on /run/credentials/systemd-tmpfiles-setup.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,nosuid,nodev,noexec,relatime)
tmpfs on /run/user/115 type tmpfs (rw,nosuid,nodev,relatime,size=3252808k,nr_inodes=813202,mode=700,uid=115,gid=125)
gvfsd-fuse on /run/user/115/gvfs type fuse.gvfsd-fuse (rw,nosuid,nodev,relatime,user_id=115,group_id=125)
tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,size=3252808k,nr_inodes=813202,mode=700,uid=1000,gid=1000)
gvfsd-fuse on /run/user/1000/gvfs type fuse.gvfsd-fuse (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
portal on /run/user/1000/doc type fuse.portal (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
Obsidian-1.6.7.AppImage on /tmp/.mount_obsidiWbRHhn type fuse.Obsidian-1.6.7.AppImage (ro,nosuid,nodev,relatime,user_id=1000,group_id=1000)
OneDriveGUI-1.1.0-x86_64.AppImage on /tmp/.mount_onedri31CDxu type fuse.OneDriveGUI-1.1.0-x86_64.AppImage (ro,nosuid,nodev,relatime,user_id=1000,group_id=1000)
superProductivity-9.0.7.AppImage on /tmp/.mount_superp5JUxht type fuse.superProductivity-9.0.7.AppImage (ro,nosuid,nodev,relatime,user_id=1000,group_id=1000)

What are all your local file system partition types

$ lsblk -f
NAME        FSTYPE     FSVER LABEL    UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
nvme0n1                                                                                   
├─nvme0n1p1 vfat       FAT32 EFI      F8EB-A7F1                             497.3M     1% /boot/efi
├─nvme0n1p2 zfs_member 5000  rootpool 13321229604715348716                                
├─nvme0n1p3 ext4       1.0            cbf5bf9c-13ac-4a87-bfcf-749fbd6aa5fc    3.5G     3% /boot
└─nvme0n1p4 swap       1              ab4d8f73-b48f-4975-a7f6-fa1754c0bb70                [SWAP]

How do you use 'onedrive'

Local onedrive folder not shared with any other system.
Syncing to my company's SharePoint site. We use Office 365.

Steps to reproduce the behaviour

Consider the following setup:

A SharePoint site named "SDI" has the following tree:

Documents/
└── foo
  ├── barbar
  └── baz

Local sync_dir: /home/dmartin/SharePoint/SDI_Documents. It is empty because the first sync has not yet been performed.
~/.config/onedrive/accounts/SharePoint_SDI_Documents/sync_list contains one entry: /foo
~/.config/onedrive/accounts/SharePoint_SDI_Documents/config contains: skip_dir = "/foo/bar*"
All other config is default, except drive_id.

Upon the first sync, the remote directory /foo/barbar is synced locally.
Delete /foo/barbar locally, since the intent was to skip this one.
On the next sync, the remote /foo/barbar is deleted, despite being named in skip_dir.

Complete Verbose Log Output

# The log contains sensitive information so I would rather not share it. If it is absolutely needed, I can take the time to sanitize it and upload, but I'm hoping my description is sufficient for you to reproduce it. Or if you have any guidance as to which parts of the log you need I can work with you.

Screenshots

Original remote tree:

Other Log Information or Details

No response

Additional context

Original discussion post: https://github.com/abraunegg/onedrive/discussions/2909

Originally I thought this behaviour manifests itself on the very first sync. Now I am not able to reproduce the delete behaviour unless I first temporarily remove the skip_dir entry, then sync, then delete the directory locally, then restore the skip_dir entry, and finally sync. I was tinkering with settings for a while and was quite confused before I realized what was happening. It seems that something was cached even during --dry-run syncs which later affected the actual syncs, although I have not proven this, it is just my recollection of the first time I noticed the behaviour.

I hope this is helpful, I have really appreciated this app, thank you for all your hard work.

abraunegg commented 5 days ago

@dalemartin Thanks for the details

Now I am not able to reproduce the delete behaviour unless I first temporarily remove the skip_dir entry, then sync, then delete the directory locally, then restore the skip_dir entry, and finally sync.

When ever 'skip_dir' is modified | hashed out | temp removed .. the client should be forcing a full resync.

Do you have any log or evidence showing the actual deletion from the client you were running?

dalemartin commented 5 days ago

Ah, thanks for reminding me of the resync. I managed to trigger the behaviour like this:

Trigger authentication using onedrive --confdir="~/.config/onedrive/accounts/SharePoint_SDI_Documents".
Perform sync using onedrive -s -v --confdir="~/.config/onedrive/accounts/SharePoint_SDI_Documents" --resync.
- Currently no skip_dir entries in the config.
Add skip_dir = "/foo/bar*" to config file and save.
Delete local /foo/barbar.
- I used Nautilus' feature 'Right Click > Delete Permanently' which I assume to be equivalent to rm -r <dir>.
Perform another sync: onedrive -s -v --confdir="~/.config/onedrive/accounts/SharePoint_SDI_Documents" --resync
- It prompts for --resync as you said.
Remote /foo/barbar is deleted.

Log:

$ onedrive -s -v --confdir="~/.config/onedrive/accounts/SharePoint_SDI_Documents"
Reading configuration file: /home/dmartin/.config/onedrive/accounts/SharePoint_SDI_Documents/config
Configuration file successfully loaded
Using 'user' configuration path for application config and state data: /home/dmartin/.config/onedrive/accounts/SharePoint_SDI_Documents
WARNING: D-Bus message bus daemon is not available; GUI notifications are disabled
Using the following path to store the runtime application log: /home/dmartin/.local/sharepoint/sdi
Application configuration file has been updated, checking if --resync needed

An application configuration change has been detected where a --resync is required

$ onedrive -s -v --confdir="~/.config/onedrive/accounts/SharePoint_SDI_Documents" --resync
Reading configuration file: /home/dmartin/.config/onedrive/accounts/SharePoint_SDI_Documents/config
Configuration file successfully loaded
Using 'user' configuration path for application config and state data: /home/dmartin/.config/onedrive/accounts/SharePoint_SDI_Documents
WARNING: D-Bus message bus daemon is not available; GUI notifications are disabled
Using the following path to store the runtime application log: /home/dmartin/.local/sharepoint/sdi

The usage of --resync will delete your local 'onedrive' client state, thus no record of your current 'sync status' will exist.
This has the potential to overwrite local versions of files with perhaps older versions of documents downloaded from OneDrive, resulting in local data loss.
If in doubt, backup your local data before using --resync

Are you sure you wish to proceed with --resync? [Y/N] y

Deleting the saved application sync status ...
Using IPv4 and IPv6 (if configured) for all network operations
Attempting to contact Microsoft OneDrive Login Service
Successfully reached Microsoft OneDrive Login Service
Checking Application Version ...
Attempting to initialise the OneDrive API ...
Configuring Global Azure AD Endpoints
The OneDrive API was initialised successfully
Opening the item database ...
WARNING: OneDrive quota information is being restricted. Please fix by speaking to your OneDrive / Office 365 Administrator.
Application Version:  onedrive v2.5.2-1+np1+1.1
Account Type:         documentLibrary
Default Drive ID:     ***********
Default Root ID:      ***********
Remaining Free Space: Not Available
Sync Engine Initialised with new Onedrive API instance
All application operations will be performed in the configured local 'sync_dir' directory: /home/dmartin/SharePoint/SDI_Documents
Fetching /delta response from the OneDrive API for Drive ID: **********
Processing API Response Bundle: 1 - Quantity of 'changes|items' in this bundle to process: 62
Skipping path - excluded by sync_list config: ./General
Skipping path - excluded by sync_list config: ./ITCoordination
Skipping path - excluded by sync_list config: ITCoordination/EmailMessages_2_2021
Skipping path - excluded by sync_list config: ITCoordination/onedrive_sandbox
Skipping path - excluded by sync_list config: ITCoordination/onedrive_sandbox/mock_hw_files
Skipping path - excluded by sync_list config: ITCoordination/onedrive_sandbox/sharepoint_conf
Skipping path - excluded by sync_list config: ITCoordination/graph_api_sandbox
Skipping path - excluded by sync_list config: ./Network Administration
Including path - included by sync_list config: ./foo
Including path - included by sync_list config: foo/barbar
Including file - included by sync_list config: foo/barbar/bar.txt
Parental Path structure needs to be created to support included file: foo/barbar
Including file - included by sync_list config: foo/barbar/fuzz.txt
Including path - included by sync_list config: foo/baz
Including file - included by sync_list config: foo/baz/baz.txt
Parental Path structure needs to be created to support included file: foo/baz
Finished processing /delta JSON response from the OneDrive API
Processing 6 applicable changes and items received from Microsoft OneDrive
Processing OneDrive JSON item batch [1/1] to ensure consistent local state
Skipping file - file path is excluded by skip_dir config: foo/barbar/bar.txt
Skipping file - file path is excluded by skip_dir config: foo/barbar/fuzz.txt
Performing a database consistency and integrity check on locally stored data
Processing DB entries for this Drive ID: *********
WARNING: OneDrive quota information is being restricted. Please fix by speaking to your OneDrive / Office 365 Administrator.
Processing: /home/dmartin/SharePoint/SDI_Documents
The directory has not changed
Processing: foo
The directory has not changed
Processing: foo/barbar
The directory has been deleted locally
Processing: foo/baz
The directory has not changed
Processing: foo/baz/baz.txt
The file has not changed
Deleted local items to delete on Microsoft OneDrive: 1
Deleting item from Microsoft OneDrive: foo/barbar
Scanning the local file system '/home/dmartin/SharePoint/SDI_Documents' for new data to upload
Performing a last examination of the most recent online data within Microsoft OneDrive to complete the reconciliation process
Fetching /delta response from the OneDrive API for Drive ID: *********
Processing API Response Bundle: 1 - Quantity of 'changes|items' in this bundle to process: 4
Finished processing /delta JSON response from the OneDrive API
No changes or items that can be applied were discovered while processing the data received from Microsoft OneDrive

Sync with Microsoft OneDrive is complete
Attempting to perform a database vacuum to optimise database
Database vacuum is complete

And the final config:

$ onedrive --confdir ~/.config/onedrive/accounts/SharePoint_SDI_Documents/ --display-config
Reading configuration file: /home/dmartin/.config/onedrive/accounts/SharePoint_SDI_Documents/config
Configuration file successfully loaded
WARNING: D-Bus message bus daemon is not available; GUI notifications are disabled
Application version                          = onedrive v2.5.2-1+np1+1.1
Compiled with                                = LDC 2100
User Application Config path                 = /home/dmartin/.config/onedrive/accounts/SharePoint_SDI_Documents/
System Application Config path               = /etc/onedrive
Applicable Application 'config' location     = /home/dmartin/.config/onedrive/accounts/SharePoint_SDI_Documents/config
Configuration file found in config location  = true - using 'config' file values to override application defaults
Applicable 'sync_list' location              = /home/dmartin/.config/onedrive/accounts/SharePoint_SDI_Documents/sync_list
Applicable 'items.sqlite3' location          = /home/dmartin/.config/onedrive/accounts/SharePoint_SDI_Documents/items.sqlite3
Config option 'drive_id'                     = ********
Config option 'sync_dir'                     = /home/dmartin/SharePoint/SDI_Documents
Config option 'enable_logging'               = true
Config option 'log_dir'                      = /home/dmartin/.local/sharepoint/sdi
Config option 'disable_notifications'        = false
Config option 'skip_dir'                     = /foo/bar*
Config option 'skip_dir_strict_match'        = false
Config option 'skip_file'                    = ~*|.~*|*.tmp|*.swp|*.partial
Config option 'skip_dotfiles'                = true
Config option 'skip_symlinks'                = false
Config option 'monitor_interval'             = 300
Config option 'monitor_log_frequency'        = 12
Config option 'monitor_fullscan_frequency'   = 12
Config option 'read_only_auth_scope'         = false
Config option 'dry_run'                      = false
Config option 'upload_only'                  = false
Config option 'download_only'                = false
Config option 'local_first'                  = false
Config option 'check_nosync'                 = false
Config option 'check_nomount'                = false
Config option 'resync'                       = false
Config option 'resync_auth'                  = false
Config option 'cleanup_local_files'          = false
Config option 'classify_as_big_delete'       = 1000
Config option 'disable_upload_validation'    = false
Config option 'disable_download_validation'  = false
Config option 'bypass_data_preservation'     = false
Config option 'no_remote_delete'             = false
Config option 'remove_source_files'          = false
Config option 'sync_dir_permissions'         = 700
Config option 'sync_file_permissions'        = 600
Config option 'space_reservation'            = 52428800
Config option 'application_id'               = d50ca740-c83f-4d1b-b616-12c519384f0c
Config option 'azure_ad_endpoint'            = 
Config option 'azure_tenant_id'              = 
Config option 'user_agent'                   = ISV|abraunegg|OneDrive Client for Linux/v2.5.2-1+np1+1.1
Config option 'force_http_11'                = false
Config option 'debug_https'                  = false
Config option 'rate_limit'                   = 0
Config option 'operation_timeout'            = 3600
Config option 'dns_timeout'                  = 60
Config option 'connect_timeout'              = 10
Config option 'data_timeout'                 = 60
Config option 'ip_protocol_version'          = 0
Config option 'threads'                      = 8
Environment var 'XDG_RUNTIME_DIR'            = true
Environment var 'DBUS_SESSION_BUS_ADDRESS'   = true
Config option 'notify_file_actions'          = false

Selective sync 'sync_list' configured        = true
sync_list config option 'sync_root_files'    = false
sync_list contents:
/foo

Config option 'sync_business_shared_items'   = false

Config option 'webhook_enabled'              = false

Recycle bin:

abraunegg commented 5 days ago

@dalemartin Thanks for the extra details - will look into this as it appears that because of this:

Including file - included by sync_list config: foo/barbar/bar.txt
Parental Path structure needs to be created to support included file: foo/barbar

The remote path structure gets queried and stored in the DB - which is why:

Processing: foo/barbar
The directory has been deleted locally

Because it did not exist locally .... thus the deletion.

abraunegg commented 1 day ago

@dalemartin Please can you test PR #2918 to resolve your issue.

First install all the require platform dependencies to build the client on your platforms. Please read https://github.com/abraunegg/onedrive/blob/master/docs/install.md#building-from-source---high-level-requirements and then follow correctly for your platform.

Once this is done, to clone the PR to resolve your issue, you can use a script like the following:

#!/bin/bash

PR=2918

rm -rf ./onedrive-pr${PR}
git clone https://github.com/abraunegg/onedrive.git onedrive-pr${PR}
cd onedrive-pr${PR}
git fetch origin pull/${PR}/head:pr${PR}
git checkout pr${PR}

# Configure and Build
./configure --enable-debug --enable-notifications; make clean; make;
./onedrive --version

This script will create a local folder called onedrive-pr2918 with the PR version.

To run the PR, you need to run the client from the PR build directory:

./onedrive <any other options needed>

To install the PR, you will need to perform sudo make install to install the PR version to your system.

When running the PR, your version should be: onedrive v2.5.2-17-g063e24e or greater.

Application Configuration

Reading configuration file: /home/alex/.config/onedrive/config
Configuration file successfully loaded
Application version                          = onedrive v2.5.2-17-g063e24e
Compiled with                                = DMD 2109
Curl version                                 = libcurl/8.6.0 OpenSSL/3.2.2 zlib/1.3.1 libidn2/2.3.7 nghttp2/1.59.0
User Application Config path                 = /home/alex/.config/onedrive
System Application Config path               = /etc/onedrive
Applicable Application 'config' location     = /home/alex/.config/onedrive/config
Configuration file found in config location  = true - using 'config' file values to override application defaults
Applicable 'sync_list' location              = /home/alex/.config/onedrive/sync_list
Applicable 'items.sqlite3' location          = /home/alex/.config/onedrive/items.sqlite3
Config option 'drive_id'                     = b!3Uyf_G4r-UKmoULFvShByvgolNg2SJRIhG4F_V15hB2En6zphwcvTJhA3eBRuImN
Config option 'sync_dir'                     = OneDrive_Issue_2910
Config option 'enable_logging'               = false
Config option 'log_dir'                      = /var/log/onedrive
Config option 'disable_notifications'        = false
Config option 'skip_dir'                     = /foo/bar*
Config option 'skip_dir_strict_match'        = false
Config option 'skip_file'                    = ~*|.~*|*.tmp|*.swp|*.partial
Config option 'skip_dotfiles'                = false
Config option 'skip_symlinks'                = false
Config option 'monitor_interval'             = 300
Config option 'monitor_log_frequency'        = 12
Config option 'monitor_fullscan_frequency'   = 12
Config option 'read_only_auth_scope'         = false
Config option 'dry_run'                      = false
Config option 'upload_only'                  = false
Config option 'download_only'                = false
Config option 'local_first'                  = false
Config option 'check_nosync'                 = false
Config option 'check_nomount'                = false
Config option 'resync'                       = false
Config option 'resync_auth'                  = false
Config option 'cleanup_local_files'          = false
Config option 'classify_as_big_delete'       = 1000
Config option 'disable_upload_validation'    = false
Config option 'disable_download_validation'  = false
Config option 'bypass_data_preservation'     = false
Config option 'no_remote_delete'             = false
Config option 'remove_source_files'          = false
Config option 'sync_dir_permissions'         = 700
Config option 'sync_file_permissions'        = 600
Config option 'space_reservation'            = 52428800
Config option 'application_id'               = d50ca740-c83f-4d1b-b616-12c519384f0c
Config option 'azure_ad_endpoint'            = 
Config option 'azure_tenant_id'              = 
Config option 'user_agent'                   = ISV|abraunegg|OneDrive Client for Linux/v2.5.2-17-g063e24e
Config option 'force_http_11'                = false
Config option 'debug_https'                  = false
Config option 'rate_limit'                   = 0
Config option 'operation_timeout'            = 3600
Config option 'dns_timeout'                  = 60
Config option 'connect_timeout'              = 10
Config option 'data_timeout'                 = 60
Config option 'ip_protocol_version'          = 0
Config option 'threads'                      = 8
Config option 'max_curl_idle'                = 120
Environment var 'XDG_RUNTIME_DIR'            = true
Environment var 'DBUS_SESSION_BUS_ADDRESS'   = true
Config option 'notify_file_actions'          = false

Selective sync 'sync_list' configured        = true
sync_list config option 'sync_root_files'    = false
sync_list contents:
# Testing for Issue #2910
/foo

Config option 'sync_business_shared_items'   = false

Config option 'webhook_enabled'              = false

Application Run

./onedrive -s -v --resync  
Reading configuration file: /home/alex/.config/onedrive/config
Configuration file successfully loaded
Using 'user' configuration path for application config and state data: /home/alex/.config/onedrive
D-Bus message bus daemon is available; GUI notifications are now enabled

The usage of --resync will delete your local 'onedrive' client state, thus no record of your current 'sync status' will exist.
This has the potential to overwrite local versions of files with perhaps older versions of documents downloaded from OneDrive, resulting in local data loss.
If in doubt, backup your local data before using --resync

Are you sure you wish to proceed with --resync? [Y/N] y

Deleting the saved application sync status ...
Using IPv4 and IPv6 (if configured) for all network operations
Attempting to contact Microsoft OneDrive Login Service
Successfully reached Microsoft OneDrive Login Service
Checking Application Version ...
Attempting to initialise the OneDrive API ...
Configuring Global Azure AD Endpoints
The OneDrive API was initialised successfully
Opening the item database ...
Application Version:  onedrive v2.5.2-17-g063e24e
Account Type:         documentLibrary
Default Drive ID:     b!3Uyf_G4r-UKmoULFvShByvgolNg2SJRIhG4F_V15hB2En6zphwcvTJhA3eBRuImN
Default Root ID:      01Q5HFMF56Y2GOVW7725BZO354PWSELRRZ
Remaining Free Space: 25599.87 GB (27487652314531 bytes)
Sync Engine Initialised with new Onedrive API instance
All application operations will be performed in the configured local 'sync_dir' directory: /home/alex/OneDrive_Issue_2910
Fetching /delta response from the OneDrive API for Drive ID: b!3Uyf_G4r-UKmoULFvShByvgolNg2SJRIhG4F_V15hB2En6zphwcvTJhA3eBRuImN
Processing API Response Bundle: 1 - Quantity of 'changes|items' in this bundle to process: 201
Skipping path - excluded by sync_list config: ./sfghvbncv
Skipping path - excluded by sync_list config: ./random_images
Skipping path - excluded by sync_list config: random_images/8VN6yerjFX2yuq7qV27A0LWmkiSsOo3L
Processing API Response Bundle: 2 - Quantity of 'changes|items' in this bundle to process: 203
Processing API Response Bundle: 3 - Quantity of 'changes|items' in this bundle to process: 120
Including path - included by sync_list config: ./foo
Including path - included by sync_list config: foo/baz
Including file - included by sync_list config: foo/baz/file2.txt
Parental Path structure needs to be created to support included file: foo/baz
Skipping path - excluded by skip_dir config: ./foo/barbar
Skipping file - file path is excluded by skip_dir config: ./foo/barbar/file1.txt
Finished processing /delta JSON response from the OneDrive API
Processing 3 applicable changes and items received from Microsoft OneDrive
Processing OneDrive JSON item batch [1/1] to ensure consistent local state
Performing a database consistency and integrity check on locally stored data
Processing DB entries for this Drive ID: b!3Uyf_G4r-UKmoULFvShByvgolNg2SJRIhG4F_V15hB2En6zphwcvTJhA3eBRuImN
Processing: OneDrive_Issue_2910
The directory has not changed
Processing: foo
The directory has not changed
Processing: foo/baz
The directory has not changed
Processing: foo/baz/file2.txt
The file has not changed
Scanning the local file system 'OneDrive_Issue_2910' for new data to upload
Skipping file - excluded by sync_list config: ./55_44_33_.trc
Skipping file - excluded by sync_list config: ./aa_dd_cc.ulg
Skipping file - excluded by sync_list config: ./17_24_14.ulg
Skipping file - excluded by sync_list config: ./data_file_.zip
Skipping path - excluded by sync_list config: ./random_images
Skipping path - excluded by sync_list config: ./sfghvbncv
Performing a last examination of the most recent online data within Microsoft OneDrive to complete the reconciliation process
Fetching /delta response from the OneDrive API for Drive ID: b!3Uyf_G4r-UKmoULFvShByvgolNg2SJRIhG4F_V15hB2En6zphwcvTJhA3eBRuImN
Processing API Response Bundle: 1 - Quantity of 'changes|items' in this bundle to process: 0
Finished processing /delta JSON response from the OneDrive API
No changes or items that can be applied were discovered while processing the data received from Microsoft OneDrive

Sync with Microsoft OneDrive is complete
Attempting to perform a database vacuum to optimise database
Database vacuum is complete

Key Points

foo/baz is included via 'sync_list' inclusion
foo/barbar is excluded via 'skip_dir' before hitting 'sync_list' as per expectation
- thus no Parental Path structure needs to be created to support included file causing DB inserts to occur for directories that will not exist, thus no longer triggers online deletion

abraunegg commented 14 hours ago

@dalemartin Any update in testing the provided PR for resolution of this issue?

abraunegg / onedrive