Bug: Segfault on Downloads

axiopaladin commented 6 months ago

Describe the bug

When trying to synchronize, the program segfaults upon trying to download any files. Playing with the sync_list to exclude any given file does not prevent this, so it doesn't seem to be linked to any particular file.

Operating System Details

Alpine Linux (Edge), aarch64

Client Installation Method

From Distribution Package

OneDrive Account Type

Business | Office365

What is your OneDrive Application Version

onedrive v2.4.25

What is your OneDrive Application Configuration

Config path                                  = /home/user/.config/onedrive
Config file found in config path             = false
Config option 'sync_dir'                     = /home/user/OneDrive
Config option 'enable_logging'               = false
Config option 'log_dir'                      = /var/log/onedrive/
Config option 'disable_notifications'        = false
Config option 'min_notify_changes'           = 5
Config option 'skip_dir'                     = 
Config option 'skip_dir_strict_match'        = false
Config option 'skip_file'                    = ~*|.~*|*.tmp
Config option 'skip_dotfiles'                = false
Config option 'skip_symlinks'                = false
Config option 'monitor_interval'             = 300
Config option 'monitor_log_frequency'        = 6
Config option 'monitor_fullscan_frequency'   = 12
Config option 'read_only_auth_scope'         = false
Config option 'dry_run'                      = false
Config option 'upload_only'                  = false
Config option 'download_only'                = false
Config option 'local_first'                  = false
Config option 'check_nosync'                 = false
Config option 'check_nomount'                = false
Config option 'resync'                       = false
Config option 'resync_auth'                  = false
Config option 'cleanup_local_files'          = false
Config option 'classify_as_big_delete'       = 1000
Config option 'disable_upload_validation'    = false
Config option 'bypass_data_preservation'     = false
Config option 'no_remote_delete'             = false
Config option 'remove_source_files'          = false
Config option 'sync_dir_permissions'         = 700
Config option 'sync_file_permissions'        = 600
Config option 'space_reservation'            = 52428800
Config option 'application_id'               = 
Config option 'azure_ad_endpoint'            = 
Config option 'azure_tenant_id'              = common
Config option 'user_agent'                   = 
Config option 'force_http_11'                = false
Config option 'debug_https'                  = false
Config option 'rate_limit'                   = 0
Config option 'operation_timeout'            = 3600
Config option 'dns_timeout'                  = 60
Config option 'connect_timeout'              = 10
Config option 'data_timeout'                 = 600
Config option 'ip_protocol_version'          = 0
Config option 'sync_root_files'              = false
Selective sync 'sync_list' configured        = true
sync_list contents:
!.git/*
!.*.autosave.xopp
/2024*
Config option 'sync_business_shared_folders' = false
Business Shared Folders configured           = false
Config option 'webhook_enabled'              = false

What is your 'curl' version

curl 8.6.0 (aarch64-alpine-linux-musl) libcurl/8.6.0 OpenSSL/3.1.5 zlib/1.3.1 brotli/1.1.0 c-ares/1.26.0 libidn2/2.3.7 libpsl/0.21.5 nghttp2/1.59.0
Release-Date: 2024-01-31
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTPS-proxy IDN IPv6 Largefile libz NTLM PSL SSL threadsafe TLS-SRP UnixSockets

Where is your 'sync_dir' located

Local

What are all your system 'mount points'

proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
dev on /dev type devtmpfs (rw,nosuid,noexec,relatime,size=10240k,nr_inodes=496328,mode=755)
run on /run type tmpfs (rw,nosuid,nodev,relatime,mode=755)
devpts on /dev/pts type devpts (rw,relatime,mode=600,ptmxmode=000)
/dev/mapper/root on / type ext4 (rw,relatime)
/dev/mmcblk0p2 on /boot type ext2 (rw,relatime)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
shm on /dev/shm type tmpfs (rw,nosuid,nodev,noexec,relatime)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,size=1988668k,nr_inodes=409600)
none on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate)
tmpfs on /run/user/10000 type tmpfs (rw,nosuid,nodev,relatime,size=397732k,nr_inodes=99433,mode=700,uid=10000,gid=10000)
gvfsd-fuse on /run/user/10000/gvfs type fuse.gvfsd-fuse (rw,nosuid,nodev,relatime,user_id=10000,group_id=10000)
portal on /run/user/10000/doc type fuse.portal (rw,nosuid,nodev,relatime,user_id=10000,group_id=10000)

What are all your local file system partition types

NAME         FSTYPE FSVER LABEL UUID FSAVAIL FSUSE% MOUNTPOINTS
mtdblock0                                           
mmcblk0                                             
├─mmcblk0p1                                         
├─mmcblk0p2                           160.1M    27% /boot
└─mmcblk0p3                                         
  └─root                               17.4G    64% /
mmcblk0boot0                                        
mmcblk0boot1                                        
zram0                                               [SWAP]

How do you use 'onedrive'

I use it as a cloud backup, synchronized to multiple systems but only one is in active use at a time.

Steps to reproduce the behaviour

I'm not sure exactly what caused it, but now any onedrive --synchronize command (unless it is --upload-only) will segfault like so:

@ onedrive --synchronize
Configuring Global Azure AD Endpoints
Initializing the Synchronization Engine ...
Syncing changes and items from OneDrive ...
Downloading file [redacted] ... 
Segmentation fault

Complete Verbose Log Output

@ onedrive --synchronize -v
Using 'user' Config Dir: /home/user/.config/onedrive
Using 'system' Config Dir: /etc/onedrive
No user or system config file found, using application defaults
Checking Application Version ...
Initializing the OneDrive API ...
Configuring Global Azure AD Endpoints
Using Curl defaults for all HTTP operations
Opening the item database ...
All operations will be performed in: /home/user/OneDrive
Application version: v2.4.25
Account Type: business
Default Drive ID: [redacted]
Default Root ID: 01Q7ZUJ4V6Y2GOVW7725BZO354PWSELRRZ
Remaining Free Space: 5493365116373
Fetching details for OneDrive Root
OneDrive Root exists in the database
Initializing the Synchronization Engine ...
Syncing changes and items from OneDrive ...
Applying changes of Path ID: 01Q7ZUJ4V6Y2GOVW7725BZO354PWSELRRZ
Updated Remaining Free Space: 5493365116373
Processing 213 OneDrive items to ensure consistent local state due to sync_list being used
Skipping item - excluded by sync_list config: ./Archive
Skipping item - excluded by sync_list config: ./Notebooks
Skipping item - excluded by sync_list config: ./Attachments
Skipping item - excluded by sync_list config: ./Microsoft Teams Chat Files
Processing 32 OneDrive items to ensure consistent local state due to sync_list being used
Downloading file [redacted] ... 
Segmentation fault

Screenshots

No response

Other Log Information or Details

No response

Additional context

This first occurred after receiving

Error AADSTS70043: The refresh token has expired or is invalid due to sign-in frequency checks by conditional access. The token was issued on 2023-12-02T05:57:20.2536447Z and the maximum allowed lifetime for this request is 7776000.

and then performing a onedrive --synchronize --reauth to renew the token. I'm not sure if this is related to the issue, or just both problems happened in the same period (the error only appeared when synchronizing for the first time after a fairly long break).

abraunegg commented 6 months ago

@axiopaladin Firstly, a segfault on download is a very odd issue - even on AARCH64 .... no issue like this has ever been reported - even on AARCH64, so this is potentially something unique to how and where the client is trying to write the temporary files when downloading - this is just a theory as your output only shows Segmentation fault and not any of the other additional output which would be generated.

Secondly - regardless, this issue, if a bug, will not be fixed in v2.4.25 - sorry.

My advice is that you change to start using v2.5.0-rc1 - please read https://github.com/abraunegg/onedrive/discussions/2662 on how to obtain, build and start using. In your instance, post compiling, you will need to run make install to ensure you install the latest client.

axiopaladin commented 6 months ago

I've compiled onedrive v2.5.0-rc1-12-ga6e2f77 and it runs into the same problem. I tried running it with the -vvv switch enabled but it doesn't seem to give much more information; here is the last chunk of debug messages before it segfaults on this version:

DEBUG: ------------------------------------------------------------------
DEBUG: Number of JSON items to process is: 447
DEBUG: Number of JSON items processed was: 447
DEBUG: Updating completed deltaLink in DB to: https://graph.microsoft.com/v1.0/drives/[redacted]
DEBUG: Number of items to download from OneDrive: 6
DEBUG: Attempting to calculate local filesystem path for [redacted] and 01Q7ZUJ4TZPPB2GLRSFZDYLQZP7KAPLBUJ
DEBUG: Attempting to calculate local filesystem path for [redacted] and 01Q7ZUJ4TZPPB2GLRSFZDYLQZP7KAPLBUJ
DEBUG: Attempting to calculate local filesystem path for [redacted] and 01Q7ZUJ4SDSWTHRZQMIJCKKCU5D5IIRKYD
DEBUG: Attempting to calculate local filesystem path for [redacted] and 01Q7ZUJ4TZPPB2GLRSFZDYLQZP7KAPLBUJ
DEBUG: Attempting to calculate local filesystem path for [redacted] and 01Q7ZUJ4TZPPB2GLRSFZDYLQZP7KAPLBUJ
DEBUG: Attempting to calculate local filesystem path for [redacted] and 01Q7ZUJ4SDSWTHRZQMIJCKKCU5D5IIRKYD
DEBUG: JSON Item calculated full path for download is: [redacted]/test.txt

Segmentation fault

One odd thing I've noticed is that this v2.5.0-rc1 version seems to convince itself that it has synced successfully, even though it hasn't. (This doesn't happen on v2.4.25.) So after an initial (or --resync) scan and segfault, invoking v2.5.0-rc1 again will report like this:

@ ./onedrive --sync
Using IPv4 and IPv6 (if configured) for all network operations
Configuring Global Azure AD Endpoints
Fetching items from the OneDrive API for Drive ID: [redacted] ..
No additional changes or items that can be applied were discovered while processing the data received from Microsoft OneDrive
Performing a database consistency and integrity check on locally stored data .
Scanning the local file system '~/OneDrive' for new data to upload .
Performing a last examination of the most recent online data within Microsoft OneDrive to complete the reconciliation process
Fetching items from the OneDrive API for Drive ID: [redacted] .
No additional changes or items that can be applied were discovered while processing the data received from Microsoft OneDrive

Sync with Microsoft OneDrive is complete

...despite the fact that it never downloaded any new files/changes.

I'm not sure what things I could try to interrogate next. Is it a problem with my cloud account? Some configuration problem on my local machine that affected both old and new versions?

abraunegg commented 6 months ago

@axiopaladin Please can you do the following:

Recompile v2.5.0-rc1 and add --enable-debug
Generate a debug log using this process: onedrive --resync --resync-auth --sync --verbose --verbose > debug_output.log 2>&1
Archive the debug log and email it through, password protect it if necessary

If you are concerned about data sensitivity, please read:

Some configuration problem on my local machine that affected both old and new versions?

There is probably something wrong with your actual local system - the debug log with compiler debugging enabled should show what is going on. When was the last time you:

Performed a full file system check (fsck etc)? You have redacted your file system type in your post so I do not know what you are running and what to suggest.
Restarted the system?
Run an exhaustive hardware diagnostics?
Run extended system memory checks?

If you could do the above steps to recompile and generated a proper debug log that would be greatly appreciated.

...despite the fact that it never downloaded any new files/changes.

This is because the online 'change' was actually processed already. Microsoft only ever send this change once, and if the file fails to download (error or otherwise) - the only mechanism to get Microsoft to resend the data is to perform a --resync

abraunegg commented 6 months ago

@axiopaladin Any update or info regarding these 4 questions:

Performed a full file system check (fsck etc)? You have redacted your file system type in your post so I do not know what you are running and what to suggest.
Restarted the system?
Run an exhaustive hardware diagnostics?
Run extended system memory checks?

JC-comp commented 6 months ago

One odd thing I've noticed is that this v2.5.0-rc1 version seems to convince itself that it has synced successfully, even though it hasn't.

@abraunegg This is precisely what issues #2660 and #2637 were about, the behavior simply does not meet users' expectations. Although there exists a solution for this, it only works until a user explicitly identifies the inconsistency. There must be some ways to prevent this. At least I still don't understand why you claim these changes are only sent once as discussed in https://github.com/abraunegg/onedrive/pull/2637#issuecomment-1986568567.

abraunegg commented 6 months ago

@JC-comp

One odd thing I've noticed is that this v2.5.0-rc1 version seems to convince itself that it has synced successfully, even though it hasn't.

@abraunegg This is precisely what issues #2660 and #2637 were about, the behavior simply does not meet users' expectations. Although there exists a solution for this, it only works until a user explicitly identifies the inconsistency. There must be some ways to prevent this. At least I still don't understand why you claim these changes are only sent once as discussed in #2637 (comment).

And this is why it would be beneficial to you to communicate offline or via MS Teams , so I can show you want I mean by this statement - and how various elements are only ever sent once because of the design of the OneDrive API - however you are reluctant to do this.

Outside of this, I have been working on a proper solution - but not that you would be aware, as you do not wish to communicate offline.

abraunegg commented 6 months ago

@axiopaladin Please can you respond regarding the questions I have raised.

abraunegg commented 6 months ago

@axiopaladin Please can you respond regarding the questions I have raised.

abraunegg commented 6 months ago

@axiopaladin

One odd thing I've noticed is that this v2.5.0-rc1 version seems to convince itself that it has synced successfully, even though it hasn't.

This particular issue has been resolved in https://github.com/abraunegg/onedrive/pull/2661/commits/3d660921dd8273a73ecab687345e178489e053df

abraunegg commented 6 months ago

@axiopaladin Please can you respond regarding the questions I have raised:

Have you performed a full file system check (fsck etc)? You have redacted your file system type in your post so I do not know what you are running and what to suggest.
When did you last restart the system?
Have you run an exhaustive hardware diagnostics?
Have you run extended system memory checks?

This issue will be closed 5th April if no response is received.

abraunegg commented 5 months ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

abraunegg / onedrive