search

abraunegg / onedrive

OneDrive Client for Linux
https://abraunegg.github.io
GNU General Public License v3.0
9.63k stars 849 forks source link

Bug: Intermittent Yet Persistent Timeout Error with Microsoft OneDrive Service Despite Internet Connectivity #2733

Closed theeggdev closed 1 month ago

theeggdev commented 1 month ago

Describe the bug

  1. OneDrive service running as root user via systemd
  2. Approximately once every day, a timeout error appears while accessing the Microsoft OneDrive service.
  3. The error message always states: “There was a timeout in accessing the Microsoft OneDrive service - Internet connectivity issue.”
  4. This error persists until the sync client is restarted with systemd.
  5. The issue does not seem to be related to actual internet connectivity, as the service works normally before the error occurs and other internet-dependent services continue to function properly.
  6. This issue appeared about 2 months after we upgrade the OneDrive syncing version to the latest 2.4.25. And since the first day of the occurance, the issue repeated consistently for 1.5 month, and still continuing now.

Operating System Details

Output of uname -a
Linux eggdevvm 5.4.0-1074-azure #77~18.04.1-Ubuntu SMP Wed Mar 30 15:36:02 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Output of lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.6 LTS
Release:        18.04
Codename:       bionic

Client Installation Method

From Source

OneDrive Account Type

SharePoint

What is your OneDrive Application Version

onedrive v2.4.25-13-g1a88d33

What is your OneDrive Application Configuration

Configuration file successfully loaded
onedrive version                             = v2.4.25-13-g1a88d33
Config path                                  = /root/.config/onedrive
Config file found in config path             = true
Config option 'drive_id'                     = <hidden; provide if needed>
Config option 'sync_dir'                     = /root/OneDrive
Config option 'enable_logging'               = false
Config option 'log_dir'                      = /var/log/onedrive/
Config option 'disable_notifications'        = false
Config option 'min_notify_changes'           = 5
Config option 'skip_dir'                     =
Config option 'skip_dir_strict_match'        = false
Config option 'skip_file'                    = ~*|.~*|*.tmp
Config option 'skip_dotfiles'                = false
Config option 'skip_symlinks'                = false
Config option 'monitor_interval'             = 300
Config option 'monitor_log_frequency'        = 6
Config option 'monitor_fullscan_frequency'   = 12
Config option 'read_only_auth_scope'         = false
Config option 'dry_run'                      = false
Config option 'upload_only'                  = false
Config option 'download_only'                = false
Config option 'local_first'                  = false
Config option 'check_nosync'                 = false
Config option 'check_nomount'                = false
Config option 'resync'                       = false
Config option 'resync_auth'                  = false
Config option 'cleanup_local_files'          = false
Config option 'classify_as_big_delete'       = 1000
Config option 'disable_upload_validation'    = false
Config option 'bypass_data_preservation'     = false
Config option 'no_remote_delete'             = false
Config option 'remove_source_files'          = false
Config option 'sync_dir_permissions'         = 700
Config option 'sync_file_permissions'        = 600
Config option 'space_reservation'            = 52428800
Config option 'application_id'               =
Config option 'azure_ad_endpoint'            =
Config option 'azure_tenant_id'              = common
Config option 'user_agent'                   =
Config option 'force_http_11'                = false
Config option 'debug_https'                  = false
Config option 'rate_limit'                   = 0
Config option 'operation_timeout'            = 3600
Config option 'dns_timeout'                  = 60
Config option 'connect_timeout'              = 30
Config option 'data_timeout'                 = 600
Config option 'ip_protocol_version'          = 1
Config option 'sync_root_files'              = false
Selective sync 'sync_list' configured        = false
Config option 'sync_business_shared_folders' = false
Business Shared Folders configured           = false
Config option 'webhook_enabled'              = false

What is your 'curl' version

curl 7.58.0 (x86_64-pc-linux-gnu) libcurl/7.58.0 OpenSSL/1.1.1 zlib/1.2.11 libidn2/2.0.4 libpsl/0.19.1 (+libidn2/2.0.4) nghttp2/1.30.0 librtmp/2.3
Release-Date: 2018-01-24
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL

Where is your 'sync_dir' located

Local

What are all your system 'mount points'

sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=16423456k,nr_inodes=4105864,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=3288444k,mode=755)
/dev/sdb1 on / type ext4 (rw,relatime,discard)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/unified type cgroup2 (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/rdma type cgroup (rw,nosuid,nodev,noexec,relatime,rdma)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=24,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=435)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M)
mqueue on /dev/mqueue type mqueue (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
configfs on /sys/kernel/config type configfs (rw,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
/dev/sdb15 on /boot/efi type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro,discard)
/dev/sda1 on /mnt type ext4 (rw,relatime,x-systemd.requires=cloud-init.service,_netdev)
lxcfs on /var/lib/lxcfs type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,size=3288440k,mode=700,uid=1000,gid=1000)

What are all your local file system partition types

NAME    FSTYPE LABEL           UUID                                 MOUNTPOINT
sda
└─sda1  ext4                   7b2e1872-d512-4f59-955a-1ba9108c4354 /mnt
sdb
├─sdb1  ext4   cloudimg-rootfs a0d5f59f-1ad5-45b5-9e23-bd4186644019 /
├─sdb14
└─sdb15 vfat   UEFI            7011-7B8D                            /boot/efi

How do you use 'onedrive'

We are syncing a SharePoint document library in our linux VM. The SharePoint document library is read and written by ~10 users using Windows OS laptops. Those users sync the SharePoint document library to their laptops using the official MS OneDrive syncing application. Our linux VM also reads and writes to the same SharePoint document library.

Steps to reproduce the behaviour

  1. Run the onedrive sync client as root user via systemd
  2. Wait for approx a day to get the “There was a timeout in accessing the Microsoft OneDrive service - Internet connectivity issue."
  3. And then you can see this network connectivity issue will not be resolved until you restart the sync client.
  4. And once it is restarted, it won't run into the same error in 1 or 2 days.

I understand it might be impossible for you to reproduce the error, so please let me know if there's anything I could provide to you further to assist the investigation.

Complete Verbose Log Output

I am going to send you a full debug log via email. 

As for the full log of the problem when running the application, I run the sync client with systemd, the log in /var/log/syslog only shows us the error “There was a timeout in accessing the Microsoft OneDrive service - Internet connectivity issue." Didn't tell us much details. 

I am thining if I shall change the following line in /lib/systemd/system/onedrive@.service to enable verbose
 ExecStart=/usr/local/bin/onedrive --monitor --confdir=/home/%i/.config/onedrive 

to provide you with more details. Please let me know if you need this. Thanks :)

Screenshots

No response

Other Log Information or Details

No response

Additional context

No response

abraunegg commented 1 month ago

@theeggdev Unfortunately where things are at in development of v2.5.x this will not be looked at.

Additionally, Ubuntu 18.x is EOL|EOS - now I know Ubuntu has this LTS program where it will be supported out to 2028, but you should strongly consider upgrading to a newer version of Ubuntu. I do not even test on this platform any more. As far as this client is concerned, Ubuntu 18.x is unsupported.

Additionally when investigating this in the past, 'curl' has been seen to internally flipflop for zero reason for DNS resolution between IPv4 and IPv6 - which heavily contributes to this issue - thus that part is is technically a 'curl' bug. The client, because of it's single threaded use of 'curl' get's hung up on this - thus this is potentially causing your issue.

How 'curl' is used has been totally reworked in v2.5.x and the client has been 100% totally re-written.

Your options to resolve:

  1. Upgrade to a more modern platform - I strongly advise you doing this anyway
  2. Configure your client with the following options and re-test: 
    ip_protocol_version = "1"
force_http_11 = "true"
  3. Upgrade your client to v2.5.0-rc2 , however there potentially will be an rc3 not too far away, and, as Ubuntu 18.x is considered unsupported, this has not been even tested - and will not be.
theeggdev commented 1 month ago

Thank you so much @abraunegg for your (as always) impressively prompt reply. We are already looking into upgrading our Ubuntu version. Will also implement your suggestions to have a try. Really appreciate your help! Have a great day :)

abraunegg commented 1 month ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.