Bug: Files modified on upload (doc/docx/xls/xlsx) with SharePoint Shared Libraries

lapseofreason commented 4 months ago

Describe the bug

When uploading certain file types (namely microsoft office documents, such as doc/docx/xls/xlsx), the files are modified when they are uploaded to a SharePoint Shared Library.

Operating System Details

Linux XXXXX 6.6.7-200.fc39.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Dec 13 21:43:37 UTC 2023 x86_64 GNU/Linux

Fedora release 39 (Thirty Nine)

Client Installation Method

From Distribution Package

OneDrive Account Type

Business | Office365

What is your OneDrive Application Version

onedrive v2.4.25

What is your OneDrive Application Configuration

Configuration file successfully loaded
onedrive version                             = v2.4.25
Config path                                  = /home/XXXXX/.config/onedrive/
Config file found in config path             = true
Config option 'drive_id'                     = XXXXX
Config option 'sync_dir'                     = /home/XXXXX/Documents/OneDrive
Config option 'enable_logging'               = true
Config option 'log_dir'                      = /home/XXXXX/.cache/onedrive/
Config option 'disable_notifications'        = false
Config option 'min_notify_changes'           = 5
Config option 'skip_dir'                     = 
Config option 'skip_dir_strict_match'        = false
Config option 'skip_file'                    = ~*|.~*|*.tmp
Config option 'skip_dotfiles'                = false
Config option 'skip_symlinks'                = false
Config option 'monitor_interval'             = 60
Config option 'monitor_log_frequency'        = 6
Config option 'monitor_fullscan_frequency'   = 12
Config option 'read_only_auth_scope'         = false
Config option 'dry_run'                      = false
Config option 'upload_only'                  = false
Config option 'download_only'                = false
Config option 'local_first'                  = false
Config option 'check_nosync'                 = false
Config option 'check_nomount'                = false
Config option 'resync'                       = false
Config option 'resync_auth'                  = false
Config option 'cleanup_local_files'          = false
Config option 'classify_as_big_delete'       = 1000
Config option 'disable_upload_validation'    = false
Config option 'bypass_data_preservation'     = false
Config option 'no_remote_delete'             = false
Config option 'remove_source_files'          = false
Config option 'sync_dir_permissions'         = 700
Config option 'sync_file_permissions'        = 600
Config option 'space_reservation'            = 52428800
Config option 'application_id'               = 
Config option 'azure_ad_endpoint'            = 
Config option 'azure_tenant_id'              = common
Config option 'user_agent'                   = 
Config option 'force_http_11'                = false
Config option 'debug_https'                  = false
Config option 'rate_limit'                   = 0
Config option 'operation_timeout'            = 3600
Config option 'dns_timeout'                  = 60
Config option 'connect_timeout'              = 10
Config option 'data_timeout'                 = 600
Config option 'ip_protocol_version'          = 0
Config option 'sync_root_files'              = false
Selective sync 'sync_list' configured        = false
Config option 'sync_business_shared_folders' = false
Business Shared Folders configured           = false
Config option 'webhook_enabled'              = false

What is your 'curl' version

curl 8.2.1 (x86_64-redhat-linux-gnu) libcurl/8.2.1 OpenSSL/3.1.1 zlib/1.2.13 brotli/1.1.0 libidn2/2.3.7 libpsl/0.21.2 (+libidn2/2.3.4) libssh/0.10.6/openssl/zlib nghttp2/1.55.1 OpenLDAP/2.6.6
Release-Date: 2023-07-26
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets

Where is your 'sync_dir' located

Local

What are all your system 'mount points'

/dev/mapper/XXXXX_vg-root_lv on / type btrfs (rw,relatime,seclabel,compress=zstd:3,ssd,discard=async,space_cache,subvolid=8612,subvol=/fedora)
/dev/mapper/XXXXX_vg-home_lv on /home type btrfs (rw,noatime,seclabel,compress=zstd:3,ssd,discard=async,space_cache,subvolid=1677,subvol=/home)
devtmpfs on /dev type devtmpfs (rw,nosuid,seclabel,size=4096k,nr_inodes=2005510,mode=755,inode64)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,seclabel,inode64)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime,seclabel)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,seclabel,size=4096k,nr_inodes=1024,mode=755,inode64)
cgroup2 on /sys/fs/cgroup/unified type cgroup2 (rw,nosuid,nodev,noexec,relatime,seclabel,nsdelegate)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,xattr,name=systemd)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,pids)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,perf_event)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,net_cls,net_prio)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,memory)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,cpu,cpuacct)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,freezer)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,hugetlb)
cgroup on /sys/fs/cgroup/rdma type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,rdma)
cgroup on /sys/fs/cgroup/misc type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,misc)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,devices)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,blkio)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime,seclabel)
efivarfs on /sys/firmware/efi/efivars type efivarfs (rw,nosuid,nodev,noexec,relatime)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
tmpfs on /run type tmpfs (rw,nosuid,nodev,seclabel,size=3218804k,nr_inodes=819200,mode=755,inode64)
selinuxfs on /sys/fs/selinux type selinuxfs (rw,nosuid,noexec,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=34,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13453)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime,seclabel)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,nosuid,nodev,relatime,seclabel,pagesize=2M)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime,seclabel)
tracefs on /sys/kernel/tracing type tracefs (rw,nosuid,nodev,noexec,relatime,seclabel)
fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,seclabel,size=8047004k,nr_inodes=1048576,inode64)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,nosuid,nodev,noexec,relatime)
tmpfs on /run/user/1005 type tmpfs (rw,nosuid,nodev,relatime,seclabel,size=1609400k,nr_inodes=402350,mode=700,uid=1005,gid=1005,inode64)
gvfsd-fuse on /run/user/1005/gvfs type fuse.gvfsd-fuse (rw,nosuid,nodev,relatime,user_id=1005,group_id=1005)
portal on /run/user/1005/doc type fuse.portal (rw,nosuid,nodev,relatime,user_id=1005,group_id=1005)

What are all your local file system partition types

NAME                                          FSTYPE      FSVER    LABEL       UUID                                   FSAVAIL FSUSE% MOUNTPOINTS

How do you use 'onedrive'

The OneDrive drive I used to test this is not shared with anyone else and not synced anywhere else.

Steps to reproduce the behaviour

Steps to reproduce:

Set up sync for a SharePoint Shared Library as described in https://github.com/abraunegg/onedrive/blob/master/docs/SharePoint-Shared-Libraries.md
Create a new empty Word file (sample.docx).
Checksum the file (sha1sum sample.docx).
Move the file to the shared folder and wait for it to be uploaded.
Log into SharePoint in the browser and download the file.
Checksum the downloaded file.

Expected result:

The checksum of both files match.

Actual result:

The checksum of the files do not match.

Not reproducable for regular file sync (not SharePoint Shared Library). The checksums of other file types do match when tested in the same way.

Complete Verbose Log Output

In the sync client the following error appears for the uploaded file:

WARNING: Uploaded file size does not match local file - skipping upload validation

Screenshots

No response

Other Log Information or Details

No response

Additional context

No response

abraunegg commented 4 months ago

@lapseofreason This is not a bug with this application.

If you increase your logging by adding --verbose you will get a Microsoft URL that details why this occurs: https://github.com/OneDrive/onedrive-api-docs/issues/935

Essentially - Microsoft is modifying your file post upload when you are uploading to SharePoint or any Business account that is back ended by SharePoint.

Please go complain to Microsoft on this issue - not here.

lapseofreason commented 4 months ago

@abraunegg Thanks for clarifying this, and sorry if this seemed offensive, I was just trying to be helpful by reporting this. I appreciate the tool, it is really useful to be able to sync on Linux.

Would if be worthwhile to add a this as a known issue to https://github.com/abraunegg/onedrive/blob/master/docs/SharePoint-Shared-Libraries.md ? I think it might help other people understand that this is not an issue with this tool, but with Microsoft.

abraunegg commented 4 months ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

abraunegg / onedrive