search

abraunegg / onedrive

OneDrive Client for Linux
https://abraunegg.github.io
GNU General Public License v3.0
10.17k stars 865 forks source link

Initialization always failing on Lubuntu 20.04 LTS. #1186

Closed alessandro-lion closed 3 years ago

alessandro-lion commented 3 years ago

Note:

Bug Report Details

Describe the bug After the enter Uri step I always get an error due to timeout. I tried to spin up a new VM and upgrade to the latest Lubuntu LTS 20.04

Application and Operating System Details:

pilion@lubuntu64:/var/log$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.1 LTS
Release:    20.04
Codename:   focal
pilion@lubuntu64:/var/log$

To Reproduce Steps to reproduce the behavior if not causing an application crash:

  1. Add the repository
  2. Install the package
  3. run the first time and copy and paste the URL into firefox
  4. Paste back the url and press enter I did it many times always same results, my internet connection is fine, on the same PC Lubuntu I can access web version of onedrive using firefox, the PC is a virtual machine running on a Windows 10 host where the Onedrive app is synching without issues.

If issue is replicated by a specific 'file' or 'path' please archive the file and path tree & email to support@mynas.com.au

Complete Verbose Log Output A clear and full log of the problem when running the application in the following manner (ie, not in monitor mode):

pilion@lubuntu64:~$ onedrive --skip-file 1 --verbose
Using 'user' Config Dir: /home/pilion/.config/onedrive
Using 'system' Config Dir: /onedrive
No user or system config file found, using application defaults
Initializing the OneDrive API ...
Configuring Global Azure AD Endpoints
Authorize this app visiting:

https://login.microsoftonline.com/common/oauth2/v2.0/authorize?client_id=d50ca740-c83f-4d1b-b616-12c519384f0c&scope=Files.ReadWrite%20Files.ReadWrite.all%20Sites.Read.All%20Sites.ReadWrite.All%20offline_access&response_type=code&redirect_uri=https://login.microsoftonline.com/common/oauth2/nativeclient

Enter the response uri: https://login.microsoftonline.com/common/oauth2/nativeclient?code=M.R3_BAY.c7b74c70-f42f-ef00-6bc8-cd5f7547a99a
ERROR: OneDrive returned an error with the following message:
  Error Message: Timeout was reached on handle 55EE39E076C0

Application has not been successfully authorised. Please check your URI response entry and try again.

pilion@lubuntu64:~$

Run the application in a separate terminal window or SSH session and provide the entire application output including the error & crash. When posing the logs, Please format log output to make it easier to read. See https://guides.github.com/features/mastering-markdown/ for more details.

Application Log Output:

likely you can reproduce it on any system, in case feel free to ask

Screenshots

Additional context pilion@lubuntu64:~$ uname -a Linux lubuntu64 5.4.0-58-generic #64-Ubuntu SMP Wed Dec 9 08:16:25 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

The onedrive account I am using is a free one registered as email *@outlook.com with over 3GB of content on it

Bug Report Checklist

abraunegg commented 3 years ago

@alessandro-lion You have a firewall issue.

You need to ensure the application can:

This is not an application issue or bug

alessandro-lion commented 3 years ago

Nope, network is OK. Definitely not on the higher speed connection, about 75ms for the latency but it is stable and all the other Onedrive clients I am randomly using with the same account since months ago works fine. After getting that error again I tried to update a small doc on the one drive folder from a Windows 7 VM hosted on the same PC of the Lubuntu, the Windows 10 Onedrive client on the hosting PC itself got the change without any error. I even powered on a legacy Win7 64bit netbook which was not in use since a months, took a bit longer since it had more than that file to sync but got the updates. Not any of them gave errors on onedrive sync except the Linux Machine, seems a bug to me. I came back to the terminal and could execute these:

pilion@lubuntu64:~$ ping -c 1 www.outlook.com
PING zrh-efz.ms-acdc.office.com (52.97.201.242) 56(84) bytes of data.
64 bytes from 52.97.201.242 (52.97.201.242): icmp_seq=1 ttl=128 time=76.2 ms

--- zrh-efz.ms-acdc.office.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 76.214/76.214/76.214/0.000 ms
pilion@lubuntu64:~$ curl -I --http2 https://login.microsoftonline.com
HTTP/1.1 200 OK
Cache-Control: no-store, no-cache
Pragma: no-cache
Content-Length: 148819
Content-Type: text/html; charset=utf-8
Expires: -1
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Content-Type-Options: nosniff
Link: <https://aadcdn.msauth.net>; rel=preconnect; crossorigin
Link: <https://aadcdn.msauth.net>; rel=dns-prefetch
Link: <https://aadcdn.msftauth.net>; rel=dns-prefetch
X-DNS-Prefetch-Control: on
P3P: CP="DSP CUR OTPi IND OTRi ONL FIN"
x-ms-request-id: 02c3e417-cbf5-4fe9-96e9-9d7694024f00
x-ms-ests-server: 2.1.11328.10 - WEULR2 ProdSlices
Set-Cookie: fpc=AqAJD5wdaRxEoSO5cEzJP0U; expires=Fri, 15-Jan-2021 19:57:50 GMT; path=/; secure; HttpOnly; SameSite=None
Set-Cookie: esctx=AQABAAAAAABeStGSRwwnTq2vHplZ9KL4_OEE3XjMg9VvQetQVGYSlMipoSh6MG5950-ALKcOGAsikAkgMNBDBcoaWUY-4qfRMSqanDyikZtwIp8rPfnFXpBxQdxksX57KbI1doqgSVtIs7dhqZLho_vIcMjiwgC5lAX6Hl7TlpKiFjFwFhzU3RB47CxltFEf5OsSF7_qOU4gAA; domain=.login.microsoftonline.com; path=/; secure; HttpOnly; SameSite=None
Set-Cookie: x-ms-gateway-slice=estsfd; path=/; secure; samesite=none; httponly
Set-Cookie: stsservicecookie=estsfd; path=/; secure; samesite=none; httponly
Date: Wed, 16 Dec 2020 19:57:49 GMT
pilion@lubuntu64:/var/log$ sudo iptables -L
[sudo] password for pilion: 
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
pilion@lubuntu64:/var/log$
abraunegg commented 3 years ago

@alessandro-lion No .. this is not a bug. This is an issue with your VM, your routing, your access lists and policies governing network access.

Something on your system, when the application is starting up, is preventing HTTPS communication / DNS lookups from occurring correctly from your VM. It might be some sort of ARP cache, network cache, VM network table - but something on your system is preventing the communication from occurring.

curl -I --http2 https://login.microsoftonline.com

The client does not use HTTPS2, because the MS API does not handle it correctly, thus, HTTP1.1 is used when the client is in operation.

This is not a bug with this client.

abraunegg commented 3 years ago

@alessandro-lion To provide further evidence to you that this is a local environment issue, please test the following PR:

git clone https://github.com/abraunegg/onedrive.git
cd onedrive
git fetch origin pull/1188/head:pr1188
git checkout pr1188
./configure; make clean; make;

When running the PR, your version should be: onedrive v2.4.8-7-ge93dddd or greater.

This has added verbosity around error handling, which the Ubuntu package lacks which should provide you with enough evidence that this issue manifesting itself because of your environment.

When running the application, use:

--verbose --verbose --debug-https

And you will see what the client is doing, what it is attempting to access and where it is failing.

Your error message:

Timeout was reached on handle

This means: The curl engine timed out. Either a response was not received, or something is blocking communication.

abraunegg commented 3 years ago

Closing issue, not a bug.

alessandro-lion commented 3 years ago

@alessandro-lion To provide further evidence to you that this is a local environment issue, please test the following PR:

git clone https://github.com/abraunegg/onedrive.git
cd onedrive
git fetch origin pull/1188/head:pr1188
git checkout pr1188
./configure; make clean; make;

When running the PR, your version should be: onedrive v2.4.8-7-ge93dddd or greater.

This has added verbosity around error handling, which the Ubuntu package lacks which should provide you with enough evidence that this issue manifesting itself because of your environment.

When running the application, use:

--verbose --verbose --debug-https

And you will see what the client is doing, what it is attempting to access and where it is failing.

Your error message:

Timeout was reached on handle

This means: The curl engine timed out. Either a response was not received, or something is blocking communication.

Probably I was not clear enough: Network is OK, Firefox, Curl, and everything else always worked fine within the Lubuntu VM . Another VM running Windows 7 on the same host works fine with Microsoft OneDrive client. Two more physical machines, one running Windows 7 and another Windows 10 are working without any issue as well, these are using the same network/router/connection/DNS.

This afternoon, immediately after another attempt failed again with timeout using the same OneDrive free client version I had already downloaded from the unofficial ubuntu ( PPA ) repository I installed rclone. Launched the rclone config tool, it is running on the same Lubuntu 20.04 LTS iside the same console window where I tried without success OneDrive free client. With it, rclone, I could authorize the access for the same outloook.com free personal account, no issues, and now I am able to mount the drive and get access to files I have on it.

Question: Which kind of network/firewall issue could be not affecting any other software which is connecting to MS Onedrive service successfully from the same network and even in the same Lubuntu virtual computer ?

abraunegg commented 3 years ago

@alessandro-lion

This afternoon, immediately after another attempt failed again with timeout using the same OneDrive free client version ... Which kind of network/firewall issue could be not affecting any other software which is connecting to MS Onedrive service successfully from the same network and even in the same Lubuntu virtual computer

Please uninstall whatever you have installed and build this client from source using the Ubuntu 20.x directions.

When running the application, use as additional switches:

--verbose --verbose --debug-https

This will give you the answer you are seeking. This is a problem with:

Just to re-iterate - this is not a problem with this client. The client cannot communicate with the required OneDrive services.

Use the verbose debug logging commands provided above to provide you with the details needed to answer your question.

The same details will also show if this is a bug or not - so please do this testing and provide the evidence of what is occurring in your environment which (and I have tried) cannot replicate at all.

abraunegg commented 3 years ago

@alessandro-lion Just to fully disclose my testing, below are the results using:

OS version:

alex@alex-lubuntu-20-04:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.1 LTS
Release:        20.04
Codename:       focal

Application version:

alex@alex-lubuntu-20-04:~$ onedrive --version
onedrive v2.4.8-0~202012020750~ubuntu20.04.1

Application configuration:

alex@alex-lubuntu-20-04:~$ onedrive --display-config
onedrive version                       = v2.4.8-0~202012020750~ubuntu20.04.1
Config path                            = /home/alex/.config/onedrive
Config file found in config path       = false
Config option 'check_nosync'           = false
Config option 'sync_dir'               = /home/alex/OneDrive
Config option 'skip_dir'               = 
Config option 'skip_file'              = ~*|.~*|*.tmp
Config option 'skip_dotfiles'          = false
Config option 'skip_symlinks'          = false
Config option 'monitor_interval'       = 300
Config option 'min_notify_changes'     = 5
Config option 'log_dir'                = /var/log/onedrive/
Config option 'classify_as_big_delete' = 1000
Config option 'upload_only'            = false
Config option 'no_remote_delete'       = false
Config option 'remove_source_files'    = false
Config option 'sync_root_files'        = false
Selective sync 'sync_list' configured  = false
Business Shared Folders configured     = false

Authenticate the application:

alex@alex-lubuntu-20-04:~$ onedrive --verbose
Using 'user' Config Dir: /home/alex/.config/onedrive
Using 'system' Config Dir: /etc/onedrive
No user or system config file found, using application defaults
Initializing the OneDrive API ...
Configuring Global Azure AD Endpoints
Authorize this app visiting:

https://login.microsoftonline.com/common/oauth2/v2.0/authorize?client_id=d50ca740-c83f-4d1b-b616-12c519384f0c&scope=Files.ReadWrite%20Files.ReadWrite.all%20Sites.Read.All%20Sites.ReadWrite.All%20offline_access&response_type=code&redirect_uri=https://login.microsoftonline.com/common/oauth2/nativeclient

Enter the response uri: https://login.microsoftonline.com/common/oauth2/nativeclient?code=<redacted>

Application has been successfully authorised, however no additional command switches were provided.

Please use --help for further assistance in regards to running this application.

alex@alex-lubuntu-20-04:~$

Authenticate the application with --verbose --verbose --debug-https:

alex@alex-lubuntu-20-04:~$ onedrive --verbose --verbose --debug-https
[DEBUG] homePath: HOME environment variable set
[DEBUG] homePath: /home/alex
[DEBUG] configDirBase: WARNING - no XDG_CONFIG_HOME environment variable set
[DEBUG] configDirBase: /home/alex/.config
[DEBUG] configDirName: Configuring application to use default config path
Using 'user' Config Dir: /home/alex/.config/onedrive
Using 'system' Config Dir: /etc/onedrive
[DEBUG] refreshTokenFilePath = /home/alex/.config/onedrive/refresh_token
[DEBUG] deltaLinkFilePath = /home/alex/.config/onedrive/delta_link
[DEBUG] databaseFilePath = /home/alex/.config/onedrive/items.sqlite3
[DEBUG] databaseFilePathDryRun = /home/alex/.config/onedrive/items-dryrun.sqlite3
[DEBUG] uploadStateFilePath = /home/alex/.config/onedrive/resume_upload
[DEBUG] userConfigFilePath = /home/alex/.config/onedrive/config
[DEBUG] syncListFilePath = /home/alex/.config/onedrive/sync_list
[DEBUG] systemConfigFilePath = /etc/onedrive/config
[DEBUG] businessSharedFolderFilePath = /home/alex/.config/onedrive/business_shared_folders
No user or system config file found, using application defaults
[DEBUG] sync_dir: Getting syncDir from config value sync_dir
[DEBUG] sync_dir: A '~' was found in configured sync_dir, automatically expanding as SHELL and USER environment variable is set
[DEBUG] syncDir: /home/alex/OneDrive
[DEBUG] Testing network to ensure network connectivity to Microsoft OneDrive Service
Initializing the OneDrive API ...
Configuring Global Azure AD Endpoints
[DEBUG] Downgrading all HTTP operations to HTTP/1.1 by default
[DEBUG] clientId    = d50ca740-c83f-4d1b-b616-12c519384f0c
[DEBUG] companyName = abraunegg
[DEBUG] appTitle    = OneDrive Client for Linux
Authorize this app visiting:

https://login.microsoftonline.com/common/oauth2/v2.0/authorize?client_id=d50ca740-c83f-4d1b-b616-12c519384f0c&scope=Files.ReadWrite%20Files.ReadWrite.all%20Sites.Read.All%20Sites.ReadWrite.All%20offline_access&response_type=code&redirect_uri=https://login.microsoftonline.com/common/oauth2/nativeclient

Enter the response uri: https://login.microsoftonline.com/common/oauth2/nativeclient?code=<redacted>
*   Trying 20.190.168.0:443...
* TCP_NODELAY set
* Connected to login.microsoftonline.com (20.190.168.0) port 443 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: C=US; ST=Washington; L=Redmond; O=Microsoft Corporation; CN=stamp2.login.microsoftonline.com
*  start date: Oct  9 00:00:00 2020 GMT
*  expire date: Oct  9 12:00:00 2021 GMT
*  subjectAltName: host "login.microsoftonline.com" matched cert's "login.microsoftonline.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
*  SSL certificate verify ok.
> POST /common/oauth2/v2.0/token HTTP/1.1
Host: login.microsoftonline.com
User-Agent: ISV|abraunegg|OneDrive Client for Linux/v2.4.8-0~202012020750~ubuntu20.04.1
Accept: */*
Content-Length: 201
Content-Type: application/x-www-form-urlencoded

* We are completely uploaded and fine
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Cache-Control: no-store, no-cache
< Pragma: no-cache
< Content-Type: application/json; charset=utf-8
< Expires: -1
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< X-Content-Type-Options: nosniff
< P3P: CP="DSP CUR OTPi IND OTRi ONL FIN"
< x-ms-request-id: 2118838e-6d6e-4d27-a4b0-b306e83b4200
< x-ms-ests-server: 2.1.11328.10 - AUSE1 ProdSlices
< Set-Cookie: fpc=AiBQ4rYTamBEnjyrKkmrUUpFOQwBAQAAADjNbdcOAAAA; expires=Sat, 16-Jan-2021 22:05:45 GMT; path=/; secure; HttpOnly; SameSite=None
< Set-Cookie: x-ms-gateway-slice=prod; path=/; secure; samesite=none; httponly
< Set-Cookie: stsservicecookie=ests; path=/; secure; samesite=none; httponly
< Date: Thu, 17 Dec 2020 22:05:44 GMT
< Content-Length: 1735
< 
[DEBUG] onedrive.perform() => OneDrive HTTP Server Response: 200
* Connection #0 to host login.microsoftonline.com left intact
[DEBUG] onedrive.perform() => HTTP Response Headers: ["strict-transport-security":"max-age=31536000; includeSubDomains", "x-ms-ests-server":"2.1.11328.10 - AUSE1 ProdSlices", "p3p":"CP=\"DSP CUR OTPi IND OTRi ONL FIN\"", "x-ms-request-id":"2118838e-6d6e-4d27-a4b0-b306e83b4200", "date":"Thu, 17 Dec 2020 22:05:44 GMT", "x-content-type-options":"nosniff", "content-length":"1735", "set-cookie":"stsservicecookie=ests; path=/; secure; samesite=none; httponly", "cache-control":"no-store, no-cache", "expires":"-1", "pragma":"no-cache", "content-type":"application/json; charset=utf-8"]
[DEBUG] Setting file permissions for: /home/alex/.config/onedrive/refresh_token

Application has been successfully authorised, however no additional command switches were provided.

Please use --help for further assistance in regards to running this application.

alex@alex-lubuntu-20-04:~$
alessandro-lion commented 3 years ago

Hello there, I confirm that issue does not show up using LCD compiled from github pr1188. I also can tell that either the package from Ubuntu repository and the one from ppa:yann1ck/onedrive gave me that on the same virtual machine Lubuntu LTS 20.04.

Thanks for the help.

I am including here below the all the commands I used to build it, I hope this could help other people with it.

sudo apt install build-essential
sudo apt install libcurl4-openssl-dev
sudo apt install libsqlite3-dev
sudo apt install pkg-config
sudo apt install git
sudo apt install curl
curl -fsS https://dlang.org/install.sh | bash -s ldc
source ~/dlang/ldc-1.24.0/activate
git clone https://github.com/abraunegg/onedrive.git
cd onedrive
git fetch origin pull/1188/head:pr1188
git checkout pr1188
./configure; make clean; make;
sudo make install
onedrive --verbose
onedrive --synchronize --verbose --dry-run
github-actions[bot] commented 3 years ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.