search

xybu / onedrive-d-old

Microsoft OneDrive client on Linux.
http://xybu.me/projects/onedrive-d/
GNU Lesser General Public License v3.0
820 stars 143 forks source link

High I/O activity when idle #155

Open jameswalmsley opened 9 years ago

jameswalmsley commented 9 years ago

I noticed that one drive-d seems to constantly write 256kb/sec to disk between sync workers. Any ideas where this comes from?

xybu commented 9 years ago

Can you check what files are open / being written in OneDrive repo or ~/.onedrive using lsof command or something?

Or use debug mode and see what the program is doing?

Sent from my Windows Phone

From: James Walmsleymailto:notifications@github.com Sent: ‎4/‎15/‎2015 4:49 PM To: xybu/onedrive-dmailto:onedrive-d@noreply.github.com Subject: [onedrive-d] High I/O activity when idle (#155)

I noticed that one drive-d seems to constantly write 256kb/sec to disk between sync workers. Any ideas where this comes from?

Reply to this email directly or view it on GitHub: https://github.com/xybu/onedrive-d/issues/155

marty90 commented 9 years ago

I've this problem too. I see a lot of file descriptors pointing to ~/.onedrive/entries.db

marty90 commented 9 years ago

Update: Using debug feature i see a continuous flow of messages, like these ones. [2015-05-03 12:18:46,990] DEBUG: worker0: got task: sy on "/home/homme/OneDrive/..." [2015-05-03 12:18:48,187] DEBUG: worker1: added task "sy" "/home/homme/OneDrive/...". Is it normal ?

gluque commented 9 years ago

Same problem. Each time onedrive-d is started (in debug mode) displays a lot of messages like these:

I am using Kubuntu 15.04 with Python 3.4.

ewann commented 9 years ago

I experienced similar symptoms on Ubuntu 15.04. In my case I found:

onedrive-d appeared to constantly loop / scan through files I had listed to sync. I found 2 possible ways to address that:

  1. Reduce the size / number of folder structures that onedrive-d is syncing: Adding entries to ~/.onedrive/ignore_v2.ini to reduce the sync surface
  2. Decrease the frequency of the "DEEP_SCAN_INTERVAL" occurs - the default for this in config_v2.json is 60 (seconds) - personally I find this too aggressive, and chose to accept a value of an hour (3600) or something similar.

Why deep scan at all? On the primary system where I use onedrive-d, inotify is not installed, and to date I haven't successfully made it work. A symptom of missing inotify can be seen in the debug log snippet below:

developer@4bb660ad782e:~$ onedrive-d start --debug Loading configuration ... OK [2015-07-21 01:14:06,688] DEBUG: MainThread: running in debug mode. Starting onedrive-d ... OK [2015-07-21 01:14:06,834] DEBUG: thread_mgr: started. [2015-07-21 01:14:06,932] DEBUG: MainThread: daemon started. [2015-07-21 01:14:09,192] CRITICAL: inotify: inotifywait was not found. Skip module.

Without this, we appear to be reliant on the DEEP_SCAN_INTERVAL to pickup changes on the local fs.

xybu commented 9 years ago

Thanks for all your help with this.

(Due to the contract with my employer I can't work on this project until I finish my job and come back to school in late August. I'll add you as developer for this repo so that you can merge the pulls yourself.)

Sent from my iPhone

On Jul 20, 2015, at 8:26 PM, Ewan notifications@github.com wrote:

I experienced similar symptoms on Ubuntu 15.04. In my case I found:

onedrive-d appeared to constantly loop / scan through files I had listed to sync. I found 2 possible ways to address that:

Reduce the size / number of folder structures that onedrive-d is syncing: Adding entries to ~/.onedrive/ignore_v2.ini to reduce the sync surface

Decrease the frequency of the "DEEP_SCAN_INTERVAL" occurs - the default for this in config_v2.json is 60 (seconds) - personally I find this too aggressive, and chose to accept a value of an hour (3600) or something similar.

Why deep scan at all? On the primary system where I use onedrive-d, inotify is not installed, and to date I haven't successfully made it work. A symptom of missing inotify can be seen in the debug log snippet below:

developer@4bb660a:~$ onedrive-d start --debug Loading configuration ... OK [2015-07-21 01:14:06,688] DEBUG: MainThread: running in debug mode. Starting onedrive-d ... OK [2015-07-21 01:14:06,834] DEBUG: thread_mgr: started. [2015-07-21 01:14:06,932] DEBUG: MainThread: daemon started. [2015-07-21 01:14:09,192] CRITICAL: inotify: inotifywait was not found. Skip module.

Without this, we appear to be reliant on the DEEP_SCAN_INTERVAL to pickup changes on the local fs.

— Reply to this email directly or view it on GitHub.

qqlovekerry commented 8 years ago

Dear ewann, I faced the same problem here. My os is Ubuntu 14.04LTS. But it seems I have inotify. The message is: DEBUG: inotify: starting inotifywait process. If I use iotop to monitor the disk write/read, I found a very hard write to my hard disk. So, I am wondering is there any way to fix this problem. I am a little worried about my disk. I understand how to change "DEEP_SCAN_INTERVAL" in config_v2.json. But I do not quite follow the first point in your last comment about reducing the size / number of folder structures. Could you please give me some suggestions? Many thanks in advance!