search

astrada / google-drive-ocamlfuse

FUSE filesystem over Google Drive
https://astrada.github.io/google-drive-ocamlfuse/
MIT License
5.6k stars 354 forks source link

Huge memory leak since system upgrade 2 days ago #354

Closed jant90 closed 5 years ago

jant90 commented 7 years ago

I've been running google-drive-ocamlfuse on Manjaro (Arch based) without any issues for months as a systemd service now but since 2 days I experience a huge memory leak. I think this started after a system upgrade I did. I'm not sure but I think the actual leaking didn't start until I did a reboot.

As for the impact of the memory leak, after a fresh reboot google-drive-ocamlfuse is using 5GB of RAM in a matter of minutes, and it just keeps increasing until the RAM and SWAP are both full.

Is it possible that an updated dependency of google-drive-ocamlfuse is causing this memory leak? If so could anyone point out which one it might be?

I checked my pacman log and here is what updated that day:

[2017-09-21 16:19] [PACMAN] synchronizing package lists
[2017-09-21 16:19] [PACMAN] Running 'pacman --color auto -Sy'
[2017-09-21 16:19] [PACMAN] synchronizing package lists
[2017-09-21 16:19] [PACMAN] Running 'pacman --color auto -S -u'
[2017-09-21 16:19] [ALPM] transaction started
[2017-09-21 16:19] [ALPM] upgraded manjaro-system (20170406-5 -> 20170915-1)
[2017-09-21 16:19] [ALPM] transaction completed
[2017-09-21 16:19] [ALPM] running 'systemd-update.hook'...
[2017-09-21 16:19] [PACMAN] starting full system upgrade
[2017-09-21 16:20] [ALPM] transaction started
[2017-09-21 16:20] [ALPM] upgraded linux-api-headers (4.10.1-1 -> 4.12.7-1)
[2017-09-21 16:20] [ALPM] warning: /etc/locale.gen installed as /etc/locale.gen.pacnew
[2017-09-21 16:20] [ALPM] upgraded glibc (2.25-7 -> 2.26-4)
[2017-09-21 16:20] [ALPM-SCRIPTLET] Generating locales...
[2017-09-21 16:20] [ALPM-SCRIPTLET]   en_US.UTF-8... done
[2017-09-21 16:20] [ALPM-SCRIPTLET]   en_US.UTF-8... done
[2017-09-21 16:20] [ALPM-SCRIPTLET]   nl_NL.UTF-8... done
[2017-09-21 16:20] [ALPM-SCRIPTLET] Generation complete.
[2017-09-21 16:20] [ALPM] upgraded lib32-glibc (2.25-7 -> 2.26-2)
[2017-09-21 16:20] [ALPM] upgraded lib32-gcc-libs (7.1.1-4 -> 7.2.0-1)
[2017-09-21 16:20] [ALPM] upgraded gcc-libs-multilib (7.1.1-4 -> 7.2.0-1)
[2017-09-21 16:20] [ALPM] upgraded ncurses (6.0+20170527-1 -> 6.0+20170902-1)
[2017-09-21 16:20] [ALPM] upgraded pcre2 (10.23-1 -> 10.30-1)
[2017-09-21 16:20] [ALPM] upgraded android-tools (8.0.0_r4-1 -> 8.0.0_r4-2)
[2017-09-21 16:20] [ALPM] upgraded expat (2.2.3-1 -> 2.2.4-1)
[2017-09-21 16:20] [ALPM] upgraded coreutils (8.27-1 -> 8.28-1)
[2017-09-21 16:20] [ALPM] upgraded android-udev (20170902-1 -> 20170910-1)
[2017-09-21 16:20] [ALPM] upgraded binutils (2.28.0-4 -> 2.29.0-1)
[2017-09-21 16:20] [ALPM] upgraded bluez (5.46-1 -> 5.47-1)
[2017-09-21 16:20] [ALPM] upgraded bluez-libs (5.46-1 -> 5.47-1)
[2017-09-21 16:20] [ALPM] upgraded btrfs-progs (4.12.1-1 -> 4.13-1)
[2017-09-21 16:20] [ALPM] upgraded harfbuzz (1.5.0-1 -> 1.5.1-1)
[2017-09-21 16:20] [ALPM] upgraded fontconfig (2.12.4-1 -> 2.12.5-1)
[2017-09-21 16:20] [ALPM-SCRIPTLET] Rebuilding fontconfig cache... done.
[2017-09-21 16:20] [ALPM] upgraded cairo (1.14.10-1 -> 1.15.8-2)
[2017-09-21 16:20] [ALPM] upgraded python-setuptools (1:36.3.0-1 -> 1:36.4.0-1)
[2017-09-21 16:20] [ALPM] upgraded python-acme (0.17.0-1 -> 0.18.1-1)
[2017-09-21 16:20] [ALPM] upgraded p11-kit (0.23.7-1 -> 0.23.8-1)
[2017-09-21 16:20] [ALPM] upgraded certbot (0.17.0-1 -> 0.18.1-1)
[2017-09-21 16:20] [ALPM] upgraded libuv (1.14.0-1 -> 1.14.1-1)
[2017-09-21 16:20] [ALPM] installed rhash (1.3.5-2)
[2017-09-21 16:20] [ALPM] upgraded cmake (3.9.1-1 -> 3.9.2-2)
[2017-09-21 16:20] [ALPM] upgraded fakeroot (1.21-2 -> 1.22-1)
[2017-09-21 16:20] [ALPM] upgraded gsm (1.0.16-1 -> 1.0.17-1)
[2017-09-21 16:20] [ALPM] warning: /etc/drirc installed as /etc/drirc.pacnew
[2017-09-21 16:21] [ALPM] upgraded mesa (17.1.8-1.1 -> 17.2.0-2)
[2017-09-21 16:21] [ALPM] upgraded ffmpeg (1:3.3.3-2 -> 1:3.3.4-1)
[2017-09-21 16:21] [ALPM] upgraded gcc-multilib (7.1.1-4 -> 7.2.0-1)
[2017-09-21 16:21] [ALPM] upgraded gpm (1.20.7-7 -> 1.20.7-8)
[2017-09-21 16:21] [ALPM] upgraded gst-plugins-bad (1.12.2-4 -> 1.12.2+27+g2b8edd9c0-1)
[2017-09-21 16:21] [ALPM] upgraded gtk-update-icon-cache (3.22.19-2 -> 3.22.21-1)
[2017-09-21 16:21] [ALPM] upgraded librsvg (2:2.40.18-1 -> 2:2.40.18+20+g63ddae4-1)
[2017-09-21 16:21] [ALPM] upgraded gtk3 (3.22.19-2 -> 3.22.21-1)
[2017-09-21 16:21] [ALPM] upgraded gtk3-print-backends (3.22.19-2 -> 3.22.21-1)
[2017-09-21 16:21] [ALPM] upgraded harfbuzz-icu (1.5.0-1 -> 1.5.1-1)
[2017-09-21 16:21] [ALPM] upgraded libtool (2.4.6-8 -> 2.4.6+40+g6ca5e224-1)
[2017-09-21 16:21] [ALPM] upgraded imagemagick (6.9.9.11-1 -> 6.9.9.13-1)
[2017-09-21 16:21] [ALPM] upgraded inxi (2.3.28-1 -> 2.3.38-1)
[2017-09-21 16:21] [ALPM] upgraded iproute2 (4.12.0-2 -> 4.13.0-1)
[2017-09-21 16:21] [ALPM] upgraded libinput (1.8.1-1 -> 1.8.2-1)
[2017-09-21 16:21] [ALPM] upgraded tslib (1.12-1 -> 1.13-1)
[2017-09-21 16:21] [ALPM] upgraded kcoreaddons (5.37.0-1 -> 5.38.0-1)
[2017-09-21 16:21] [ALPM] upgraded kauth (5.37.0-1 -> 5.38.0-1)
[2017-09-21 16:21] [ALPM] upgraded kitemmodels (5.37.0-1 -> 5.38.0-1)
[2017-09-21 16:21] [ALPM] upgraded lib32-libelf (0.169-1 -> 0.169-2)
[2017-09-21 16:21] [ALPM] upgraded lib32-fontconfig (2.12.3-1 -> 2.12.5-1)
[2017-09-21 16:21] [ALPM-SCRIPTLET] Rebuilding 32-bit fontconfig cache... done.
[2017-09-21 16:21] [ALPM] upgraded lib32-libxau (1.0.8-1 -> 1.0.8-2)
[2017-09-21 16:21] [ALPM] upgraded lib32-libxinerama (1.1.3-1 -> 1.1.3-2)
[2017-09-21 16:21] [ALPM] upgraded lib32-ncurses (6.0+20170527-1 -> 6.0+20170527-2)
[2017-09-21 16:21] [ALPM] upgraded lib32-mesa (17.1.8-1.1 -> 17.2.0-1)
[2017-09-21 16:21] [ALPM] upgraded lib32-gtk3 (3.22.19-1 -> 3.22.21-0.1)
[2017-09-21 16:21] [ALPM] upgraded lib32-libsm (1.2.2-1 -> 1.2.2-2)
[2017-09-21 16:21] [ALPM] upgraded lib32-libpulse (10.0-1 -> 11.0-1)
[2017-09-21 16:21] [ALPM] upgraded libxslt (1.1.29+42+gac341cbd-1 -> 1.1.30-2)
[2017-09-21 16:21] [ALPM] upgraded linux-firmware (20170622.7d2c913-1 -> 20170907.a61ac5c-1)
[2017-09-21 16:21] [ALPM] upgraded mkinitcpio (23-1 -> 23-2)
[2017-09-21 16:21] [ALPM] upgraded linux49 (4.9.48-1 -> 4.9.50-1)
[2017-09-21 16:21] [ALPM-SCRIPTLET] >>> Updating module dependencies. Please wait ...
[2017-09-21 16:21] [ALPM] upgraded linux49-r8168 (8.044.02-20 -> 8.044.02-21)
[2017-09-21 16:21] [ALPM] upgraded manjaro-release (17.0.4-1 -> 17.0.5-1)
[2017-09-21 16:21] [ALPM] upgraded manjaro-settings-manager (0.5.3-4 -> 0.5.3-6)
[2017-09-21 16:21] [ALPM] upgraded manjaro-settings-manager-notifier (0.5.3-4 -> 0.5.3-6)
[2017-09-21 16:21] [ALPM] upgraded miniupnpc (2.0.20170509-1 -> 2.0.20170509-2)
[2017-09-21 16:21] [ALPM] upgraded ninja (1.8.1-1 -> 1.8.2-1)
[2017-09-21 16:21] [ALPM] upgraded openbsd-netcat (1.130_3-1 -> 1.178_3-1)
[2017-09-21 16:21] [ALPM] installed gcab (0.7+16+g4bc4be1-1)
[2017-09-21 16:21] [ALPM] installed libstemmer (0+337-2)
[2017-09-21 16:21] [ALPM] installed appstream-glib (0.6.13-1)
[2017-09-21 16:21] [ALPM] installed archlinux-appstream-data (20170909-1)
[2017-09-21 16:21] [ALPM] warning: /etc/pamac.conf installed as /etc/pamac.conf.pacnew
[2017-09-21 16:21] [ALPM] upgraded pamac (5.1.1-3 -> 6.0.1-1)
[2017-09-21 16:21] [ALPM] upgraded python-psutil (5.3.0-1 -> 5.3.1-1)
[2017-09-21 16:21] [ALPM] upgraded python2-setuptools (1:36.3.0-1 -> 1:36.4.0-1)
[2017-09-21 16:21] [ALPM] upgraded python2-acme (0.17.0-1 -> 0.18.1-1)
[2017-09-21 16:21] [ALPM] upgraded python2-psutil (5.3.0-1 -> 5.3.1-1)
[2017-09-21 16:21] [ALPM] upgraded python2-pyasn1 (0.3.3-1 -> 0.3.4-1)
[2017-09-21 16:21] [ALPM] upgraded ruby (2.4.1-3 -> 2.4.2-1)
[2017-09-21 16:21] [ALPM] upgraded wxgtk-common (3.0.3.1-8 -> 3.0.3.1-9)
[2017-09-21 16:21] [ALPM] upgraded wxgtk2 (3.0.3.1-8 -> 3.0.3.1-9)
[2017-09-21 16:21] [ALPM] upgraded wxgtk3 (3.0.3.1-8 -> 3.0.3.1-9)
[2017-09-21 16:21] [ALPM] upgraded wxsqlite3 (3.5.6-1 -> 3.5.7-1)
[2017-09-21 16:21] [ALPM] upgraded xfce4-xkb-plugin (0.8.0-1 -> 0.8.1-1)
[2017-09-21 16:21] [ALPM] transaction completed
[2017-09-21 16:21] [ALPM] running '90-linux49.hook'...
[2017-09-21 16:21] [ALPM-SCRIPTLET] ==> Building image from preset: /etc/mkinitcpio.d/linux49.preset: 'default'
[2017-09-21 16:21] [ALPM-SCRIPTLET]   -> -k /boot/vmlinuz-4.9-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-4.9-x86_64.img
[2017-09-21 16:21] [ALPM-SCRIPTLET] ==> Starting build: 4.9.50-1-MANJARO
[2017-09-21 16:21] [ALPM-SCRIPTLET]   -> Running build hook: [base]
[2017-09-21 16:21] [ALPM-SCRIPTLET]   -> Running build hook: [udev]
[2017-09-21 16:21] [ALPM-SCRIPTLET]   -> Running build hook: [autodetect]
[2017-09-21 16:21] [ALPM-SCRIPTLET]   -> Running build hook: [modconf]
[2017-09-21 16:21] [ALPM-SCRIPTLET]   -> Running build hook: [block]
[2017-09-21 16:21] [ALPM-SCRIPTLET]   -> Running build hook: [keyboard]
[2017-09-21 16:21] [ALPM-SCRIPTLET]   -> Running build hook: [keymap]
[2017-09-21 16:21] [ALPM-SCRIPTLET]   -> Running build hook: [resume]
[2017-09-21 16:21] [ALPM-SCRIPTLET]   -> Running build hook: [filesystems]
[2017-09-21 16:21] [ALPM-SCRIPTLET]   -> Running build hook: [fsck]
[2017-09-21 16:21] [ALPM-SCRIPTLET] ==> Generating module dependencies
[2017-09-21 16:21] [ALPM-SCRIPTLET] ==> Creating gzip-compressed initcpio image: /boot/initramfs-4.9-x86_64.img
[2017-09-21 16:21] [ALPM-SCRIPTLET] ==> Image generation successful
[2017-09-21 16:21] [ALPM-SCRIPTLET] ==> Building image from preset: /etc/mkinitcpio.d/linux49.preset: 'fallback'
[2017-09-21 16:21] [ALPM-SCRIPTLET]   -> -k /boot/vmlinuz-4.9-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-4.9-x86_64-fallback.img -S autodetect
[2017-09-21 16:21] [ALPM-SCRIPTLET] ==> Starting build: 4.9.50-1-MANJARO
[2017-09-21 16:21] [ALPM-SCRIPTLET]   -> Running build hook: [base]
[2017-09-21 16:21] [ALPM-SCRIPTLET]   -> Running build hook: [udev]
[2017-09-21 16:21] [ALPM-SCRIPTLET]   -> Running build hook: [modconf]
[2017-09-21 16:21] [ALPM-SCRIPTLET]   -> Running build hook: [block]
[2017-09-21 16:21] [ALPM] running '99-grub.hook'...
[2017-09-21 16:21] [ALPM-SCRIPTLET] Generating grub configuration file ...
[2017-09-21 16:21] [ALPM-SCRIPTLET] Found background: /usr/share/grub/background.png
[2017-09-21 16:21] [ALPM-SCRIPTLET] Found Intel Microcode image
[2017-09-21 16:21] [ALPM-SCRIPTLET] Found linux image: /boot/vmlinuz-4.9-x86_64
[2017-09-21 16:21] [ALPM-SCRIPTLET] Found initrd image: /boot/initramfs-4.9-x86_64.img
[2017-09-21 16:21] [ALPM-SCRIPTLET] Found initrd fallback image: /boot/initramfs-4.9-x86_64-fallback.img
[2017-09-21 16:21] [ALPM-SCRIPTLET] Found memtest86+ image: /boot/memtest86+/memtest.bin
[2017-09-21 16:21] [ALPM-SCRIPTLET] done
[2017-09-21 16:21] [ALPM] running 'detect-old-perl-modules.hook'...
[2017-09-21 16:21] [ALPM] running 'gdk-pixbuf-query-loaders.hook'...
[2017-09-21 16:21] [ALPM] running 'glib-compile-schemas.hook'...
[2017-09-21 16:21] [ALPM] running 'gtk-query-immodules-3.0-32.hook'...
[2017-09-21 16:21] [ALPM] running 'gtk-query-immodules-3.0.hook'...
[2017-09-21 16:21] [ALPM] running 'gtk-update-icon-cache.hook'...
[2017-09-21 16:21] [ALPM] running 'systemd-hwdb.hook'...
[2017-09-21 16:21] [ALPM] running 'systemd-tmpfiles.hook'...
[2017-09-21 16:21] [ALPM] running 'systemd-update.hook'...
[2017-09-21 16:21] [ALPM] running 'texinfo-install.hook'...
[2017-09-21 16:22] [ALPM] running 'update-desktop-database.hook'...
[2017-09-21 16:22] [ALPM] running 'update-mime-database.hook'...
[2017-09-21 16:25] [PACMAN] Running 'pacman --color auto -U /tmp/yaourt-tmp-jan/PKGDEST.ATO/emby-server-3.2.31.0-1-x86_64.pkg.tar.xz'
[2017-09-21 16:26] [ALPM] transaction started
[2017-09-21 16:26] [ALPM] upgraded emby-server (3.2.30.0-1 -> 3.2.31.0-1)
[2017-09-21 16:26] [ALPM-SCRIPTLET] 
[2017-09-21 16:26] [ALPM-SCRIPTLET] Emby is not compatible with the 10-bit versionof libx264. Please stick to the
[2017-09-21 16:26] [ALPM-SCRIPTLET] 8-bit version if you plan on using emby's transcoding features.
[2017-09-21 16:26] [ALPM-SCRIPTLET] 
[2017-09-21 16:26] [ALPM] transaction completed
[2017-09-21 16:26] [ALPM] running 'systemd-update.hook'...
[2017-09-21 16:31] [PACMAN] Running 'pacman --color auto -U /tmp/yaourt-tmp-jan/PKGDEST.Xe9/plex-media-server-1.9.1.4272-1-x86_64.pkg.tar.xz'
[2017-09-21 16:31] [ALPM] transaction started
[2017-09-21 16:31] [ALPM] upgraded plex-media-server (1.8.4.4249-1 -> 1.9.1.4272-1)
[2017-09-21 16:31] [ALPM] transaction completed
[2017-09-21 16:31] [ALPM] running 'systemd-sysusers.hook'...
[2017-09-21 16:31] [ALPM] running 'systemd-update.hook'...
[2017-09-21 16:33] [PACMAN] Running 'pacman -Rnsv ninja'
[2017-09-21 16:33] [ALPM] transaction started
[2017-09-21 16:33] [ALPM] removed ninja (1.8.2-1)
[2017-09-21 16:33] [ALPM] transaction completed
[2017-09-21 16:33] [ALPM] running 'systemd-update.hook'...
[2017-09-21 16:34] [PACMAN] Running 'pacman --color auto -Sy'
[2017-09-21 16:34] [PACMAN] synchronizing package lists
[2017-09-22 12:51] [PACMAN] Running 'pacman --color auto -Sy'
[2017-09-22 12:51] [PACMAN] synchronizing package lists
[2017-09-22 13:03] [PACMAN] Running 'pacman --color auto -U /tmp/yaourt-tmp-jan/PKGDEST.3vb/emby-server-3.2.32.0-1-x86_64.pkg.tar.xz'
[2017-09-22 13:03] [ALPM] transaction started
[2017-09-22 13:03] [ALPM] upgraded emby-server (3.2.31.0-1 -> 3.2.32.0-1)
[2017-09-22 13:03] [ALPM-SCRIPTLET] 
[2017-09-22 13:03] [ALPM-SCRIPTLET] Emby is not compatible with the 10-bit versionof libx264. Please stick to the
[2017-09-22 13:03] [ALPM-SCRIPTLET] 8-bit version if you plan on using emby's transcoding features.
[2017-09-22 13:03] [ALPM-SCRIPTLET] 
[2017-09-22 13:03] [ALPM] transaction completed
[2017-09-22 13:03] [ALPM] running 'systemd-update.hook'...
[2017-09-22 13:12] [PACMAN] Running 'pacman --color auto -U /tmp/yaourt-tmp-jan/PKGDEST.Cqp/google-chrome-61.0.3163.100-1-x86_64.pkg.tar.xz'
[2017-09-22 13:12] [ALPM] transaction started
[2017-09-22 13:12] [ALPM] upgraded google-chrome (61.0.3163.91-1 -> 61.0.3163.100-1)
[2017-09-22 13:12] [ALPM-SCRIPTLET] (B==>(B NOTE:(B Custom flags should be put directly in: ~/.config/chrome-flags.conf(B
[2017-09-22 13:12] [ALPM-SCRIPTLET] (B==>(B NOTE:(B The launcher is called: 'google-chrome-stable'(B
[2017-09-22 13:12] [ALPM] transaction completed
[2017-09-22 13:12] [ALPM] running 'gtk-update-icon-cache.hook'...
[2017-09-22 13:12] [ALPM] running 'systemd-update.hook'...
[2017-09-22 13:12] [ALPM] running 'update-desktop-database.hook'...
[2017-09-23 12:20] [PACMAN] Running 'pacman --color auto -Sy'
[2017-09-23 12:20] [PACMAN] synchronizing package lists
[2017-09-23 16:22] [PACMAN] Running 'pacman --color auto -Sy'
[2017-09-23 16:22] [PACMAN] synchronizing package lists

And here is my config file:

apps_script_icon=
async_upload=true
cache_directory=
client_id=xxxxxxxxxx
client_secret=xxxxxxxxxx
connect_timeout_ms=5000
curl_debug_off=false
data_directory=
delete_forever_in_trash_folder=false
docs_file_extension=true
document_format=odt
document_icon=
download_docs=true
drawing_format=png
drawing_icon=
form_format=zip
form_icon=
fusion_table_format=desktop
fusion_table_icon=
keep_duplicates=false
large_file_read_only=false
large_file_threshold_mb=16
log_directory=
lost_and_found=false
low_speed_limit=0
low_speed_time=0
map_format=desktop
map_icon=
max_cache_size_mb=512
max_download_speed=0
max_memory_cache_size=78643200
max_retries=8
max_upload_chunk_size=1099511627776
max_upload_speed=0
memory_buffer_size=7864320
metadata_cache_time=180000
presentation_format=pdf
presentation_icon=
read_ahead_buffers=1
read_only=false
root_folder=
shared_with_me=false
spreadsheet_format=ods
spreadsheet_icon=
sqlite3_busy_timeout=5000
stream_large_files=true
umask=0o022
verification_code=
kurdtpage commented 6 years ago

I am experiencing the same issue, using version 0.6.21 on Ubuntu 17.10 (artful) with 5GB RAM. The issue is worse when using a program that accesses multiple files at the same time, like an IDE. Eventually the RAM gets filled up and the system halts, regardless of CPU usage.

astrada commented 5 years ago

Should be fixed in the latest versions.