search

crazy-max / ftpgrab

Grab your files periodically from a remote FTP or SFTP server easily
https://crazymax.dev/ftpgrab/
MIT License
502 stars 77 forks source link

FTPGrab freezes on Docker for Mac 4.25.0 #315

Closed nothing2obvi closed 11 months ago

nothing2obvi commented 1 year ago

Support guidelines

I've found a bug and checked that ...

Description

Hi, I would greatly appreciate any help with this. Thanks!

I don't know if it has to do with me recently upgrading to Docker for Mac 4.25.0, but I've noticed that FTPGrab freezes all the time now. Sometimes it will just say that a download has finished and stop there, even though there are still more downloads to go. Other times it will just keep listing "Already running" over and over again for hours, despite the file being small (less than 500 MB) or even when there are no downloads available at all.

I also can't seem to restart the container in Portainer. I have to force quit Docker then open it again.

Expected behaviour

FTPGrab should finish all available downloads without freezing.

Actual behaviour

FTPGrab finishes one download but stops there. There are still more downloads to go but the logs don't change at all and no file is being downloaded. Other times it will just keep listing "Already running" over and over again for hours, despite the file being small (less than 500 MB) or if there are no downloads available at all.

Steps to reproduce

  1. Run FTPGrab with let's say 10 new files available for download.
  2. FTPGrab downloads 2 files.
  3. It stops there. The last log entry is the last downloaded file. It never goes back to saying, "Next run in 14 min and 59 sec."
  4. Note that no new file is being downloaded.

OR

  1. Run FTPGrab with let's say 10 new files available for download.
  2. FTPGrab downloads 2 files.
  3. Logs keep repeating "Already running" for hours even though the files are pretty small (under 500 MB).
  4. Note that no new file is being downloaded.

FTPGrab version

7.8.0

Docker info

Client:
 Version:    24.0.6
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2-desktop.5
    Path:     /Users/username/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.23.0-desktop.1
    Path:     /Users/username/.docker/cli-plugins/docker-compose
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /Users/username/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.20
    Path:     /Users/username/.docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.9
    Path:     /Users/username/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/username/.docker/cli-plugins/docker-sbom
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     /Users/username/.docker/cli-plugins/docker-scan
  scout: Docker Scout (Docker Inc.)
    Version:  v1.0.9
    Path:     /Users/username/.docker/cli-plugins/docker-scout

Server:
 Containers: 23
  Running: 15
  Paused: 0
  Stopped: 8
 Images: 49
 Server Version: 24.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 8165feabfdfe38c65b599c4993d227328c231fca
 runc version: v1.1.8-0-g82f18fe
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.4.16-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 8
 Total Memory: 5.793GiB
 Name: linuxkit-6680f319fdec
 ID: 6fcfe290-f2c1-47eb-a1be-ac8867b2baad
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

Docker Compose config

11/18/2023
02:36:55 PM
Sat, 18 Nov 2023 14:36:55 PST FTL Cannot create grabber error="Cannot open database: timeout"
11/18/2023
02:36:55 PM
Sat, 18 Nov 2023 14:36:55 PST INF Starting FTPGrab version=v7.8.0
11/18/2023
02:36:55 PM
Sat, 18 Nov 2023 14:36:55 PST INF Configuration loaded from file: /ftpgrab.yml
11/18/2023
02:36:55 PM
Sat, 18 Nov 2023 14:36:55 PST INF Configuration loaded from 2 environment variables
11/18/2023
02:36:55 PM
Sat, 18 Nov 2023 14:36:55 PST WRN No notifier available

Logs

11/20/2023
05:51:11 PM
Mon, 20 Nov 2023 17:51:11 PST INF Starting FTPGrab version=v7.8.0
11/20/2023
05:51:11 PM
Mon, 20 Nov 2023 17:51:11 PST INF Configuration loaded from file: /ftpgrab.yml
11/20/2023
05:51:11 PM
Mon, 20 Nov 2023 17:51:11 PST INF Configuration loaded from 2 environment variables
11/20/2023
05:51:11 PM
Mon, 20 Nov 2023 17:51:11 PST WRN No notifier available
11/20/2023
05:51:14 PM
Mon, 20 Nov 2023 17:51:14 PST INF 10 file(s) found
11/20/2023
05:52:40 PM
Mon, 20 Nov 2023 17:52:40 PST INF File successfully downloaded dest=/download/file1.mkv
11/20/2023
05:53:10 PM
Mon, 20 Nov 2023 17:53:10 PST INF File successfully downloaded dest=/download/file2.mkv"

Other logs:

11/26/2023
05:40:04 PM
Sun, 26 Nov 2023 17:40:04 PST INF Next run in 4 minutes 59 seconds (2023-11-26 17:45:00 -0800 PST)
11/26/2023
05:45:00 PM
Sun, 26 Nov 2023 17:45:00 PST WRN No notifier available
11/26/2023
05:45:04 PM
Sun, 26 Nov 2023 17:45:04 PST INF 36 file(s) found
11/26/2023
05:50:00 PM
Sun, 26 Nov 2023 17:50:00 PST WRN Already running
11/26/2023
05:55:00 PM
Sun, 26 Nov 2023 17:55:00 PST WRN Already running
11/26/2023
06:00:00 PM
Sun, 26 Nov 2023 18:00:00 PST WRN Already running
11/26/2023
06:05:00 PM
Sun, 26 Nov 2023 18:05:00 PST WRN Already running
11/26/2023
06:10:00 PM
Sun, 26 Nov 2023 18:10:00 PST WRN Already running
11/26/2023
06:15:00 PM
Sun, 26 Nov 2023 18:15:00 PST WRN Already running
11/26/2023
06:20:00 PM
Sun, 26 Nov 2023 18:20:00 PST WRN Already running
11/26/2023
06:25:00 PM
Sun, 26 Nov 2023 18:25:00 PST WRN Already running
11/26/2023
06:30:00 PM
Sun, 26 Nov 2023 18:30:00 PST WRN Already running
11/26/2023
06:35:00 PM
Sun, 26 Nov 2023 18:35:00 PST WRN Already running
11/26/2023
06:40:00 PM
Sun, 26 Nov 2023 18:40:00 PST WRN Already running
11/26/2023
06:45:00 PM
Sun, 26 Nov 2023 18:45:00 PST WRN Already running
11/26/2023
06:50:00 PM
Sun, 26 Nov 2023 18:50:00 PST WRN Already running
11/26/2023
06:55:00 PM
Sun, 26 Nov 2023 18:55:00 PST WRN Already running
11/26/2023
07:00:00 PM
Sun, 26 Nov 2023 19:00:00 PST WRN Already running
11/26/2023
07:05:00 PM
Sun, 26 Nov 2023 19:05:00 PST WRN Already running
11/26/2023
07:10:00 PM
Sun, 26 Nov 2023 19:10:00 PST WRN Already running
11/26/2023
07:15:00 PM
Sun, 26 Nov 2023 19:15:00 PST WRN Already running
11/26/2023
07:20:00 PM
Sun, 26 Nov 2023 19:20:00 PST WRN Already running
11/26/2023
07:25:00 PM
Sun, 26 Nov 2023 19:25:00 PST WRN Already running
11/26/2023
07:30:00 PM
Sun, 26 Nov 2023 19:30:00 PST WRN Already running

Additional info

No response

nothing2obvi commented 1 year ago

The issue persists on Docker for Mac 4.25.2 on M1 Mac Mini.

I am running another instance of FTPGrab 7.8.0 via Docker for Mac 4.20.1 on an older Mac Mini (Intel), and I have no issues with it.

nothing2obvi commented 12 months ago

The issue persists on Docker for Mac 4.26.0 on M1 Mac Mini.

nothing2obvi commented 11 months ago

Since switching Docker for Mac 4.26.0's file-sharing system from osxfs to VirtioFS, I haven't had the problem. I will keep this issue open for a while longer and close if FTPGrab continues to operate successfully.

crazy-max commented 11 months ago

@nothing2obvi Can you try with latest release?

nothing2obvi commented 11 months ago

@crazy-max As I stated previously, switching Docker from osxfs to VirtioFS seems to have fixed the issue. I upgraded to FTPGrab 7.9.0 a few days ago and no issues whatsoever! Will close this issue. Thank you!