search

ipsingh06 / seedsync

Sync your seedbox. Fast. And more.
https://ipsingh06.github.io/seedsync/
Apache License 2.0
313 stars 43 forks source link

"Cannot Create Directory" on remote server. #129

Closed UpperCenter closed 1 year ago

UpperCenter commented 1 year ago

Hi All,

I have an ongoing issue with SeedSync on my Unraid server. It seems like SeedSync is trying to create a /home/seedsync folder on my remote seedbox, but as it's a shared seedbox I don't have any rights to create new directories in /home.

Snippet from logs:

buffer (last 100 chars): b'mkdir: cannot create directory \xe2\x80\x98/home/seedsync\xe2\x80\x99: Permission denied\r\n'

My config:

[General]
debug = False
verbose = False

[Lftp]
remote_address = emerald.usbx.me
remote_username = user
remote_password = password
remote_port = 22
remote_path = /home/user/Downloads/completed
local_path = /downloads
remote_path_to_scan_script = /home/user/bin/tmp-seedsync
use_ssh_key = False
num_max_parallel_downloads = 5
num_max_parallel_files_per_download = 4
num_max_connections_per_root_file = 4
num_max_connections_per_dir_file = 4
num_max_total_connections = 16
use_temp_file = True

[Controller]
interval_ms_remote_scan = 30000
interval_ms_local_scan = 10000
interval_ms_downloading_scan = 1000
extract_path = /tmp
use_local_path_as_extract_path = True

[Web]
port = 8800

[AutoQueue]
enabled = True
patterns_only = True
auto_extract = True

Args:

args: ['/usr/local/sbin/ssh', '-p', '22', '-o', 'StrictHostKeyChecking=no', '-o', 'UserKnownHostsFile=/dev/null', '-o', 'LogLevel=error', '-o', 'PubkeyAuthentication=no', 'user@emerald.usbx.me', "'/home/user/bin/tmp-seedsync/scanfs' '/home/user/Downloads/completed'"]

I can confirm that the scanfs folder exists on my remote seedbox. Running the above SSH command in my terminal: image

Thanks.

quadcom commented 1 year ago

Check the owner and permissions of the share your local download target is set to.

You can use the fix permissions tool to correct.

Also, you'll need to add the UUID and GUID to the template.

UpperCenter commented 1 year ago

So the issue is related to my local server, rather than the remote one? I'm passing through the PUID and PGID that my containers usually run as. I'm not sure if the issue is due to migrating from Syncthing, which was previously using these folder paths. Full Docker run:

docker run
  -d
  --name='seedsync'
  --net='proxy-net'
  -e TZ="Europe/London"
  -e HOST_OS="Unraid"
  -e HOST_HOSTNAME="io"
  -e HOST_CONTAINERNAME="seedsync"
  -e 'PUID'='99'
  -e 'PGID'='100'
  -e 'UMASK'='000'
  -l net.unraid.docker.managed=dockerman
  -l net.unraid.docker.webui='http://[IP]:[PORT:8800]/'
  -l net.unraid.docker.icon='https://raw.githubusercontent.com/selfhosters/unRAID-CA-templates/master/templates/img/seedsync.png'
  -p '8800:8800/tcp'
  -v '/mnt/user/appdata/seedsync':'/config':'rw'
  -v '/mnt/user/data/torrents/':'/downloads':'rw'
  -u 99:100 'ipsingh06/seedsync'
quadcom commented 1 year ago

Yeah, the crate DIR problem is on the local side as the scanfs is in the server. That's the only thing that gets with in the server side. Everything else is a pull.

I may be wrong here but I think you can drop the last line in the run command

-u 99:100 'ipsingh06/seedsync'

UUID 99 on unraid is root. Declaring a different username for the UUID might be causing the problem.

I moved from a debian VM to using the Unraid Seedsync template that's in the apps tab. Seemed to work fine.

But yeah, check the target permissions on the mapped share

On Thu, Jul 27, 2023, 11:17 a.m. Tom Hindby @.***> wrote:

So the issue is related to my local server, rather than the remote one? I'm passing through the PUID and PGID that my containers usually run as. I'm not sure if the issue is due to migrating from Syncthing, which was previously using these folder paths. Full Docker run:

docker run -d --name='seedsync' --net='proxy-net' -e TZ="Europe/London" -e HOST_OS="Unraid" -e HOST_HOSTNAME="io" -e HOST_CONTAINERNAME="seedsync" -e 'PUID'='99' -e 'PGID'='100' -e 'UMASK'='000' -l net.unraid.docker.managed=dockerman -l net.unraid.docker.webui='http://[IP]:[PORT:8800]/' -l net.unraid.docker.icon='https://raw.githubusercontent.com/selfhosters/unRAID-CA-templates/master/templates/img/seedsync.png' -p '8800:8800/tcp' -v '/mnt/user/appdata/seedsync':'/config':'rw' -v '/mnt/user/data/torrents/':'/downloads':'rw' -u 99:100 'ipsingh06/seedsync'

— Reply to this email directly, view it on GitHub https://github.com/ipsingh06/seedsync/issues/129#issuecomment-1653832551, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABWB625IKYKFSH3EUUZOWG3XSKA7TANCNFSM6AAAAAA2Z2HDME . You are receiving this because you commented.Message ID: @.***>

UpperCenter commented 1 year ago

If I don't add -u 99:100, the container doesn't run and crashes with "Permission Denied" on my local download folder: PermissionError: [Errno 13] Permission denied: '/downloads/Films'

quadcom commented 1 year ago

And what are the Unraid owner/permissions on the share where you are downloading to?

image

I did see that you are passing the UUID:GUID to SS incorrectly. It must be a var named "user" and be formatted as such "99:100"

image

While I was figuring this out as well, I found that adding the extra parameters { --user="99:100" } got things working as I had expected it to. image

UpperCenter commented 1 year ago

I think my permissions are the same as yours... image

quadcom commented 1 year ago

what about the permissions/ownership of "data", the parent folder?

quadcom commented 1 year ago

Your permissions are different it seems.

image

quadcom commented 1 year ago

it appears yours are owned by "home"

Run the Docker Safe New Perms tool to fix the perms/owner on the shares {skipping appdata obvisouly}

quadcom commented 1 year ago

I also created a script to fix the perms after transfers were completed. You can add it to User Scripts, and it run it in the background on startup.

Edit the 'monitord_dirs' variable at the start to target your SS incoming folder. It will ignore the SS temp files, so make sure you have that turned on in the SS config.

#!/bin/bash

# Monitored directories
monitored_dirs=("/mnt/user/dvr/")

# Excluded extensions
excluded_exts=("lftp" "lftp-pget-status")

# Function to check ownership and permissions of files and directories
check_obj() {
    if [[ -e "$1" ]]; then
        # File or directory exists
        # Exclude files with excluded extensions
        for ext in "${excluded_exts[@]}"; do
            if [[ "$1" == *".$ext" ]]; then
                return
            fi
        done

        # Check ownership
        owner=$(stat -c '%U' "$1")
        if [[ "$owner" != "nobody" ]]; then
            # Set ownership to nobody
            chown nobody "$1"
        fi

        # Check permissions
        if [[ -d "$1" ]]; then
            # Directory
            perms=$(stat -c '%a' "$1")
            if [[ "$perms" != "777" ]]; then
                # Set permissions to 'drwxrwxrwx'
                chmod 777 "$1"
            fi
        else
            # File
            perms=$(stat -c '%a' "$1")
            if [[ "$perms" != "666" ]]; then
                # Set permissions to '-rw-rw-rw-'
                chmod 666 "$1"
            fi
        fi
    fi
}

# Watch for changes to files and directories
inotifywait -r -m -e create,modify,move,attrib "${monitored_dirs[@]}" |
while read path action file; do
    # Check ownership and permissions of changed object
    check_obj "$path/$file"
done
UpperCenter commented 1 year ago

Thanks for that. I ran the Safe New Permissions tool and restarted the container, it did change the permissions on the folders to nobody:users but that didn't fix the SS container error.

I've updated the paths and ran the script. Doesn't seem like it's doing anything? image

Thanks

UpperCenter commented 1 year ago

Permissions now, after a reboot and running both Safe New Perms and the bash script above on array startup: image

Slightly updated docker run:

docker run
  -d
  --name='seedsync'
  --net='proxy-net'
  -e TZ="Europe/London"
  -e HOST_OS="Unraid"
  -e HOST_HOSTNAME="io"
  -e HOST_CONTAINERNAME="seedsync"
  -e 'UMASK'='000'
  -l net.unraid.docker.managed=dockerman
  -l net.unraid.docker.webui='http://[IP]:[PORT:8800]/'
  -l net.unraid.docker.icon='https://raw.githubusercontent.com/selfhosters/unRAID-CA-templates/master/templates/img/seedsync.png'
  -p '8800:8800/tcp'
  -v '/mnt/user/appdata/seedsync':'/config':'rw'
  -v '/mnt/user/data/torrents/':'/downloads':'rw'
  --user=99:100 'ipsingh06/seedsync'
UpperCenter commented 1 year ago

Tried running with --privileged=true which didn't make any difference either.

quadcom commented 1 year ago

Please post a Seedsync log from startup

On Sun, Jul 30, 2023, 3:31 p.m. Tom Hindby @.***> wrote:

Tried running with --privileged=true which didn't make any difference either.

— Reply to this email directly, view it on GitHub https://github.com/ipsingh06/seedsync/issues/129#issuecomment-1657248202, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABWB62YMU3AW4FBLVEAR663XS2ZA7ANCNFSM6AAAAAA2Z2HDME . You are receiving this because you commented.Message ID: @.***>

UpperCenter commented 1 year ago

Sure:

2023-08-01T17:48:23.210133668Z 2023-08-01 18:48:23,210 - INFO - seedsync (MainProcess/MainThread) - Debug mode is enabled.
2023-08-01T17:48:23.210212242Z 2023-08-01 18:48:23,210 - DEBUG - seedsync (MainProcess/MainThread) - Config:
2023-08-01T17:48:23.210393502Z 2023-08-01 18:48:23,210 - DEBUG - seedsync (MainProcess/MainThread) -   General.debug: True
2023-08-01T17:48:23.210422034Z 2023-08-01 18:48:23,210 - DEBUG - seedsync (MainProcess/MainThread) -   General.verbose: False
2023-08-01T17:48:23.210450837Z 2023-08-01 18:48:23,210 - DEBUG - seedsync (MainProcess/MainThread) -   Lftp.remote_address: host.example.com
2023-08-01T17:48:23.210482445Z 2023-08-01 18:48:23,210 - DEBUG - seedsync (MainProcess/MainThread) -   Lftp.remote_username: username
2023-08-01T17:48:23.210507052Z 2023-08-01 18:48:23,210 - DEBUG - seedsync (MainProcess/MainThread) -   Lftp.remote_password: password
2023-08-01T17:48:23.210555689Z 2023-08-01 18:48:23,210 - DEBUG - seedsync (MainProcess/MainThread) -   Lftp.remote_port: 22
2023-08-01T17:48:23.210620790Z 2023-08-01 18:48:23,210 - DEBUG - seedsync (MainProcess/MainThread) -   Lftp.remote_path: /home/username/Downloads/completed
2023-08-01T17:48:23.210755898Z 2023-08-01 18:48:23,210 - DEBUG - seedsync (MainProcess/MainThread) -   Lftp.local_path: /downloads
2023-08-01T17:48:23.210787643Z 2023-08-01 18:48:23,210 - DEBUG - seedsync (MainProcess/MainThread) -   Lftp.remote_path_to_scan_script: /home/username/bin/tmp-seedsync
2023-08-01T17:48:23.210808285Z 2023-08-01 18:48:23,210 - DEBUG - seedsync (MainProcess/MainThread) -   Lftp.use_ssh_key: False
2023-08-01T17:48:23.210847603Z 2023-08-01 18:48:23,210 - DEBUG - seedsync (MainProcess/MainThread) -   Lftp.num_max_parallel_downloads: 5
2023-08-01T17:48:23.210866656Z 2023-08-01 18:48:23,210 - DEBUG - seedsync (MainProcess/MainThread) -   Lftp.num_max_parallel_files_per_download: 4
2023-08-01T17:48:23.210915798Z 2023-08-01 18:48:23,210 - DEBUG - seedsync (MainProcess/MainThread) -   Lftp.num_max_connections_per_root_file: 4
2023-08-01T17:48:23.210938656Z 2023-08-01 18:48:23,210 - DEBUG - seedsync (MainProcess/MainThread) -   Lftp.num_max_connections_per_dir_file: 4
2023-08-01T17:48:23.210972572Z 2023-08-01 18:48:23,210 - DEBUG - seedsync (MainProcess/MainThread) -   Lftp.num_max_total_connections: 16
2023-08-01T17:48:23.211001579Z 2023-08-01 18:48:23,210 - DEBUG - seedsync (MainProcess/MainThread) -   Lftp.use_temp_file: True
2023-08-01T17:48:23.211021826Z 2023-08-01 18:48:23,210 - DEBUG - seedsync (MainProcess/MainThread) -   Controller.interval_ms_remote_scan: 30000
2023-08-01T17:48:23.211053877Z 2023-08-01 18:48:23,211 - DEBUG - seedsync (MainProcess/MainThread) -   Controller.interval_ms_local_scan: 10000
2023-08-01T17:48:23.211088307Z 2023-08-01 18:48:23,211 - DEBUG - seedsync (MainProcess/MainThread) -   Controller.interval_ms_downloading_scan: 1000
2023-08-01T17:48:23.211110539Z 2023-08-01 18:48:23,211 - DEBUG - seedsync (MainProcess/MainThread) -   Controller.extract_path: /tmp
2023-08-01T17:48:23.211158115Z 2023-08-01 18:48:23,211 - DEBUG - seedsync (MainProcess/MainThread) -   Controller.use_local_path_as_extract_path: True
2023-08-01T17:48:23.211185610Z 2023-08-01 18:48:23,211 - DEBUG - seedsync (MainProcess/MainThread) -   Web.port: 8800
2023-08-01T17:48:23.211210570Z 2023-08-01 18:48:23,211 - DEBUG - seedsync (MainProcess/MainThread) -   AutoQueue.enabled: True
2023-08-01T17:48:23.211285642Z 2023-08-01 18:48:23,211 - DEBUG - seedsync (MainProcess/MainThread) -   AutoQueue.patterns_only: True
2023-08-01T17:48:23.211294416Z 2023-08-01 18:48:23,211 - DEBUG - seedsync (MainProcess/MainThread) -   AutoQueue.auto_extract: True
2023-08-01T17:48:23.211309750Z 2023-08-01 18:48:23,211 - DEBUG - seedsync (MainProcess/MainThread) - Args:
2023-08-01T17:48:23.211346832Z 2023-08-01 18:48:23,211 - DEBUG - seedsync (MainProcess/MainThread) -   local_path_to_scanfs: /app/scanfs
2023-08-01T17:48:23.211370533Z 2023-08-01 18:48:23,211 - DEBUG - seedsync (MainProcess/MainThread) -   html_path: /app/html
2023-08-01T17:48:23.211393577Z 2023-08-01 18:48:23,211 - DEBUG - seedsync (MainProcess/MainThread) -   debug: True
2023-08-01T17:48:23.211425315Z 2023-08-01 18:48:23,211 - DEBUG - seedsync (MainProcess/MainThread) -   exit: False
2023-08-01T17:48:23.212453427Z 2023-08-01 18:48:23,212 - INFO - seedsync (MainProcess/MainThread) - Starting SeedSync
2023-08-01T17:48:23.227679719Z 2023-08-01 18:48:23,227 - INFO - seedsync (MainProcess/MainThread) - Platform: x86_64
2023-08-01T17:48:23.846978944Z 2023-08-01 18:48:23,846 - DEBUG - seedsync.Controller.Model (MainProcess/MainThread) - LftpModel: Adding a listener
2023-08-01T17:48:23.847096064Z 2023-08-01 18:48:23,847 - DEBUG - seedsync.AutoQueue (MainProcess/MainThread) - Auto-Queue Patterns:
2023-08-01T17:48:23.847308630Z 2023-08-01 18:48:23,847 - INFO - seedsync.WebApp (MainProcess/MainThread) - Html path set to: /app/html
2023-08-01T17:48:23.855237162Z 2023-08-01 18:48:23,855 - DEBUG - seedsync.ControllerJob (MainProcess/ControllerJob) - Thread ControllerJob started
2023-08-01T17:48:23.855363815Z 2023-08-01 18:48:23,855 - DEBUG - seedsync.ControllerJob (MainProcess/ControllerJob) - Calling setup for ControllerJob
2023-08-01T17:48:23.855408077Z 2023-08-01 18:48:23,855 - DEBUG - seedsync.Controller (MainProcess/ControllerJob) - Starting controller
2023-08-01T17:48:23.855719349Z 2023-08-01 18:48:23,855 - DEBUG - seedsync.WebAppJob (MainProcess/WebAppJob) - Thread WebAppJob started
2023-08-01T17:48:23.855943605Z 2023-08-01 18:48:23,855 - DEBUG - seedsync.WebAppJob (MainProcess/WebAppJob) - Calling setup for WebAppJob
2023-08-01T17:48:23.856245287Z 2023-08-01 18:48:23,856 - DEBUG - web_access (MainProcess/Thread-3) - Starting web server
2023-08-01T17:48:23.856370208Z 2023-08-01 18:48:23,856 - DEBUG - seedsync.WebAppJob (MainProcess/WebAppJob) - Finished setup for WebAppJob
2023-08-01T17:48:23.881840929Z 2023-08-01 18:48:23,881 - DEBUG - seedsync.Controller.MPLogger (MainProcess/MPLoggerListener) - Started listener thread
2023-08-01T17:48:23.887055948Z 2023-08-01 18:48:23,883 - DEBUG - seedsync.ControllerJob (MainProcess/ControllerJob) - Finished setup for ControllerJob
2023-08-01T17:48:23.887422932Z 2023-08-01 18:48:23,867 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Started process
2023-08-01T17:48:23.888502307Z 2023-08-01 18:48:23,867 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Running a scan
2023-08-01T17:48:23.889015545Z 2023-08-01 18:48:23,864 - DEBUG - ActiveScanner (ActiveScanner/ActiveScanner) - Started process
2023-08-01T17:48:23.890215477Z 2023-08-01 18:48:23,870 - DEBUG - RemoteScanner (RemoteScanner/RemoteScanner) - Started process
2023-08-01T17:48:23.890720553Z 2023-08-01 18:48:23,871 - DEBUG - RemoteScanner (RemoteScanner/RemoteScanner) - Running a scan
2023-08-01T17:48:23.892458053Z 2023-08-01 18:48:23,882 - DEBUG - ExtractProcess (ExtractProcess/ExtractProcess) - Started process
2023-08-01T17:48:23.892990223Z 2023-08-01 18:48:23,884 - DEBUG - ExtractDispatch (ExtractProcess/ExtractWorker) - Started worker thread
2023-08-01T17:48:23.993966627Z 2023-08-01 18:48:23,933 - DEBUG - RemoteScanner.RemoteScanner (RemoteScanner/RemoteScanner) - Local scanfs md5sum = e9b7600451a779ef32c2dc00f018d204
2023-08-01T17:48:23.993997357Z 2023-08-01 18:48:23,934 - DEBUG - RemoteScanner.RemoteScanner.Sshcp (RemoteScanner/RemoteScanner) - Command: ssh -p 22 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=error -o PubkeyAuthentication=no username@host.example.com "md5sum /home/username/bin/tmp-seedsync/scanfs | awk '{print $1}' || echo"
2023-08-01T17:48:24.795739890Z 2023-08-01 18:48:24,790 - DEBUG - RemoteScanner.RemoteScanner.Sshcp (RemoteScanner/RemoteScanner) - Return code: 0
2023-08-01T17:48:24.795765415Z 2023-08-01 18:48:24,790 - DEBUG - RemoteScanner.RemoteScanner.Sshcp (RemoteScanner/RemoteScanner) - Command took 0.856s
2023-08-01T17:48:24.795776046Z 2023-08-01 18:48:24,790 - INFO - RemoteScanner.RemoteScanner (RemoteScanner/RemoteScanner) - Skipping remote scanfs installation: already installed
2023-08-01T17:48:24.795782323Z 2023-08-01 18:48:24,790 - DEBUG - RemoteScanner.RemoteScanner.Sshcp (RemoteScanner/RemoteScanner) - Command: ssh -p 22 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=error -o PubkeyAuthentication=no username@host.example.com "'/home/username/bin/tmp-seedsync/scanfs' '/home/username/Downloads/completed'"
2023-08-01T17:48:25.797438051Z 2023-08-01 18:48:25,731 - DEBUG - RemoteScanner.RemoteScanner.Sshcp (RemoteScanner/RemoteScanner) - Return code: 0
2023-08-01T17:48:25.797486054Z 2023-08-01 18:48:25,732 - DEBUG - RemoteScanner.RemoteScanner.Sshcp (RemoteScanner/RemoteScanner) - Command took 0.941s
2023-08-01T17:48:25.797574876Z 2023-08-01 18:48:25,734 - DEBUG - RemoteScanner (RemoteScanner/RemoteScanner) - Scan took 1.863s
2023-08-01T17:48:26.186417630Z 2023-08-01 18:48:26,186 - DEBUG - seedsync.Controller.Model (MainProcess/ControllerJob) - LftpModel: Adding file 'Books'
2023-08-01T17:48:26.186490371Z 2023-08-01 18:48:26,186 - DEBUG - seedsync.Controller.Model (MainProcess/ControllerJob) - LftpModel: Adding file 'Films'
2023-08-01T17:48:26.186505398Z 2023-08-01 18:48:26,186 - DEBUG - seedsync.Controller.Model (MainProcess/ControllerJob) - LftpModel: Adding file 'Audiobooks'
2023-08-01T17:48:26.186547081Z 2023-08-01 18:48:26,186 - DEBUG - seedsync.Controller.Model (MainProcess/ControllerJob) - LftpModel: Adding file 'TV'
2023-08-01T17:48:26.186583768Z 2023-08-01 18:48:26,186 - DEBUG - seedsync.Controller.Model (MainProcess/ControllerJob) - LftpModel: Adding file 'Music'
2023-08-01T17:48:26.186634931Z 2023-08-01 18:48:26,186 - DEBUG - seedsync.Controller.Model (MainProcess/ControllerJob) - LftpModel: Adding file '.stignore'
2023-08-01T17:48:26.186667184Z 2023-08-01 18:48:26,186 - DEBUG - seedsync.Controller.Model (MainProcess/ControllerJob) - LftpModel: Adding file '.stfolder'
2023-08-01T17:48:33.110112827Z 2023-08-01 18:48:33,020 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Scan took 9.153s
2023-08-01T17:48:33.378982561Z 2023-08-01 18:48:33,378 - DEBUG - seedsync.Controller.Model (MainProcess/ControllerJob) - LftpModel: Updating file 'Films'
2023-08-01T17:48:33.379032793Z 2023-08-01 18:48:33,378 - DEBUG - seedsync.Controller.Model (MainProcess/ControllerJob) - LftpModel: Updating file 'TV'
2023-08-01T17:48:33.379101873Z 2023-08-01 18:48:33,379 - DEBUG - seedsync.Controller.Model (MainProcess/ControllerJob) - LftpModel: Updating file '.stfolder'
2023-08-01T17:48:33.379161568Z 2023-08-01 18:48:33,379 - DEBUG - seedsync.Controller.Model (MainProcess/ControllerJob) - LftpModel: Updating file 'Books'
2023-08-01T17:48:33.379218884Z 2023-08-01 18:48:33,379 - DEBUG - seedsync.Controller.Model (MainProcess/ControllerJob) - LftpModel: Updating file 'Audiobooks'
2023-08-01T17:48:33.379245463Z 2023-08-01 18:48:33,379 - DEBUG - seedsync.Controller.Model (MainProcess/ControllerJob) - LftpModel: Updating file 'Music'
2023-08-01T17:48:33.911563613Z 2023-08-01 18:48:33,869 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Running a scan
2023-08-01T17:48:33.911617694Z 2023-08-01 18:48:33,885 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Scan took 0.016s
2023-08-01T17:48:43.928140202Z 2023-08-01 18:48:43,869 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Running a scan
2023-08-01T17:48:43.928202199Z 2023-08-01 18:48:43,896 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Scan took 0.027s
2023-08-01T17:48:53.882070110Z 2023-08-01 18:48:53,881 - DEBUG - seedsync (MainProcess/MainThread) - Persisting states to file
2023-08-01T17:48:53.945535707Z 2023-08-01 18:48:53,870 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Running a scan
2023-08-01T17:48:53.945682106Z 2023-08-01 18:48:53,872 - DEBUG - RemoteScanner (RemoteScanner/RemoteScanner) - Running a scan
2023-08-01T17:48:53.945695374Z 2023-08-01 18:48:53,872 - DEBUG - RemoteScanner.RemoteScanner.Sshcp (RemoteScanner/RemoteScanner) - Command: ssh -p 22 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=error -o PubkeyAuthentication=no username@host.example.com "'/home/username/bin/tmp-seedsync/scanfs' '/home/username/Downloads/completed'"
2023-08-01T17:48:53.945732189Z 2023-08-01 18:48:53,888 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Scan took 0.018s
2023-08-01T17:48:54.747338427Z 2023-08-01 18:48:54,697 - DEBUG - RemoteScanner.RemoteScanner.Sshcp (RemoteScanner/RemoteScanner) - Return code: 0
2023-08-01T17:48:54.747408102Z 2023-08-01 18:48:54,697 - DEBUG - RemoteScanner.RemoteScanner.Sshcp (RemoteScanner/RemoteScanner) - Command took 0.824s
2023-08-01T17:48:54.747458804Z 2023-08-01 18:48:54,699 - DEBUG - RemoteScanner (RemoteScanner/RemoteScanner) - Scan took 0.826s
2023-08-01T17:49:03.962682540Z 2023-08-01 18:49:03,871 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Running a scan
2023-08-01T17:49:03.962751085Z 2023-08-01 18:49:03,884 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Scan took 0.013s
2023-08-01T17:49:13.879900327Z 2023-08-01 18:49:13,872 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Running a scan
2023-08-01T17:49:13.980250641Z 2023-08-01 18:49:13,888 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Scan took 0.017s
2023-08-01T17:49:23.896847714Z 2023-08-01 18:49:23,873 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Running a scan
2023-08-01T17:49:23.897135994Z 2023-08-01 18:49:23,873 - DEBUG - RemoteScanner (RemoteScanner/RemoteScanner) - Running a scan
2023-08-01T17:49:23.897195318Z 2023-08-01 18:49:23,873 - DEBUG - RemoteScanner.RemoteScanner.Sshcp (RemoteScanner/RemoteScanner) - Command: ssh -p 22 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=error -o PubkeyAuthentication=no username@host.example.com "'/home/username/bin/tmp-seedsync/scanfs' '/home/username/Downloads/completed'"
2023-08-01T17:49:23.897339633Z 2023-08-01 18:49:23,888 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Scan took 0.015s
2023-08-01T17:49:23.902097417Z 2023-08-01 18:49:23,901 - DEBUG - seedsync (MainProcess/MainThread) - Persisting states to file
2023-08-01T17:49:24.799030078Z 2023-08-01 18:49:24,706 - DEBUG - RemoteScanner.RemoteScanner.Sshcp (RemoteScanner/RemoteScanner) - Return code: 0
2023-08-01T17:49:24.799096088Z 2023-08-01 18:49:24,707 - DEBUG - RemoteScanner.RemoteScanner.Sshcp (RemoteScanner/RemoteScanner) - Command took 0.833s
2023-08-01T17:49:24.799142065Z 2023-08-01 18:49:24,708 - DEBUG - RemoteScanner (RemoteScanner/RemoteScanner) - Scan took 0.835s
2023-08-01T17:49:33.915240670Z 2023-08-01 18:49:33,873 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Running a scan
2023-08-01T17:49:33.915304345Z 2023-08-01 18:49:33,887 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Scan took 0.014s
2023-08-01T17:49:43.931101456Z 2023-08-01 18:49:43,874 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Running a scan
2023-08-01T17:49:43.931179959Z 2023-08-01 18:49:43,888 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Scan took 0.014s
2023-08-01T17:49:53.922308133Z 2023-08-01 18:49:53,922 - DEBUG - seedsync (MainProcess/MainThread) - Persisting states to file
2023-08-01T17:49:53.948351164Z 2023-08-01 18:49:53,874 - DEBUG - RemoteScanner (RemoteScanner/RemoteScanner) - Running a scan
2023-08-01T17:49:53.948382318Z 2023-08-01 18:49:53,874 - DEBUG - RemoteScanner.RemoteScanner.Sshcp (RemoteScanner/RemoteScanner) - Command: ssh -p 22 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=error -o PubkeyAuthentication=no username@host.example.com "'/home/username/bin/tmp-seedsync/scanfs' '/home/username/Downloads/completed'"
2023-08-01T17:49:53.948441826Z 2023-08-01 18:49:53,874 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Running a scan
2023-08-01T17:49:53.948505476Z 2023-08-01 18:49:53,890 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Scan took 0.016s
2023-08-01T17:50:03.965381156Z 2023-08-01 18:50:03,875 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Running a scan
2023-08-01T17:50:03.965467090Z 2023-08-01 18:50:03,889 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Scan took 0.013s
2023-08-01T17:50:13.881830913Z 2023-08-01 18:50:13,876 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Running a scan
2023-08-01T17:50:13.982244811Z 2023-08-01 18:50:13,890 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Scan took 0.014s
quadcom commented 1 year ago

I didn't see any attempt to dl anything there

UpperCenter commented 1 year ago

image The option to queue files is greyed out, error at the top of the screen is "An error occurred while installing scanner script to remote server: 'Timed out'."

More logs:


01/08/2023 19:25:25
2023-08-01 19:25:25,479 - DEBUG - seedsync (MainProcess/MainThread) - Persisting states to file
01/08/2023 19:25:27
2023-08-01 19:25:27,587 - ERROR - RemoteScanner.RemoteScanner.Sshcp (RemoteScanner/RemoteScanner) - Timed out
01/08/2023 19:25:27
Traceback (most recent call last):
01/08/2023 19:25:27
  File "/app/python/ssh/sshcp.py", line 101, in __run_command
01/08/2023 19:25:27
    i = sp.expect(
01/08/2023 19:25:27
  File "/usr/local/lib/python3.8/site-packages/pexpect/spawnbase.py", line 343, in expect
01/08/2023 19:25:27
    return self.expect_list(compiled_pattern_list,
01/08/2023 19:25:27
  File "/usr/local/lib/python3.8/site-packages/pexpect/spawnbase.py", line 372, in expect_list
01/08/2023 19:25:27
    return exp.expect_loop(timeout)
01/08/2023 19:25:27
  File "/usr/local/lib/python3.8/site-packages/pexpect/expect.py", line 181, in expect_loop
01/08/2023 19:25:27
    return self.timeout(e)
01/08/2023 19:25:27
  File "/usr/local/lib/python3.8/site-packages/pexpect/expect.py", line 144, in timeout
01/08/2023 19:25:27
    raise exc
01/08/2023 19:25:27
pexpect.exceptions.TIMEOUT: Timeout exceeded.
01/08/2023 19:25:27
<pexpect.pty_spawn.spawn object at 0x147f995e04c0>
01/08/2023 19:25:27
command: /usr/local/sbin/ssh
01/08/2023 19:25:27
args: ['/usr/local/sbin/ssh', '-p', '22', '-o', 'StrictHostKeyChecking=no', '-o', 'UserKnownHostsFile=/dev/null', '-o', 'LogLevel=error', '-o', 'PubkeyAuthentication=no', 'user@host.example.com', "'/home/user/bin/tmp-seedsync/scanfs' '/home/user/Downloads/completed'"]
01/08/2023 19:25:27
buffer (last 100 chars): b'\r\n'
01/08/2023 19:25:27
before (last 100 chars): b'\r\n'
01/08/2023 19:25:27
after: <class 'pexpect.exceptions.TIMEOUT'>
01/08/2023 19:25:27
match: None
01/08/2023 19:25:27
match_index: None
01/08/2023 19:25:27
exitstatus: None
01/08/2023 19:25:27
flag_eof: False
01/08/2023 19:25:27
pid: 361
01/08/2023 19:25:27
child_fd: 39
01/08/2023 19:25:27
closed: False
01/08/2023 19:25:27
timeout: 30
01/08/2023 19:25:27
delimiter: <class 'pexpect.exceptions.EOF'>
01/08/2023 19:25:27
logfile: None
01/08/2023 19:25:27
logfile_read: None
01/08/2023 19:25:27
logfile_send: None
01/08/2023 19:25:27
maxread: 2000
01/08/2023 19:25:27
ignorecase: False
01/08/2023 19:25:27
searchwindowsize: None
01/08/2023 19:25:27
delaybeforesend: 0.05
01/08/2023 19:25:27
delayafterclose: 0.1
01/08/2023 19:25:27
delayafterterminate: 0.1
01/08/2023 19:25:27
searcher: searcher_re:
01/08/2023 19:25:27
    0: EOF
01/08/2023 19:25:27
    1: re.compile(b'password: ')
01/08/2023 19:25:27
    2: re.compile(b'lost connection')
01/08/2023 19:25:27
    3: re.compile(b'Could not resolve hostname')
01/08/2023 19:25:27
    4: re.compile(b'Connection refused')
01/08/2023 19:25:27
2023-08-01 19:25:27,588 - ERROR - RemoteScanner.RemoteScanner.Sshcp (RemoteScanner/RemoteScanner) - Command output before:
01/08/2023 19:25:27
b'\r\n'
01/08/2023 19:25:27
2023-08-01 19:25:27,588 - WARNING - RemoteScanner.RemoteScanner (RemoteScanner/RemoteScanner) - Caught an SshcpError: Timed out
01/08/2023 19:25:27
2023-08-01 19:25:27,588 - DEBUG - RemoteScanner (RemoteScanner/RemoteScanner) - Scan took 180.459s
01/08/2023 19:25:27
2023-08-01 19:25:27,588 - DEBUG - RemoteScanner (RemoteScanner/RemoteScanner) - Running a scan
01/08/2023 19:25:27
2023-08-01 19:25:27,588 - DEBUG - RemoteScanner.RemoteScanner.Sshcp (RemoteScanner/RemoteScanner) - Command: ssh -p 22 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=error -o PubkeyAuthentication=no user@host.example.com "'/home/user/bin/tmp-seedsync/scanfs' '/home/user/Downloads/completed'"
01/08/2023 19:25:27
2023-08-01 19:25:27,827 - DEBUG - seedsync.Controller.Model (MainProcess/ControllerJob) - LftpModel: Removing file '.stignore'
01/08/2023 19:25:27
2023-08-01 19:25:27,828 - DEBUG - seedsync.Controller.Model (MainProcess/ControllerJob) - LftpModel: Updating file 'Films'
01/08/2023 19:25:27
2023-08-01 19:25:27,828 - DEBUG - seedsync.Controller.Model (MainProcess/ControllerJob) - LftpModel: Updating file 'TV'
01/08/2023 19:25:27
2023-08-01 19:25:27,828 - DEBUG - seedsync.Controller.Model (MainProcess/ControllerJob) - LftpModel: Updating file '.stfolder'
01/08/2023 19:25:27
2023-08-01 19:25:27,828 - DEBUG - seedsync.Controller.Model (MainProcess/ControllerJob) - LftpModel: Updating file 'Books'
01/08/2023 19:25:27
2023-08-01 19:25:27,828 - DEBUG - seedsync.Controller.Model (MainProcess/ControllerJob) - LftpModel: Updating file 'Audiobooks'
01/08/2023 19:25:27
2023-08-01 19:25:27,828 - DEBUG - seedsync.Controller.Model (MainProcess/ControllerJob) - LftpModel: Updating file 'Music'
01/08/2023 19:25:34
2023-08-01 19:25:34,028 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Running a scan
01/08/2023 19:25:34
2023-08-01 19:25:34,043 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Scan took 0.016s
01/08/2023 19:25:44
2023-08-01 19:25:44,029 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Running a scan
01/08/2023 19:25:44
2023-08-01 19:25:44,042 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Scan took 0.014s
01/08/2023 19:25:54
2023-08-01 19:25:54,029 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Running a scan
01/08/2023 19:25:54
2023-08-01 19:25:54,043 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Scan took 0.014s
01/08/2023 19:25:55
2023-08-01 19:25:55,498 - DEBUG - seedsync (MainProcess/MainThread) - Persisting states to file
01/08/2023 19:25:57
2023-08-01 19:25:57,667 - ERROR - RemoteScanner.RemoteScanner.Sshcp (RemoteScanner/RemoteScanner) - Timed out
01/08/2023 19:25:57
Traceback (most recent call last):
01/08/2023 19:25:57
  File "/app/python/ssh/sshcp.py", line 76, in __run_command
01/08/2023 19:25:57
    i = sp.expect([
01/08/2023 19:25:57
  File "/usr/local/lib/python3.8/site-packages/pexpect/spawnbase.py", line 343, in expect
01/08/2023 19:25:57
    return self.expect_list(compiled_pattern_list,
01/08/2023 19:25:57
  File "/usr/local/lib/python3.8/site-packages/pexpect/spawnbase.py", line 372, in expect_list
01/08/2023 19:25:57
    return exp.expect_loop(timeout)
01/08/2023 19:25:57
  File "/usr/local/lib/python3.8/site-packages/pexpect/expect.py", line 181, in expect_loop
01/08/2023 19:25:57
    return self.timeout(e)
01/08/2023 19:25:57
  File "/usr/local/lib/python3.8/site-packages/pexpect/expect.py", line 144, in timeout
01/08/2023 19:25:57
    raise exc
01/08/2023 19:25:57
pexpect.exceptions.TIMEOUT: Timeout exceeded.
01/08/2023 19:25:57
<pexpect.pty_spawn.spawn object at 0x147f995e0910>
01/08/2023 19:25:57
command: /usr/local/sbin/ssh
01/08/2023 19:25:57
args: ['/usr/local/sbin/ssh', '-p', '22', '-o', 'StrictHostKeyChecking=no', '-o', 'UserKnownHostsFile=/dev/null', '-o', 'LogLevel=error', '-o', 'PubkeyAuthentication=no', 'user@host.example.com', "'/home/user/bin/tmp-seedsync/scanfs' '/home/user/Downloads/completed'"]
01/08/2023 19:25:57
buffer (last 100 chars): b'mkdir: cannot create directory \xe2\x80\x98/home/seedsync\xe2\x80\x99: Permission denied\r\n'
01/08/2023 19:25:57
before (last 100 chars): b'mkdir: cannot create directory \xe2\x80\x98/home/seedsync\xe2\x80\x99: Permission denied\r\n'
01/08/2023 19:25:57
after: <class 'pexpect.exceptions.TIMEOUT'>
01/08/2023 19:25:57
match: None
01/08/2023 19:25:57
match_index: None
01/08/2023 19:25:57
exitstatus: None
01/08/2023 19:25:57
flag_eof: False
01/08/2023 19:25:57
pid: 367
01/08/2023 19:25:57
child_fd: 40
01/08/2023 19:25:57
closed: False
01/08/2023 19:25:57
timeout: 30
01/08/2023 19:25:57
delimiter: <class 'pexpect.exceptions.EOF'>
01/08/2023 19:25:57
logfile: None
01/08/2023 19:25:57
logfile_read: None
01/08/2023 19:25:57
logfile_send: None
01/08/2023 19:25:57
maxread: 2000
01/08/2023 19:25:57
ignorecase: False
01/08/2023 19:25:57
searchwindowsize: None
01/08/2023 19:25:57
delaybeforesend: 0.05
01/08/2023 19:25:57
delayafterclose: 0.1
01/08/2023 19:25:57
delayafterterminate: 0.1
01/08/2023 19:25:57
searcher: searcher_re:
01/08/2023 19:25:57
    0: re.compile(b'password: ')
01/08/2023 19:25:57
    1: EOF
01/08/2023 19:25:57
    2: re.compile(b'lost connection')
01/08/2023 19:25:57
    3: re.compile(b'Could not resolve hostname')
01/08/2023 19:25:57
    4: re.compile(b'Connection refused')
01/08/2023 19:25:57
2023-08-01 19:25:57,668 - ERROR - RemoteScanner.RemoteScanner.Sshcp (RemoteScanner/RemoteScanner) - Command output before:
01/08/2023 19:25:57
b'mkdir: cannot create directory \xe2\x80\x98/home/seedsync\xe2\x80\x99: Permission denied\r\n'
01/08/2023 19:25:57
2023-08-01 19:25:57,668 - WARNING - RemoteScanner.RemoteScanner (RemoteScanner/RemoteScanner) - Caught an SshcpError: Timed out
01/08/2023 19:25:57
2023-08-01 19:25:57,668 - DEBUG - RemoteScanner (RemoteScanner/RemoteScanner) - Scan took 30.080s
01/08/2023 19:25:57
2023-08-01 19:25:57,668 - DEBUG - RemoteScanner (RemoteScanner/RemoteScanner) - Running a scan
01/08/2023 19:25:57
2023-08-01 19:25:57,668 - DEBUG - RemoteScanner.RemoteScanner.Sshcp (RemoteScanner/RemoteScanner) - Command: ssh -p 22 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=error -o PubkeyAuthentication=no user@host.example.com "'/home/user/bin/tmp-seedsync/scanfs' '/home/user/Downloads/completed'"
01/08/2023 19:26:04
2023-08-01 19:26:04,031 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Running a scan
01/08/2023 19:26:04
2023-08-01 19:26:04,044 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Scan took 0.014s
01/08/2023 19:26:14
2023-08-01 19:26:14,031 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Running a scan
01/08/2023 19:26:14
2023-08-01 19:26:14,045 - DEBUG - LocalScanner (LocalScanner/LocalScanner) - Scan took 0.014s

It looks like the credentials to the remote server are incorrect, but I know for a fact that they are correct.

quadcom commented 1 year ago

I do see that you are trying to sync a single parent folder. But that's not how SS works. It messes with the sync functionality.

Set your remote target folder to where all downloads reside and see if that fixes your problems.

UpperCenter commented 1 year ago

Sorry, not sure what you mean. I have my remote path set to /home/user/Downloads/completed and changing it to /home/user/Downloads didn't change anything.

Thanks

quadcom commented 1 year ago

The completed location on your seedbox is filtering categories (i.e. movies, TV shows, books, etc.)

Those are parent categories. Seedsync can only sync whole folders. It doesn't work well with category folders.

Have all items that need to be synced in one folder and sync the root of that folder using Seedsync to your local destination. Your local destination should look exactly the same as your remote location when the sync is finished.

If you require categories to be synced due to automations in the local side, your only option is using syncthing. I moved away from syncthing because it is so slow and I don't need any post processing automations.

On Mon, Aug 7, 2023, 3:50 a.m. Tom Hindby @.***> wrote:

Sorry, not sure what you mean. I have my remote path set to /home/user/Downloads/completed and changing it to /home/user/Downloads didn't change anything.

Thanks

— Reply to this email directly, view it on GitHub https://github.com/ipsingh06/seedsync/issues/129#issuecomment-1667367890, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABWB62YLDCTWRAUNY4NC743XUCM3VANCNFSM6AAAAAA2Z2HDME . You are receiving this because you commented.Message ID: @.***>

UpperCenter commented 1 year ago

I rely quite heavily on the automation piece. I guess I will have to stick with Syncthing for now then. Thanks for all the help everyone.

quadcom commented 1 year ago

As an option, you could set up multiple SS instances and sync each parent into a corresponding local target—a little more to manage in a way, but not impossible. The project is pretty much abandoned, as far as I can tell. I have not heard from the dev in a long time, so it won't be updated routinely. Once it's up and running, you can pretty much forget about it. Shouldn't be too hard to get going in Docker.