Closed keppo070 closed 4 years ago
Looks like the database is either not being created, or cannot be accessed. This results in a cascade of errors.
Can you please provide:
ls -l /path/to/config/dir
groups [username]
version: "2" services: watcher3: image: ellnic/watcher3 container_name: watcher3 environment:
Distributor ID: Ubuntu Description: Ubuntu 18.04.3 LTS Release: 18.04 Codename: bionic
Python 2.7.15+ (default, Oct 7 2019, 17:39:04) [GCC 7.4.0] on linux2 Type "help", "copyright", "credits" or "license" for more information.
total 4 -rw-rw-r-- 1 patrick patrick 388 nov 16 09:53 docker-compose.yml
patrick adm cdrom sudo dip plugdev lpadmin sambashare libvirt nfsgrp smbgrp
I also run Bionic, so I know it’s workable. Let’s get this running for you.
Your config folder is empty so it seems the errors you see in the log are because watcher is unable to create the database to start. This is probably permissions related.
You don’t appear to be in the docker group.
Can you run:
sudo usermod -a patrick -G docker
Then logout and back in and try again.
Also, are you running the docker from the bionic repos or have you added the docker official repo?
I used the usermod but same problem. I SSH into ubuntu, but I dont think thats a problem.
I checked the error and it is the same as above. Still nothing in the config folder. I tried to run the docker without using docker-compose but that gave the same result.
Dont know exactly how to check which repo i used. I ran
apt list --installed | grep docker
result: docker-ce/bionic,now 5:19.03.5~3-0~ubuntu-bionic amd64 [installed] docker-ce-cli/bionic,now 5:19.03.5~3-0~ubuntu-bionic amd64 [installed,automatic]
I guess i'm using the bionic repo.
I have several other docker-containers running. Most of them are from linuxserver. I start them much the same way as I do with the ellnic/watcher3 container.
If you don't remember installing from the Docker repo, then you almost certainly didn't. You need to manually add their repo.
Try this first:
sudo chmod -R 755 /path/to/watcher/config/folder
then
sudo chown -R patrick:patrick /path/to/watcher/config/folder
Delete the container and recreate without the UMASK, leave all else. Does it work now?
Note: Compose shouldn't be doing it, although I don't use it. I just run:
docker run -d --name=watcher3 -v /mnt/Docker/docker-configs/watcher3:/config -v /mnt/Tank/:/Tank -e PGID=1001 -e PUID=1000 -p 9191:9090 --restart unless-stopped ellnic/watcher3
NB: Group 1001 is a general media group, so I don't use 1000, but it shouldn't make a diff.
ls -al
on /home/patrick/docker-config
gives me
total 12
drwxrwxr-x 2 patrick patrick 4096 nov 16 09:53 .
drwxrwxr-x 30 patrick patrick 4096 nov 16 09:32 ..
-rw-rw-r-- 1 patrick patrick 388 nov 16 09:53 docker-compose.yml
I mount the config in my homefolder. That should give me enough rights.
I tried directly without compose (I just left out the -d)
sudo docker run --name=watcher3 -v /home/patrick/docker-config/watcher3:/config -v /var/usenet/tempdata/complete:/downloads -v /var/usenet/media/video/movie:/movies -e PUID=1000 -e PGID=1000 -p 9091:9090 --restart unless-stopped ellnic/watcher3
Same result. Opening SQL DB and then the traceback starts.
I removed the images. I saw that I had a docker from bbqsauce and removed that one also. Started over but nothing. Same result.
I noticed just now that there is a slight difference. I used both time the same cmdline.
in the last error dump after a rmi and a pull
Creating database file /config/db/database.sqlite
Creating tables.
Opening SQL DB.
Traceback
Before it was only
Opening SQL DB.
Traceback
Nothing is created in config thou
Hrm. This isn’t the behaviour I get. Can you tell me if you get the same result with:
sudo docker run --name=watcher3-test -v /tmp/docker-test:/config -e PUID=1000 -e PGID=1000 -p 9091:9090 --restart unless-stopped ellnic/watcher3
sudo docker run --name=watcher3-test -v /tmp/docker-test:/config -e PUID=1000 -e PGID=1000 -p 9091:9090 --restart unless-stopped ellnic/watcher3
Creating database file /config/db/database.sqlite Creating tables. Opening SQL DB. Traceback (most recent call last): File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2141, in _wrap_pool_connect return fn() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 387, in connect return _ConnectionFairy._checkout(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 766, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 516, in checkout rec = pool._do_get() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 1229, in _do_get return self._create_connection() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 333, in _create_connection return _ConnectionRecord(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 461, in init self.connect(first_connect_check=True) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 651, in connect connection = pool._invoke_creator(self) File "/opt/watcher3/lib/sqlalchemy/engine/strategies.py", line 105, in connect return dialect.connect(*cargs, *cparams) File "/opt/watcher3/lib/sqlalchemy/engine/default.py", line 385, in connect return self.dbapi.connect(cargs, **cparams) sqlite3.OperationalError: unable to open database file
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/opt/watcher3/core/sqldb.py", line 120, in init self.create_database(DB_NAME) File "/opt/watcher3/core/sqldb.py", line 139, in create_database self.metadata.create_all(self.engine) File "/opt/watcher3/lib/sqlalchemy/sql/schema.py", line 3848, in create_all tables=tables) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1922, in _run_visitor with self._optional_conn_ctx_manager(connection) as conn: File "/usr/lib/python3.6/contextlib.py", line 81, in enter return next(self.gen) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1915, in _optional_conn_ctx_manager with self.contextual_connect() as conn: File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2106, in contextual_connect self._wrap_pool_connect(self.pool.connect, None), File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2145, in _wrap_pool_connect e, dialect, self) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1456, in _handle_dbapi_exception_noconnection exc_info File "/opt/watcher3/lib/sqlalchemy/util/compat.py", line 202, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/opt/watcher3/lib/sqlalchemy/util/compat.py", line 185, in reraise raise value.with_traceback(tb) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2141, in _wrap_pool_connect return fn() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 387, in connect return _ConnectionFairy._checkout(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 766, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 516, in checkout rec = pool._do_get() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 1229, in _do_get return self._create_connection() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 333, in _create_connection return _ConnectionRecord(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 461, in init self.connect(first_connect_check=True) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 651, in connect connection = pool._invoke_creator(self) File "/opt/watcher3/lib/sqlalchemy/engine/strategies.py", line 105, in connect return dialect.connect(*cargs, cparams) File "/opt/watcher3/lib/sqlalchemy/engine/default.py", line 385, in connect return self.dbapi.connect(*cargs, *cparams) sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file Traceback (most recent call last): File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2141, in _wrap_pool_connect return fn() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 387, in connect return _ConnectionFairy._checkout(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 766, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 516, in checkout rec = pool._do_get() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 1229, in _do_get return self._create_connection() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 333, in _create_connection return _ConnectionRecord(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 461, in init self.connect(first_connect_check=True) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 651, in connect connection = pool._invoke_creator(self) File "/opt/watcher3/lib/sqlalchemy/engine/strategies.py", line 105, in connect return dialect.connect(cargs, cparams) File "/opt/watcher3/lib/sqlalchemy/engine/default.py", line 385, in connect return self.dbapi.connect(*cargs, **cparams) sqlite3.OperationalError: unable to open database file
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/watcher3/watcher.py", line 84, in
it seems I have possibly installed from docker because apt-cache policy
give also an entry for a docker repository. I am not so sure I have installed from the ubuntu repository.
500 https://download.docker.com/linux/ubuntu bionic/stable amd64 Packages
release o=Docker,a=bionic,l=Docker CE,c=stable,b=amd64
origin download.docker.com
Ok, try without the PUID and PGID. This will run entirely as root:
sudo docker run --name=watcher3-test -v /tmp/docker-test:/config -p 9091:9090 --restart unless-stopped ellnic/watcher3
it seems to be shorter. It took it bit longer but I got the error msg
sudo docker run --name=watcher3-test -v /tmp/docker-test:/config -p 9091:9090 --restart unless-stopped ellnic/watcher3 Creating database file /config/db/database.sqlite Creating tables. Opening SQL DB. Traceback (most recent call last): File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2141, in _wrap_pool_connect return fn() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 387, in connect return _ConnectionFairy._checkout(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 766, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 516, in checkout rec = pool._do_get() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 1229, in _do_get return self._create_connection() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 333, in _create_connection return _ConnectionRecord(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 461, in init self.connect(first_connect_check=True) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 651, in connect connection = pool._invoke_creator(self) File "/opt/watcher3/lib/sqlalchemy/engine/strategies.py", line 105, in connect return dialect.connect(*cargs, *cparams) File "/opt/watcher3/lib/sqlalchemy/engine/default.py", line 385, in connect return self.dbapi.connect(cargs, **cparams) sqlite3.OperationalError: unable to open database file
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/opt/watcher3/core/sqldb.py", line 120, in init self.create_database(DB_NAME) File "/opt/watcher3/core/sqldb.py", line 139, in create_database self.metadata.create_all(self.engine) File "/opt/watcher3/lib/sqlalchemy/sql/schema.py", line 3848, in create_all tables=tables) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1922, in _run_visitor with self._optional_conn_ctx_manager(connection) as conn: File "/usr/lib/python3.6/contextlib.py", line 81, in enter return next(self.gen) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1915, in _optional_conn_ctx_manager with self.contextual_connect() as conn: File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2106, in contextual_connect self._wrap_pool_connect(self.pool.connect, None), File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2145, in _wrap_pool_connect e, dialect, self) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1456, in _handle_dbapi_exception_noconnection exc_info File "/opt/watcher3/lib/sqlalchemy/util/compat.py", line 202, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/opt/watcher3/lib/sqlalchemy/util/compat.py", line 185, in reraise raise value.with_traceback(tb) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2141, in _wrap_pool_connect return fn() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 387, in connect return _ConnectionFairy._checkout(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 766, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 516, in checkout rec = pool._do_get() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 1229, in _do_get return self._create_connection() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 333, in _create_connection return _ConnectionRecord(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 461, in init self.connect(first_connect_check=True) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 651, in connect connection = pool._invoke_creator(self) File "/opt/watcher3/lib/sqlalchemy/engine/strategies.py", line 105, in connect return dialect.connect(*cargs, cparams) File "/opt/watcher3/lib/sqlalchemy/engine/default.py", line 385, in connect return self.dbapi.connect(*cargs, *cparams) sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file Traceback (most recent call last): File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2141, in _wrap_pool_connect return fn() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 387, in connect return _ConnectionFairy._checkout(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 766, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 516, in checkout rec = pool._do_get() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 1229, in _do_get return self._create_connection() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 333, in _create_connection return _ConnectionRecord(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 461, in init self.connect(first_connect_check=True) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 651, in connect connection = pool._invoke_creator(self) File "/opt/watcher3/lib/sqlalchemy/engine/strategies.py", line 105, in connect return dialect.connect(cargs, cparams) File "/opt/watcher3/lib/sqlalchemy/engine/default.py", line 385, in connect return self.dbapi.connect(*cargs, **cparams) sqlite3.OperationalError: unable to open database file
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/watcher3/watcher.py", line 84, in
Ok, I’m kinda stumped here. I’m by no means a docker expert. Do you get the same results with:
sudo docker run --name=watcher3-testbbq -v /tmp/docker-testbbq:/config -p 9091:9090 --restart unless-stopped barbequesauce/watcher3
sudo dpkg -l | grep -i docker
ii docker-ce 5:19.03.5~3-0~ubuntu-bionic amd64 Docker: the open-source application container engine
ii docker-ce-cli 5:19.03.5~3-0~ubuntu-bionic amd64 Docker CLI: the open-source application container engine
It seems I'm running the Ubuntu version. Dont know if this is still important for you
BTW: I appreciate the time your spending on this, but if this is related only to my setup. Feel free to abandon the issue.
sudo docker run --name=watcher3-testbbq -v /tmp/docker-testbbq:/config -p 9091:9090 --restart unless-stopped barbequesauce/watcher3
Creating database file /config/db/database.sqlite
Creating tables.
Opening SQL DB.
Traceback....
It’s not a problem. :-) it’s as much a learning curve for me. I am more interested in finding out what it is in case this pops up again.
The only major difference I can see here is that I am using the official docker repos. I’ve never used the bionic versions because the docker own repos are naturally more up to date. Maybe the next step would be to move to that? It’s not impossible that some change they’ve made to the engine could be causing this.
Have a look here: https://docs.docker.com/install/linux/docker-ce/ubuntu/
Note that it specifically runs through removal of distro own versions then installing theirs.
Try this and see if that helps.
when I run docker version
I get this
Client: Docker Engine - Community
Version: 19.03.5
API version: 1.40
Go version: go1.12.12
Git commit: 633a0ea838
Built: Wed Nov 13 07:29:52 2019
OS/Arch: linux/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 19.03.5
API version: 1.40 (minimum version 1.12)
Go version: go1.12.12
Git commit: 633a0ea838
Built: Wed Nov 13 07:28:22 2019
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.2.10
GitCommit: b34a5c8af56e510852c35414db4c1f4fa6172339
runc:
Version: 1.0.0-rc8+dev
GitCommit: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
docker-init:
Version: 0.18.0
GitCommit: fec3683
So is this the ubuntu version or the docker version? I will try the docker image on a virtualbox first. To be honest I am not looking forward to replace my docker installation. Can you confirm which version this is?
Ah.... you are definitely running the docker repo version.
I’ve just managed to reproduce this. Give me a second. Just changing the build file.
Ok, that hasn't made a difference. Trying one more thing and if this doesn't work I might have to talk to the guys on the docker forums. Standby.
Ok, still no change. This seems to be a recent change, as I couldn't reproduce before and now I can. Looks like docker was updated on Nov 13th. So something must have changed. I'm sure it's something to do with permissions / root.
It's late here now (I'm in the UK) but give me a couple of days to figure this out and I will report back ASAP.
@barbequesauce @scambra
Might have some bad news. I have made several changes to the docker file and am still having no luck here, so I tried running using python instead, but mimicking the docker run args. I used sudo for the sake of eliminating permission issues:
sudo python3 watcher.py -c /home/ellnic/Desktop/watcher-test/config/watcher.cfg -l /home/ellnic/Desktop/watcher-test/config/logs/ --db /home/ellnic/Desktop/watcher-test/config/db/database.sqlite --plugins /home/ellnic/Desktop/watcher-test/config/plugins/ --posters /home/ellnic/Desktop/watcher-test/config/posters
And unfortunately, I am getting similar errors as the above reported by @keppo070
So it looks like it's not a docker error, but something to do with the way flags are handled. If I specify no flags, it runs fine. Can either of you reproduce this error with these or similar arguments? Has anything changed that might affect this?
Error is raising on self.metadata.create_all(self.engine) line, which creates tables. I have added new fields to tables recently, and I got no issue updating DB, but it may be related. I have tried renaming config directory, to start with new one, and I had no issue.
I'm using docker, but I use own build file, based in old file from linuxserver (https://github.com/linuxserver-archive/docker-watcher-stale), because I run in raspberry pi and other images didn't work. Maybe my docker image is using a different sqlite version, it's based on lsiobase/alpine:3.8:
root@027ef64b34b8:/# apk list *sqlite*
sqlite-libs-3.25.3-r1 armhf {sqlite} (Public-Domain) [installed]
On Nov 15th, I added these 2 columns:
sqla.Column('download_progress', sqla.INT),
sqla.Column('download_time', sqla.TIMESTAMP)
No column was using TIMESTAMP type before, can it be related? It could probably be changed to INT without affecting current code.
@ellnic can you try changing download_time to sqla.INT? Is it failing?
The docker image I put together also uses Alpine 3.8, although not ARM.
There is a minor change in the sqlite version:
sqlite-libs-3.25.3-r2 x86_64 {sqlite} (Public-Domain) [installed]
I am testing using my master which does not have the sqla.Column('download_time', sqla.TIMESTAMP) code yet.
I have just checked using barbequesauce master and the results are the same when:
sqla.Column('download_time', sqla.TIMESTAMP)
is changed to
sqla.Column('download_time', sqla.INT)
I am not sure if that is the cause. If no flags are specified, ie. just running:
python3 watcher.py
It starts fine, and places the required files in the userdata folder. It is only when we specify some flags that the error occurs. Eg:
sudo python3 watcher.py -c /home/ellnic/Desktop/watcher-test/config/watcher.cfg
It doesn't seem to matter what flag... --db, -c, -l etc. It's when the files are meant to be somewhere other than the userdata folder.
Edit:
Specifying just the --userdata flag works:
python3 watcher.py --userdata /home/ellnic/Desktop/watcher-test
But it creates files with default names:
config.cfg watcher.sqlite
Whereas in our docker we use:
watcher.cfg
/db/database.sqlite
This was originally done to facilitate anyone wanting to move from hurricane/watcher3, which at the time seemed to have more activity than the others.
I could change the dockerfile to just use --userdata as an intermediate fix, but it would require manual editing of user files to keep their existing config and db. :-/
Edit: I am guessing that since the error exists in my master as well as this one, it's likely to be something from here: https://github.com/ellnic/Watcher3/pull/15 or before, so prior to 21st October.
My docker file has a start script wich runs watcher with this command:
exec \
s6-setuidgid abc python3 watcher.py --db /config/db/database.sqlite \
--conf /config/config.cfg --log /config/logs/ --plugins /config/plugins/ --posters /config/posters
So it uses flags too. Linuxserver docker image doesn't use CMD in Dockerfile, it has /etc/services.d/watcher/run script, which calls exec to run watcher. Using something like that, script could move files from previous locations to new one, and rename them to new name.
I don't know if it's related to older docker version, I haven't updated it for a while:
Client: Docker Engine - Community
Version: 19.03.1
API version: 1.40
Go version: go1.12.5
Git commit: 74b1e89
Built: Thu Jul 25 21:32:16 2019
OS/Arch: linux/arm
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 19.03.1
API version: 1.40 (minimum version 1.12)
Go version: go1.12.5
Git commit: 74b1e89
Built: Thu Jul 25 21:26:16 2019
OS/Arch: linux/arm
Experimental: false
containerd:
Version: 1.2.6
GitCommit: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
runc:
Version: 1.0.0-rc8
GitCommit: 425e105d5a03fabd737a126ad93d62a9eeede87f
docker-init:
Version: 0.18.0
GitCommit: fec3683
Stretch repo for raspbian has 19.03.4 available, but not 19.03.5
It's nothing to do with docker, because I can get the error without docker.
If I pull a fresh barbequesauce master, and run the same command as your docker from terminal:
python3 watcher.py --db /config/db/database.sqlite --conf /config/config.cfg --log /config/logs/ --plugins /config/plugins/ --posters /config/posters
I get:
Creating database file /config/db/database.sqlite
Creating tables.
Opening SQL DB.
Traceback (most recent call last):
[truncated]
Can you try this on your end? Just run from terminal.
I know we can probably run a script to rename the config files then switch to the --userdata flag for the docker, but that doesn't solve the issue of the other flags not working.
More info. It only happens when there are no files present. If I take my docker config files and place them in the copy on the desktop and run:
~/Desktop/Watcher3-master » sudo python3 watcher.py --db test-config/db/database.sqlite --conf test-config/watcher.cfg --log test-config/logs/ --plugins test-config/plugins/ --posters test-config/posters -p 1234
I get a success:
Connected to database sqlite:///test-config/db/database.sqlite Config file found, merging any new options. Clearing Mako cache. [21/Nov/2019:17:36:05] ENGINE Listening for SIGTERM. [21/Nov/2019:17:36:05] ENGINE Listening for SIGHUP. [21/Nov/2019:17:36:05] ENGINE Listening for SIGUSR1. [21/Nov/2019:17:36:05] ENGINE Bus STARTING [21/Nov/2019:17:36:05] ENGINE Serving on http://0.0.0.0:1234 [21/Nov/2019:17:36:05] ENGINE Bus STARTED
@scambra Can you test my specifying a new location for your docker config, and see if it starts?
Edit:
python3 watcher.py
fails to create new files
but:
python3 watcher.py --userdata userdata
succeeds
Insert confused Scooby-Doo sound here.
Yeah, not quite sure what has happened.
@keppo070 I have created a fresh new database and config files. Please can you download this: https://www.dropbox.com/s/d72wp41365hjtgg/config.zip?dl=0
And put in your config folder, then tell me if the container starts.
I put the config in my config folder. Must problay edit some stuff but it is starting up!
watcher3 | [21/Nov/2019:18:03:09] ENGINE Listening for SIGTERM.
watcher3 | [21/Nov/2019:18:03:09] ENGINE Listening for SIGHUP.
watcher3 | [21/Nov/2019:18:03:09] ENGINE Listening for SIGUSR1.
watcher3 | [21/Nov/2019:18:03:09] ENGINE Bus STARTING
watcher3 | CherryPy Checker:
watcher3 | dir is an absolute path, even though a root is provided.
watcher3 | section: [/posters]
watcher3 | root: '/opt/watcher3'
watcher3 | dir: '/config/posters'
watcher3 |
watcher3 | CherryPy Checker:
watcher3 | dir is an absolute path, even though a root is provided.
watcher3 | section: [/posters/missing_poster.jpg]
watcher3 | root: '/opt/watcher3'
watcher3 | dir: '/config/posters'
watcher3 |
watcher3 | CherryPy Checker:
watcher3 | dir is an absolute path, even though a root is provided.
watcher3 | section: [/logs]
watcher3 | root: '/opt/watcher3'
watcher3 | dir: '/config/logs/'
watcher3 |
watcher3 | [21/Nov/2019:18:03:09] ENGINE Serving on http://0.0.0.0:9090
watcher3 | [21/Nov/2019:18:03:09] ENGINE Bus STARTED
Fantastic.
So the problem exists when no config files are present unless specifying the --userdata flag.
I am working on a fix for the docker. It won't fix the flags issue, but it will solve the docker in the meantime.
Ok so there we go. It took a while but I have fixed the ellnic docker. I have added an entrypoint script that looks for database and config files by the old names and renames/moves that to the defaults. I changed the run command to just use the --userdata flag and point to the /config folder.
This results in:
Notes: The entrypoint script attempts to remove the old db folder, but I used rmdir specifically because it will NOT remove any dir that isn't empty (in case the mv fails for some reason). In my case, the rmdir fails because I have database backups that are not moved out of that folder.
I also changed the way the container runs. I have added support for APP_UID and APP_GID instead of PUID and PGID. If the entry script sees that APP_UID or APP_GID have been specified, it will chown the /opt/watcher and /config dirs to those IDs. If they aren't specified, the chown will be to 0:0.
I have altered my readme, but I will include a new example run command here now:
docker run -d \ --name=watcher3 \ -v /path/to/config/:/config \ -v /path/to/downloads/:/downloads \ -v /path/to/movies/:/movies \ -e UMASK_SET=022 \ -e APP_UID=1000 -e APP_GID=1000 \ -p 9090:9090 \ ellnic/watcher3
Closing comments:
Credits: @scambra for the idea about using entrypoint to rename the existing config files and the Sickgear guys for inspiration drawn from their docker files.
@keppo070 If it's not too much trouble, could you stop your container, backup your config dir pull ellnic/watcher3 again and fire up using the new args and see if that works.
certainly not too much problem!
I used this docker-compose file
version: "2"
services:
watcher3:
image: ellnic/watcher3
container_name: watcher3
environment:
- APP_UID=1000
- APP_GID=1000
- UMASK_SET=022
volumes:
- /home/patrick/docker-config/watcher3:/config
- /var/usenet/tempdata/complete:/downloads
- /var/usenet/media/video/movie:/movies
ports:
- 9091:9090
restart: unless-stopped
it creates some files and poster folder. Backup folder is created by me
total 76
drwxrwxr-x 4 patrick patrick 4096 nov 22 15:42 ./
drwxrwxr-x 31 patrick patrick 4096 nov 21 19:32 ../
drwxrwxr-x 6 patrick patrick 4096 nov 22 15:36 backup/
-rw-r--r-- 1 patrick patrick 12079 nov 22 15:39 config.cfg
-rw-rw-r-- 1 patrick patrick 394 nov 22 15:33 docker-compose.yml
drwxr-xr-x 2 patrick patrick 4096 nov 22 15:39 posters/
-rw-r--r-- 1 patrick patrick 40960 nov 22 15:39 watcher.sqlite
docker ps -l
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
db6d69a33542 ellnic/watcher3 "/entrypoint.sh watc…" 7 minutes ago Up 7 minutes 9090/tcp watcher3_watcher3_run_56440f0c7600
But I cant connect to the webservice on 9090. I first had to stop sabnzbd which also uses 9090. The docker does not expose 9091. I can however bash into this docker and confirm its up, the first docker was continuously restarting.
Probably unrelated but when it was running I tried to import my media. Watcher was not able to import all my media files from /movies
. When I tried a single folder under /movies/folder
then it could open the media and import it.
If unrelated then I will create another issue
Ok, that's definitely an improvement. It also seems to be creating files with the correct permissions.
About the ports: As mentioned, I don't use compose, but from what I have just read, your compose file might have to use quotes around the port numbers like this:
ports:
- "9091:9090"
Try that and see if it correctly maps to 9091. I also run sabnzbd and use 9191 for watcher, but I use the -p flag with the docker command.
Can you exec into the container and check the permissions on the movies folder that you had an issue importing from? If the permissions are correct, this might be another issue to do with importing. I am guessing since you managed to import a single sub folder that the app user has permissions to view that folder, else it wouldn't have been able to import anything.
it working now with the docker-comopse.yml. I think I made a mistake with starting the docker previously. I can start the gui on 9091.
about the import. I exec into the docker
bash-4.4# ls -al
total 84
drwxr-xr-x 1 root root 4096 Nov 22 21:20 .
drwxr-xr-x 1 root root 4096 Nov 22 21:20 ..
-rwxr-xr-x 1 root root 0 Nov 22 21:20 .dockerenv
drwxr-xr-x 1 root root 4096 Nov 21 23:19 bin
drwxrwxr-x 4 1000 1000 4096 Nov 22 21:26 config
drwxr-xr-x 5 root root 340 Nov 22 21:23 dev
drwxrwxr-x 17 1000 1000 4096 Oct 24 12:02 downloads
-rwxr-xr-x 1 root root 786 Nov 21 23:19 entrypoint.sh
drwxr-xr-x 1 root root 4096 Nov 22 21:20 etc
drwxr-xr-x 2 root root 4096 Mar 6 2019 home
drwxr-xr-x 1 root root 4096 Mar 6 2019 lib
drwxr-xr-x 5 root root 4096 Mar 6 2019 media
drwxr-xr-x 2 root root 4096 Mar 6 2019 mnt
drwxrwxrwx 92 1000 1000 4096 Nov 21 21:00 movies
drwxr-xr-x 1 root root 4096 Nov 21 23:19 opt
dr-xr-xr-x 276 root root 0 Nov 22 21:23 proc
drwx------ 2 root root 4096 Mar 6 2019 root
drwxr-xr-x 2 root root 4096 Mar 6 2019 run
drwxr-xr-x 1 root root 4096 Nov 21 23:19 sbin
drwxr-xr-x 2 root root 4096 Mar 6 2019 srv
dr-xr-xr-x 13 root root 0 Nov 22 21:23 sys
drwxrwxrwt 2 root root 4096 Mar 6 2019 tmp
drwxr-xr-x 1 root root 4096 Nov 21 23:19 usr
drwxr-xr-x 1 root root 4096 Nov 21 23:19 var
When I ls -al the /movies I can see my media all in separate folders but I cannot import them all. It says no new movies found.
When I type /movies/movie.1940 and klik scan directory i does find 1 movie.
The import function seems to have trouble with a folder with VOB media files in it. I moved the folder with VOB to another location but the import does not work.
I guess this is not Docker related, because I can "see" the files and when selecting a single media folder the import works.
I have just altered the entrypoint script to revert the names of config files back to what they were before. I have done this because the database backups start to make a mess of the config folder so they are better in db/ IMO. As a plus point though, this will double as a renaming script for anyone moving from a non-hurricane based image (as I think pretty much all the ones I have seen use default names). I have also added the --poster and --plugins flags now the folder creation is fixed so that they are present in the /config folder as before.
FYI I just pulled the new docker and "lost" the 2 movies I had imported manually.
Apologies. I can see why you lost your movies. I will have it fixed soon. Standby.
Not sure about the movies folder issues. The perms are correct so I think this may be separate.
Ok, pull ellnic/watcher3 again but before you run it go to your config folder. You probably have:
Delete those first, then run it. I forgot to change the flags in the run command back to the original names. It was still using —userdata, which will use default names.
Just so you know, I very often tinker with my docker and break it. You may wish to use barbecuesauce instead.
If I make worthwhile changes they’ll be PR’d.
My docker is a bit like fedora. A tinker test bed that often breaks 😄 I’m pretty much not an expert with quite a lot so often get it wrong or notice I’ve typoed 15 times. My bash also sucks so this entry point script has been a PITA. I was trying to do something clever, but probably not a good idea for the first time at midnight :-)
its working!! and i still have all of my 2 movies. Great! Everything works, thanks for the effort. I thought your docker was the official docker because bbqsauce mentioned it.
I will make a separate issue about the import.
Before submitting an issue...
Enable debug logging in settings/server.
Repeat the actions required to reproduce the error.
Provide all information requested below. Issues without adequate information will be closed without response.
Commit: Operating System: Python Version [python -V]: What happened: ** Logs: If logs contain information you do not wish to share with the world you may email them to
watchernzb@gmail.com
. Include the issue number in the subject line of the message.