barbequesauce / Watcher3

Fork of nosmokingbandit/Watcher3 as the original seems inactive. PRs welcome, support is community based.
Other
69 stars 18 forks source link

Docker wont start #71

Closed keppo070 closed 4 years ago

keppo070 commented 4 years ago

Before submitting an issue...

Provide all information requested below. Issues without adequate information will be closed without response.

Commit: Operating System: Python Version [python -V]: What happened: ** Logs: If logs contain information you do not wish to share with the world you may email them to watchernzb@gmail.com. Include the issue number in the subject line of the message.

watcher3    | Creating database file /config/db/database.sqlite
watcher3    | Creating tables.
watcher3    | Opening SQL DB.
watcher3    | Traceback (most recent call last):
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2141, in _wrap_pool_connect
watcher3    |     return fn()
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 387, in connect
watcher3    |     return _ConnectionFairy._checkout(self)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 766, in _checkout
watcher3    |     fairy = _ConnectionRecord.checkout(pool)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 516, in checkout
watcher3    |     rec = pool._do_get()
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 1229, in _do_get
watcher3    |     return self._create_connection()
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 333, in _create_connection
watcher3    |     return _ConnectionRecord(self)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 461, in __init__
watcher3    |     self.__connect(first_connect_check=True)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 651, in __connect
watcher3    |     connection = pool._invoke_creator(self)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/engine/strategies.py", line 105, in connect
watcher3    |     return dialect.connect(*cargs, **cparams)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/engine/default.py", line 385, in connect
watcher3    |     return self.dbapi.connect(*cargs, **cparams)
watcher3    | sqlite3.OperationalError: unable to open database file
watcher3    |
watcher3    | The above exception was the direct cause of the following exception:
watcher3    |
watcher3    | Traceback (most recent call last):
watcher3    |   File "/opt/watcher3/core/sqldb.py", line 120, in __init__
watcher3    |     self.create_database(DB_NAME)
watcher3    |   File "/opt/watcher3/core/sqldb.py", line 139, in create_database
watcher3    |     self.metadata.create_all(self.engine)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/sql/schema.py", line 3848, in create_all
watcher3    |     tables=tables)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1922, in _run_visitor
watcher3    |     with self._optional_conn_ctx_manager(connection) as conn:
watcher3    |   File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
watcher3    |     return next(self.gen)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1915, in _optional_conn_ctx_manager
watcher3    |     with self.contextual_connect() as conn:
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2106, in contextual_connect
watcher3    |     self._wrap_pool_connect(self.pool.connect, None),
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2145, in _wrap_pool_connect
watcher3    |     e, dialect, self)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1456, in _handle_dbapi_exception_noconnection
watcher3    |     exc_info
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/util/compat.py", line 202, in raise_from_cause
watcher3    |     reraise(type(exception), exception, tb=exc_tb, cause=cause)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/util/compat.py", line 185, in reraise
watcher3    |     raise value.with_traceback(tb)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2141, in _wrap_pool_connect
watcher3    |     return fn()
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 387, in connect
watcher3    |     return _ConnectionFairy._checkout(self)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 766, in _checkout
watcher3    |     fairy = _ConnectionRecord.checkout(pool)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 516, in checkout
watcher3    |     rec = pool._do_get()
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 1229, in _do_get
watcher3    |     return self._create_connection()
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 333, in _create_connection
watcher3    |     return _ConnectionRecord(self)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 461, in __init__
watcher3    |     self.__connect(first_connect_check=True)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 651, in __connect
watcher3    |     connection = pool._invoke_creator(self)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/engine/strategies.py", line 105, in connect
watcher3    |     return dialect.connect(*cargs, **cparams)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/engine/default.py", line 385, in connect
watcher3    |     return self.dbapi.connect(*cargs, **cparams)
watcher3    | sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
watcher3    | Traceback (most recent call last):
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2141, in _wrap_pool_connect
watcher3    |     return fn()
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 387, in connect
watcher3    |     return _ConnectionFairy._checkout(self)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 766, in _checkout
watcher3    |     fairy = _ConnectionRecord.checkout(pool)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 516, in checkout
watcher3    |     rec = pool._do_get()
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 1229, in _do_get
watcher3    |     return self._create_connection()
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 333, in _create_connection
watcher3    |     return _ConnectionRecord(self)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 461, in __init__
watcher3    |     self.__connect(first_connect_check=True)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 651, in __connect
watcher3    |     connection = pool._invoke_creator(self)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/engine/strategies.py", line 105, in connect
watcher3    |     return dialect.connect(*cargs, **cparams)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/engine/default.py", line 385, in connect
watcher3    |     return self.dbapi.connect(*cargs, **cparams)
watcher3    | sqlite3.OperationalError: unable to open database file
watcher3    |
watcher3    | The above exception was the direct cause of the following exception:
watcher3    |
watcher3    | Traceback (most recent call last):
watcher3    |   File "/opt/watcher3/watcher.py", line 84, in <module>
watcher3    |     core.sql = sqldb.SQL()
watcher3    |   File "/opt/watcher3/core/sqldb.py", line 120, in __init__
watcher3    |     self.create_database(DB_NAME)
watcher3    |   File "/opt/watcher3/core/sqldb.py", line 139, in create_database
watcher3    |     self.metadata.create_all(self.engine)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/sql/schema.py", line 3848, in create_all
watcher3    |     tables=tables)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1922, in _run_visitor
watcher3    |     with self._optional_conn_ctx_manager(connection) as conn:
watcher3    |   File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
watcher3    |     return next(self.gen)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1915, in _optional_conn_ctx_manager
watcher3    |     with self.contextual_connect() as conn:
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2106, in contextual_connect
watcher3    |     self._wrap_pool_connect(self.pool.connect, None),
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2145, in _wrap_pool_connect
watcher3    |     e, dialect, self)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1456, in _handle_dbapi_exception_noconnection
watcher3    |     exc_info
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/util/compat.py", line 202, in raise_from_cause
watcher3    |     reraise(type(exception), exception, tb=exc_tb, cause=cause)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/util/compat.py", line 185, in reraise
watcher3    |     raise value.with_traceback(tb)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2141, in _wrap_pool_connect
watcher3    |     return fn()
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 387, in connect
watcher3    |     return _ConnectionFairy._checkout(self)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 766, in _checkout
watcher3    |     fairy = _ConnectionRecord.checkout(pool)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 516, in checkout
watcher3    |     rec = pool._do_get()
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 1229, in _do_get
watcher3    |     return self._create_connection()
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 333, in _create_connection
watcher3    |     return _ConnectionRecord(self)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 461, in __init__
watcher3    |     self.__connect(first_connect_check=True)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/pool.py", line 651, in __connect
watcher3    |     connection = pool._invoke_creator(self)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/engine/strategies.py", line 105, in connect
watcher3    |     return dialect.connect(*cargs, **cparams)
watcher3    |   File "/opt/watcher3/lib/sqlalchemy/engine/default.py", line 385, in connect
watcher3    |     return self.dbapi.connect(*cargs, **cparams)
watcher3    | sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
ellnic commented 4 years ago

Looks like the database is either not being created, or cannot be accessed. This results in a cascade of errors.

Can you please provide:

  1. The docker command used to run the container
  2. Your OS, e.g Debian 10, Ubuntu 18.04
  3. Your python version
  4. Details of the config dir ls -l /path/to/config/dir
  5. Group info for your user groups [username]
keppo070 commented 4 years ago
  1. version: "2" services: watcher3: image: ellnic/watcher3 container_name: watcher3 environment:

    • PUID=1000
    • PGID=1000
    • UMASK_SET=022 volumes:
    • /home/patrick/docker-config/watcher3:/config
    • /var/usenet/tempdata/complete:/downloads
    • /var/usenet/media/video/movie:/movies ports:
    • 9091:9090 restart: unless-stopped
  2. Distributor ID: Ubuntu Description: Ubuntu 18.04.3 LTS Release: 18.04 Codename: bionic

  3. Python 2.7.15+ (default, Oct 7 2019, 17:39:04) [GCC 7.4.0] on linux2 Type "help", "copyright", "credits" or "license" for more information.

  4. total 4 -rw-rw-r-- 1 patrick patrick 388 nov 16 09:53 docker-compose.yml

  5. patrick adm cdrom sudo dip plugdev lpadmin sambashare libvirt nfsgrp smbgrp

ellnic commented 4 years ago

I also run Bionic, so I know it’s workable. Let’s get this running for you.

Your config folder is empty so it seems the errors you see in the log are because watcher is unable to create the database to start. This is probably permissions related.

You don’t appear to be in the docker group.

Can you run:

sudo usermod -a patrick -G docker

Then logout and back in and try again.

Also, are you running the docker from the bionic repos or have you added the docker official repo?

keppo070 commented 4 years ago

I used the usermod but same problem. I SSH into ubuntu, but I dont think thats a problem.

I checked the error and it is the same as above. Still nothing in the config folder. I tried to run the docker without using docker-compose but that gave the same result.

Dont know exactly how to check which repo i used. I ran

apt list --installed | grep docker

result: docker-ce/bionic,now 5:19.03.5~3-0~ubuntu-bionic amd64 [installed] docker-ce-cli/bionic,now 5:19.03.5~3-0~ubuntu-bionic amd64 [installed,automatic]

I guess i'm using the bionic repo.

I have several other docker-containers running. Most of them are from linuxserver. I start them much the same way as I do with the ellnic/watcher3 container.

ellnic commented 4 years ago

If you don't remember installing from the Docker repo, then you almost certainly didn't. You need to manually add their repo.

Try this first:

sudo chmod -R 755 /path/to/watcher/config/folder

then

sudo chown -R patrick:patrick /path/to/watcher/config/folder

Delete the container and recreate without the UMASK, leave all else. Does it work now?

Note: Compose shouldn't be doing it, although I don't use it. I just run:

docker run -d --name=watcher3 -v /mnt/Docker/docker-configs/watcher3:/config -v /mnt/Tank/:/Tank -e PGID=1001 -e PUID=1000 -p 9191:9090 --restart unless-stopped ellnic/watcher3 NB: Group 1001 is a general media group, so I don't use 1000, but it shouldn't make a diff.

keppo070 commented 4 years ago

ls -al on /home/patrick/docker-config gives me

total 12
drwxrwxr-x  2 patrick patrick 4096 nov 16 09:53 .
drwxrwxr-x 30 patrick patrick 4096 nov 16 09:32 ..
-rw-rw-r--  1 patrick patrick  388 nov 16 09:53 docker-compose.yml

I mount the config in my homefolder. That should give me enough rights.

I tried directly without compose (I just left out the -d) sudo docker run --name=watcher3 -v /home/patrick/docker-config/watcher3:/config -v /var/usenet/tempdata/complete:/downloads -v /var/usenet/media/video/movie:/movies -e PUID=1000 -e PGID=1000 -p 9091:9090 --restart unless-stopped ellnic/watcher3

Same result. Opening SQL DB and then the traceback starts.

I removed the images. I saw that I had a docker from bbqsauce and removed that one also. Started over but nothing. Same result.

I noticed just now that there is a slight difference. I used both time the same cmdline.

in the last error dump after a rmi and a pull

Creating database file /config/db/database.sqlite
Creating tables.
Opening SQL DB.
Traceback

Before it was only

Opening SQL DB.
Traceback

Nothing is created in config thou

ellnic commented 4 years ago

Hrm. This isn’t the behaviour I get. Can you tell me if you get the same result with:

sudo docker run --name=watcher3-test -v /tmp/docker-test:/config -e PUID=1000 -e PGID=1000 -p 9091:9090 --restart unless-stopped ellnic/watcher3

keppo070 commented 4 years ago

sudo docker run --name=watcher3-test -v /tmp/docker-test:/config -e PUID=1000 -e PGID=1000 -p 9091:9090 --restart unless-stopped ellnic/watcher3

Creating database file /config/db/database.sqlite Creating tables. Opening SQL DB. Traceback (most recent call last): File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2141, in _wrap_pool_connect return fn() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 387, in connect return _ConnectionFairy._checkout(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 766, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 516, in checkout rec = pool._do_get() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 1229, in _do_get return self._create_connection() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 333, in _create_connection return _ConnectionRecord(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 461, in init self.connect(first_connect_check=True) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 651, in connect connection = pool._invoke_creator(self) File "/opt/watcher3/lib/sqlalchemy/engine/strategies.py", line 105, in connect return dialect.connect(*cargs, *cparams) File "/opt/watcher3/lib/sqlalchemy/engine/default.py", line 385, in connect return self.dbapi.connect(cargs, **cparams) sqlite3.OperationalError: unable to open database file

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/opt/watcher3/core/sqldb.py", line 120, in init self.create_database(DB_NAME) File "/opt/watcher3/core/sqldb.py", line 139, in create_database self.metadata.create_all(self.engine) File "/opt/watcher3/lib/sqlalchemy/sql/schema.py", line 3848, in create_all tables=tables) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1922, in _run_visitor with self._optional_conn_ctx_manager(connection) as conn: File "/usr/lib/python3.6/contextlib.py", line 81, in enter return next(self.gen) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1915, in _optional_conn_ctx_manager with self.contextual_connect() as conn: File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2106, in contextual_connect self._wrap_pool_connect(self.pool.connect, None), File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2145, in _wrap_pool_connect e, dialect, self) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1456, in _handle_dbapi_exception_noconnection exc_info File "/opt/watcher3/lib/sqlalchemy/util/compat.py", line 202, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/opt/watcher3/lib/sqlalchemy/util/compat.py", line 185, in reraise raise value.with_traceback(tb) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2141, in _wrap_pool_connect return fn() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 387, in connect return _ConnectionFairy._checkout(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 766, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 516, in checkout rec = pool._do_get() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 1229, in _do_get return self._create_connection() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 333, in _create_connection return _ConnectionRecord(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 461, in init self.connect(first_connect_check=True) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 651, in connect connection = pool._invoke_creator(self) File "/opt/watcher3/lib/sqlalchemy/engine/strategies.py", line 105, in connect return dialect.connect(*cargs, cparams) File "/opt/watcher3/lib/sqlalchemy/engine/default.py", line 385, in connect return self.dbapi.connect(*cargs, *cparams) sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file Traceback (most recent call last): File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2141, in _wrap_pool_connect return fn() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 387, in connect return _ConnectionFairy._checkout(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 766, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 516, in checkout rec = pool._do_get() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 1229, in _do_get return self._create_connection() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 333, in _create_connection return _ConnectionRecord(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 461, in init self.connect(first_connect_check=True) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 651, in connect connection = pool._invoke_creator(self) File "/opt/watcher3/lib/sqlalchemy/engine/strategies.py", line 105, in connect return dialect.connect(cargs, cparams) File "/opt/watcher3/lib/sqlalchemy/engine/default.py", line 385, in connect return self.dbapi.connect(*cargs, **cparams) sqlite3.OperationalError: unable to open database file

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/opt/watcher3/watcher.py", line 84, in core.sql = sqldb.SQL() File "/opt/watcher3/core/sqldb.py", line 120, in init self.create_database(DB_NAME) File "/opt/watcher3/core/sqldb.py", line 139, in create_database self.metadata.create_all(self.engine) File "/opt/watcher3/lib/sqlalchemy/sql/schema.py", line 3848, in create_all tables=tables) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1922, in _run_visitor with self._optional_conn_ctx_manager(connection) as conn: File "/usr/lib/python3.6/contextlib.py", line 81, in enter return next(self.gen) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1915, in _optional_conn_ctx_manager with self.contextual_connect() as conn: File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2106, in contextual_connect self._wrap_pool_connect(self.pool.connect, None), File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2145, in _wrap_pool_connect e, dialect, self) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1456, in _handle_dbapi_exception_noconnection exc_info File "/opt/watcher3/lib/sqlalchemy/util/compat.py", line 202, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/opt/watcher3/lib/sqlalchemy/util/compat.py", line 185, in reraise raise value.with_traceback(tb) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2141, in _wrap_pool_connect return fn() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 387, in connect return _ConnectionFairy._checkout(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 766, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 516, in checkout rec = pool._do_get() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 1229, in _do_get return self._create_connection() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 333, in _create_connection return _ConnectionRecord(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 461, in init self.connect(first_connect_check=True) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 651, in connect connection = pool._invoke_creator(self) File "/opt/watcher3/lib/sqlalchemy/engine/strategies.py", line 105, in connect return dialect.connect(*cargs, *cparams) File "/opt/watcher3/lib/sqlalchemy/engine/default.py", line 385, in connect return self.dbapi.connect(cargs, **cparams) sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file

keppo070 commented 4 years ago

it seems I have possibly installed from docker because apt-cache policy give also an entry for a docker repository. I am not so sure I have installed from the ubuntu repository.

 500 https://download.docker.com/linux/ubuntu bionic/stable amd64 Packages
     release o=Docker,a=bionic,l=Docker CE,c=stable,b=amd64
     origin download.docker.com
ellnic commented 4 years ago

Ok, try without the PUID and PGID. This will run entirely as root:

sudo docker run --name=watcher3-test -v /tmp/docker-test:/config -p 9091:9090 --restart unless-stopped ellnic/watcher3

keppo070 commented 4 years ago

it seems to be shorter. It took it bit longer but I got the error msg

sudo docker run --name=watcher3-test -v /tmp/docker-test:/config -p 9091:9090 --restart unless-stopped ellnic/watcher3 Creating database file /config/db/database.sqlite Creating tables. Opening SQL DB. Traceback (most recent call last): File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2141, in _wrap_pool_connect return fn() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 387, in connect return _ConnectionFairy._checkout(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 766, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 516, in checkout rec = pool._do_get() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 1229, in _do_get return self._create_connection() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 333, in _create_connection return _ConnectionRecord(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 461, in init self.connect(first_connect_check=True) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 651, in connect connection = pool._invoke_creator(self) File "/opt/watcher3/lib/sqlalchemy/engine/strategies.py", line 105, in connect return dialect.connect(*cargs, *cparams) File "/opt/watcher3/lib/sqlalchemy/engine/default.py", line 385, in connect return self.dbapi.connect(cargs, **cparams) sqlite3.OperationalError: unable to open database file

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/opt/watcher3/core/sqldb.py", line 120, in init self.create_database(DB_NAME) File "/opt/watcher3/core/sqldb.py", line 139, in create_database self.metadata.create_all(self.engine) File "/opt/watcher3/lib/sqlalchemy/sql/schema.py", line 3848, in create_all tables=tables) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1922, in _run_visitor with self._optional_conn_ctx_manager(connection) as conn: File "/usr/lib/python3.6/contextlib.py", line 81, in enter return next(self.gen) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1915, in _optional_conn_ctx_manager with self.contextual_connect() as conn: File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2106, in contextual_connect self._wrap_pool_connect(self.pool.connect, None), File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2145, in _wrap_pool_connect e, dialect, self) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1456, in _handle_dbapi_exception_noconnection exc_info File "/opt/watcher3/lib/sqlalchemy/util/compat.py", line 202, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/opt/watcher3/lib/sqlalchemy/util/compat.py", line 185, in reraise raise value.with_traceback(tb) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2141, in _wrap_pool_connect return fn() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 387, in connect return _ConnectionFairy._checkout(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 766, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 516, in checkout rec = pool._do_get() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 1229, in _do_get return self._create_connection() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 333, in _create_connection return _ConnectionRecord(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 461, in init self.connect(first_connect_check=True) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 651, in connect connection = pool._invoke_creator(self) File "/opt/watcher3/lib/sqlalchemy/engine/strategies.py", line 105, in connect return dialect.connect(*cargs, cparams) File "/opt/watcher3/lib/sqlalchemy/engine/default.py", line 385, in connect return self.dbapi.connect(*cargs, *cparams) sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file Traceback (most recent call last): File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2141, in _wrap_pool_connect return fn() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 387, in connect return _ConnectionFairy._checkout(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 766, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 516, in checkout rec = pool._do_get() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 1229, in _do_get return self._create_connection() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 333, in _create_connection return _ConnectionRecord(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 461, in init self.connect(first_connect_check=True) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 651, in connect connection = pool._invoke_creator(self) File "/opt/watcher3/lib/sqlalchemy/engine/strategies.py", line 105, in connect return dialect.connect(cargs, cparams) File "/opt/watcher3/lib/sqlalchemy/engine/default.py", line 385, in connect return self.dbapi.connect(*cargs, **cparams) sqlite3.OperationalError: unable to open database file

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/opt/watcher3/watcher.py", line 84, in core.sql = sqldb.SQL() File "/opt/watcher3/core/sqldb.py", line 120, in init self.create_database(DB_NAME) File "/opt/watcher3/core/sqldb.py", line 139, in create_database self.metadata.create_all(self.engine) File "/opt/watcher3/lib/sqlalchemy/sql/schema.py", line 3848, in create_all tables=tables) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1922, in _run_visitor with self._optional_conn_ctx_manager(connection) as conn: File "/usr/lib/python3.6/contextlib.py", line 81, in enter return next(self.gen) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1915, in _optional_conn_ctx_manager with self.contextual_connect() as conn: File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2106, in contextual_connect self._wrap_pool_connect(self.pool.connect, None), File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2145, in _wrap_pool_connect e, dialect, self) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 1456, in _handle_dbapi_exception_noconnection exc_info File "/opt/watcher3/lib/sqlalchemy/util/compat.py", line 202, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/opt/watcher3/lib/sqlalchemy/util/compat.py", line 185, in reraise raise value.with_traceback(tb) File "/opt/watcher3/lib/sqlalchemy/engine/base.py", line 2141, in _wrap_pool_connect return fn() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 387, in connect return _ConnectionFairy._checkout(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 766, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 516, in checkout rec = pool._do_get() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 1229, in _do_get return self._create_connection() File "/opt/watcher3/lib/sqlalchemy/pool.py", line 333, in _create_connection return _ConnectionRecord(self) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 461, in init self.connect(first_connect_check=True) File "/opt/watcher3/lib/sqlalchemy/pool.py", line 651, in connect connection = pool._invoke_creator(self) File "/opt/watcher3/lib/sqlalchemy/engine/strategies.py", line 105, in connect return dialect.connect(*cargs, *cparams) File "/opt/watcher3/lib/sqlalchemy/engine/default.py", line 385, in connect return self.dbapi.connect(cargs, **cparams) sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file

ellnic commented 4 years ago

Ok, I’m kinda stumped here. I’m by no means a docker expert. Do you get the same results with:

sudo docker run --name=watcher3-testbbq -v /tmp/docker-testbbq:/config -p 9091:9090 --restart unless-stopped barbequesauce/watcher3

keppo070 commented 4 years ago

sudo dpkg -l | grep -i docker ii docker-ce 5:19.03.5~3-0~ubuntu-bionic amd64 Docker: the open-source application container engine ii docker-ce-cli 5:19.03.5~3-0~ubuntu-bionic amd64 Docker CLI: the open-source application container engine

It seems I'm running the Ubuntu version. Dont know if this is still important for you

BTW: I appreciate the time your spending on this, but if this is related only to my setup. Feel free to abandon the issue.

keppo070 commented 4 years ago

sudo docker run --name=watcher3-testbbq -v /tmp/docker-testbbq:/config -p 9091:9090 --restart unless-stopped barbequesauce/watcher3 Creating database file /config/db/database.sqlite Creating tables. Opening SQL DB. Traceback....

ellnic commented 4 years ago

It’s not a problem. :-) it’s as much a learning curve for me. I am more interested in finding out what it is in case this pops up again.

The only major difference I can see here is that I am using the official docker repos. I’ve never used the bionic versions because the docker own repos are naturally more up to date. Maybe the next step would be to move to that? It’s not impossible that some change they’ve made to the engine could be causing this.

Have a look here: https://docs.docker.com/install/linux/docker-ce/ubuntu/

Note that it specifically runs through removal of distro own versions then installing theirs.

Try this and see if that helps.

keppo070 commented 4 years ago

when I run docker version I get this

Client: Docker Engine - Community
 Version:           19.03.5
 API version:       1.40
 Go version:        go1.12.12
 Git commit:        633a0ea838
 Built:             Wed Nov 13 07:29:52 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.5
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.12
  Git commit:       633a0ea838
  Built:            Wed Nov 13 07:28:22 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.10
  GitCommit:        b34a5c8af56e510852c35414db4c1f4fa6172339
 runc:
  Version:          1.0.0-rc8+dev
  GitCommit:        3e425f80a8c931f88e6d94a8c831b9d5aa481657
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

So is this the ubuntu version or the docker version? I will try the docker image on a virtualbox first. To be honest I am not looking forward to replace my docker installation. Can you confirm which version this is?

ellnic commented 4 years ago

Ah.... you are definitely running the docker repo version.

I’ve just managed to reproduce this. Give me a second. Just changing the build file.

ellnic commented 4 years ago

Ok, that hasn't made a difference. Trying one more thing and if this doesn't work I might have to talk to the guys on the docker forums. Standby.

ellnic commented 4 years ago

Ok, still no change. This seems to be a recent change, as I couldn't reproduce before and now I can. Looks like docker was updated on Nov 13th. So something must have changed. I'm sure it's something to do with permissions / root.

It's late here now (I'm in the UK) but give me a couple of days to figure this out and I will report back ASAP.

ellnic commented 4 years ago

https://forums.docker.com/t/unable-to-write-config-files-permissions-issue/84626

ellnic commented 4 years ago

@barbequesauce @scambra

Might have some bad news. I have made several changes to the docker file and am still having no luck here, so I tried running using python instead, but mimicking the docker run args. I used sudo for the sake of eliminating permission issues:

sudo python3 watcher.py -c /home/ellnic/Desktop/watcher-test/config/watcher.cfg -l /home/ellnic/Desktop/watcher-test/config/logs/ --db /home/ellnic/Desktop/watcher-test/config/db/database.sqlite --plugins /home/ellnic/Desktop/watcher-test/config/plugins/ --posters /home/ellnic/Desktop/watcher-test/config/posters

And unfortunately, I am getting similar errors as the above reported by @keppo070

So it looks like it's not a docker error, but something to do with the way flags are handled. If I specify no flags, it runs fine. Can either of you reproduce this error with these or similar arguments? Has anything changed that might affect this?

scambra commented 4 years ago

Error is raising on self.metadata.create_all(self.engine) line, which creates tables. I have added new fields to tables recently, and I got no issue updating DB, but it may be related. I have tried renaming config directory, to start with new one, and I had no issue.

I'm using docker, but I use own build file, based in old file from linuxserver (https://github.com/linuxserver-archive/docker-watcher-stale), because I run in raspberry pi and other images didn't work. Maybe my docker image is using a different sqlite version, it's based on lsiobase/alpine:3.8:

root@027ef64b34b8:/# apk list *sqlite*
sqlite-libs-3.25.3-r1 armhf {sqlite} (Public-Domain) [installed]
scambra commented 4 years ago

On Nov 15th, I added these 2 columns:

                                    sqla.Column('download_progress', sqla.INT),
                                    sqla.Column('download_time', sqla.TIMESTAMP)

No column was using TIMESTAMP type before, can it be related? It could probably be changed to INT without affecting current code.

@ellnic can you try changing download_time to sqla.INT? Is it failing?

ellnic commented 4 years ago

The docker image I put together also uses Alpine 3.8, although not ARM.

There is a minor change in the sqlite version:

sqlite-libs-3.25.3-r2 x86_64 {sqlite} (Public-Domain) [installed]

I am testing using my master which does not have the sqla.Column('download_time', sqla.TIMESTAMP) code yet.

I have just checked using barbequesauce master and the results are the same when:

sqla.Column('download_time', sqla.TIMESTAMP)

is changed to

sqla.Column('download_time', sqla.INT)

I am not sure if that is the cause. If no flags are specified, ie. just running:

python3 watcher.py

It starts fine, and places the required files in the userdata folder. It is only when we specify some flags that the error occurs. Eg:

sudo python3 watcher.py -c /home/ellnic/Desktop/watcher-test/config/watcher.cfg

It doesn't seem to matter what flag... --db, -c, -l etc. It's when the files are meant to be somewhere other than the userdata folder.

Edit:

Specifying just the --userdata flag works:

python3 watcher.py --userdata /home/ellnic/Desktop/watcher-test

But it creates files with default names:

config.cfg watcher.sqlite

Whereas in our docker we use:

watcher.cfg
/db/database.sqlite

This was originally done to facilitate anyone wanting to move from hurricane/watcher3, which at the time seemed to have more activity than the others.

I could change the dockerfile to just use --userdata as an intermediate fix, but it would require manual editing of user files to keep their existing config and db. :-/

Edit: I am guessing that since the error exists in my master as well as this one, it's likely to be something from here: https://github.com/ellnic/Watcher3/pull/15 or before, so prior to 21st October.

scambra commented 4 years ago

My docker file has a start script wich runs watcher with this command:

exec \
        s6-setuidgid abc python3 watcher.py --db /config/db/database.sqlite \
        --conf /config/config.cfg --log /config/logs/ --plugins /config/plugins/ --posters /config/posters

So it uses flags too. Linuxserver docker image doesn't use CMD in Dockerfile, it has /etc/services.d/watcher/run script, which calls exec to run watcher. Using something like that, script could move files from previous locations to new one, and rename them to new name.

I don't know if it's related to older docker version, I haven't updated it for a while:

Client: Docker Engine - Community
 Version:           19.03.1
 API version:       1.40
 Go version:        go1.12.5
 Git commit:        74b1e89
 Built:             Thu Jul 25 21:32:16 2019
 OS/Arch:           linux/arm
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.1
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.5
  Git commit:       74b1e89
  Built:            Thu Jul 25 21:26:16 2019
  OS/Arch:          linux/arm
  Experimental:     false
 containerd:
  Version:          1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Stretch repo for raspbian has 19.03.4 available, but not 19.03.5

ellnic commented 4 years ago

It's nothing to do with docker, because I can get the error without docker.

If I pull a fresh barbequesauce master, and run the same command as your docker from terminal:

python3 watcher.py --db /config/db/database.sqlite --conf /config/config.cfg --log /config/logs/ --plugins /config/plugins/ --posters /config/posters

I get:

Creating database file /config/db/database.sqlite
Creating tables.
Opening SQL DB.
Traceback (most recent call last):
[truncated]

Can you try this on your end? Just run from terminal.

I know we can probably run a script to rename the config files then switch to the --userdata flag for the docker, but that doesn't solve the issue of the other flags not working.

ellnic commented 4 years ago

More info. It only happens when there are no files present. If I take my docker config files and place them in the copy on the desktop and run:

~/Desktop/Watcher3-master » sudo python3 watcher.py --db test-config/db/database.sqlite --conf test-config/watcher.cfg --log test-config/logs/ --plugins test-config/plugins/ --posters test-config/posters -p 1234

I get a success:

Connected to database sqlite:///test-config/db/database.sqlite Config file found, merging any new options. Clearing Mako cache. [21/Nov/2019:17:36:05] ENGINE Listening for SIGTERM. [21/Nov/2019:17:36:05] ENGINE Listening for SIGHUP. [21/Nov/2019:17:36:05] ENGINE Listening for SIGUSR1. [21/Nov/2019:17:36:05] ENGINE Bus STARTING [21/Nov/2019:17:36:05] ENGINE Serving on http://0.0.0.0:1234 [21/Nov/2019:17:36:05] ENGINE Bus STARTED

@scambra Can you test my specifying a new location for your docker config, and see if it starts?

Edit:

python3 watcher.py fails to create new files

but:

python3 watcher.py --userdata userdata succeeds

barbequesauce commented 4 years ago

Insert confused Scooby-Doo sound here.

ellnic commented 4 years ago

Yeah, not quite sure what has happened.

@keppo070 I have created a fresh new database and config files. Please can you download this: https://www.dropbox.com/s/d72wp41365hjtgg/config.zip?dl=0

And put in your config folder, then tell me if the container starts.

keppo070 commented 4 years ago

I put the config in my config folder. Must problay edit some stuff but it is starting up!

watcher3    | [21/Nov/2019:18:03:09] ENGINE Listening for SIGTERM.
watcher3    | [21/Nov/2019:18:03:09] ENGINE Listening for SIGHUP.
watcher3    | [21/Nov/2019:18:03:09] ENGINE Listening for SIGUSR1.
watcher3    | [21/Nov/2019:18:03:09] ENGINE Bus STARTING
watcher3    | CherryPy Checker:
watcher3    | dir is an absolute path, even though a root is provided.
watcher3    | section: [/posters]
watcher3    | root: '/opt/watcher3'
watcher3    | dir: '/config/posters'
watcher3    |
watcher3    | CherryPy Checker:
watcher3    | dir is an absolute path, even though a root is provided.
watcher3    | section: [/posters/missing_poster.jpg]
watcher3    | root: '/opt/watcher3'
watcher3    | dir: '/config/posters'
watcher3    |
watcher3    | CherryPy Checker:
watcher3    | dir is an absolute path, even though a root is provided.
watcher3    | section: [/logs]
watcher3    | root: '/opt/watcher3'
watcher3    | dir: '/config/logs/'
watcher3    |
watcher3    | [21/Nov/2019:18:03:09] ENGINE Serving on http://0.0.0.0:9090
watcher3    | [21/Nov/2019:18:03:09] ENGINE Bus STARTED
ellnic commented 4 years ago

Fantastic.

So the problem exists when no config files are present unless specifying the --userdata flag.

ellnic commented 4 years ago

I am working on a fix for the docker. It won't fix the flags issue, but it will solve the docker in the meantime.

ellnic commented 4 years ago

Ok so there we go. It took a while but I have fixed the ellnic docker. I have added an entrypoint script that looks for database and config files by the old names and renames/moves that to the defaults. I changed the run command to just use the --userdata flag and point to the /config folder.

This results in:

  1. No loss of data for existing users and no need to change file names themselves.
  2. Correct creation of the required files when they do not exist (since the --userdata flag is working) - solving this issue.

Notes: The entrypoint script attempts to remove the old db folder, but I used rmdir specifically because it will NOT remove any dir that isn't empty (in case the mv fails for some reason). In my case, the rmdir fails because I have database backups that are not moved out of that folder.

I also changed the way the container runs. I have added support for APP_UID and APP_GID instead of PUID and PGID. If the entry script sees that APP_UID or APP_GID have been specified, it will chown the /opt/watcher and /config dirs to those IDs. If they aren't specified, the chown will be to 0:0.

I have altered my readme, but I will include a new example run command here now:

docker run -d \ --name=watcher3 \ -v /path/to/config/:/config \ -v /path/to/downloads/:/downloads \ -v /path/to/movies/:/movies \ -e UMASK_SET=022 \ -e APP_UID=1000 -e APP_GID=1000 \ -p 9090:9090 \ ellnic/watcher3

Closing comments:

  1. The remaining flags that aren't working properly still need looking at. I will open another issue for this.
  2. By default, the plugin and logs dirs reside in /opt/watcher3 when just using the --userdata flag. Whereas before we manually specified that they would be in the /config folder with -l and --plugins flags. If the default for these is changed to the userdata folder, they will automatically revert to being in the /config folder with just the use of --userdata. This may disrupt users who are just running via python though so maybe best to sort those flags then pop them on the end of the CMD line.

Credits: @scambra for the idea about using entrypoint to rename the existing config files and the Sickgear guys for inspiration drawn from their docker files.

ellnic commented 4 years ago

@keppo070 If it's not too much trouble, could you stop your container, backup your config dir pull ellnic/watcher3 again and fire up using the new args and see if that works.

keppo070 commented 4 years ago

certainly not too much problem!

I used this docker-compose file

version: "2"
services:
  watcher3:
    image: ellnic/watcher3
    container_name: watcher3
    environment:
      - APP_UID=1000
      - APP_GID=1000
      - UMASK_SET=022
    volumes:
      - /home/patrick/docker-config/watcher3:/config
      - /var/usenet/tempdata/complete:/downloads
      - /var/usenet/media/video/movie:/movies
    ports:
      - 9091:9090
    restart: unless-stopped

it creates some files and poster folder. Backup folder is created by me

total 76
drwxrwxr-x  4 patrick patrick  4096 nov 22 15:42 ./
drwxrwxr-x 31 patrick patrick  4096 nov 21 19:32 ../
drwxrwxr-x  6 patrick patrick  4096 nov 22 15:36 backup/
-rw-r--r--  1 patrick patrick 12079 nov 22 15:39 config.cfg
-rw-rw-r--  1 patrick patrick   394 nov 22 15:33 docker-compose.yml
drwxr-xr-x  2 patrick patrick  4096 nov 22 15:39 posters/
-rw-r--r--  1 patrick patrick 40960 nov 22 15:39 watcher.sqlite

docker ps -l

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
db6d69a33542        ellnic/watcher3     "/entrypoint.sh watc…"   7 minutes ago       Up 7 minutes        9090/tcp            watcher3_watcher3_run_56440f0c7600

But I cant connect to the webservice on 9090. I first had to stop sabnzbd which also uses 9090. The docker does not expose 9091. I can however bash into this docker and confirm its up, the first docker was continuously restarting.

keppo070 commented 4 years ago

Probably unrelated but when it was running I tried to import my media. Watcher was not able to import all my media files from /movies. When I tried a single folder under /movies/folder then it could open the media and import it.

If unrelated then I will create another issue

ellnic commented 4 years ago

Ok, that's definitely an improvement. It also seems to be creating files with the correct permissions.

About the ports: As mentioned, I don't use compose, but from what I have just read, your compose file might have to use quotes around the port numbers like this:

ports: - "9091:9090"

Try that and see if it correctly maps to 9091. I also run sabnzbd and use 9191 for watcher, but I use the -p flag with the docker command.

Can you exec into the container and check the permissions on the movies folder that you had an issue importing from? If the permissions are correct, this might be another issue to do with importing. I am guessing since you managed to import a single sub folder that the app user has permissions to view that folder, else it wouldn't have been able to import anything.

keppo070 commented 4 years ago

it working now with the docker-comopse.yml. I think I made a mistake with starting the docker previously. I can start the gui on 9091.

about the import. I exec into the docker

bash-4.4# ls -al
total 84
drwxr-xr-x    1 root     root          4096 Nov 22 21:20 .
drwxr-xr-x    1 root     root          4096 Nov 22 21:20 ..
-rwxr-xr-x    1 root     root             0 Nov 22 21:20 .dockerenv
drwxr-xr-x    1 root     root          4096 Nov 21 23:19 bin
drwxrwxr-x    4 1000     1000          4096 Nov 22 21:26 config
drwxr-xr-x    5 root     root           340 Nov 22 21:23 dev
drwxrwxr-x   17 1000     1000          4096 Oct 24 12:02 downloads
-rwxr-xr-x    1 root     root           786 Nov 21 23:19 entrypoint.sh
drwxr-xr-x    1 root     root          4096 Nov 22 21:20 etc
drwxr-xr-x    2 root     root          4096 Mar  6  2019 home
drwxr-xr-x    1 root     root          4096 Mar  6  2019 lib
drwxr-xr-x    5 root     root          4096 Mar  6  2019 media
drwxr-xr-x    2 root     root          4096 Mar  6  2019 mnt
drwxrwxrwx   92 1000     1000          4096 Nov 21 21:00 movies
drwxr-xr-x    1 root     root          4096 Nov 21 23:19 opt
dr-xr-xr-x  276 root     root             0 Nov 22 21:23 proc
drwx------    2 root     root          4096 Mar  6  2019 root
drwxr-xr-x    2 root     root          4096 Mar  6  2019 run
drwxr-xr-x    1 root     root          4096 Nov 21 23:19 sbin
drwxr-xr-x    2 root     root          4096 Mar  6  2019 srv
dr-xr-xr-x   13 root     root             0 Nov 22 21:23 sys
drwxrwxrwt    2 root     root          4096 Mar  6  2019 tmp
drwxr-xr-x    1 root     root          4096 Nov 21 23:19 usr
drwxr-xr-x    1 root     root          4096 Nov 21 23:19 var

When I ls -al the /movies I can see my media all in separate folders but I cannot import them all. It says no new movies found.

When I type /movies/movie.1940 and klik scan directory i does find 1 movie.

The import function seems to have trouble with a folder with VOB media files in it. I moved the folder with VOB to another location but the import does not work.

I guess this is not Docker related, because I can "see" the files and when selecting a single media folder the import works.

ellnic commented 4 years ago

I have just altered the entrypoint script to revert the names of config files back to what they were before. I have done this because the database backups start to make a mess of the config folder so they are better in db/ IMO. As a plus point though, this will double as a renaming script for anyone moving from a non-hurricane based image (as I think pretty much all the ones I have seen use default names). I have also added the --poster and --plugins flags now the folder creation is fixed so that they are present in the /config folder as before.

keppo070 commented 4 years ago

FYI I just pulled the new docker and "lost" the 2 movies I had imported manually.

ellnic commented 4 years ago

Apologies. I can see why you lost your movies. I will have it fixed soon. Standby.

Not sure about the movies folder issues. The perms are correct so I think this may be separate.

ellnic commented 4 years ago

Ok, pull ellnic/watcher3 again but before you run it go to your config folder. You probably have:

  1. Config.cfg
  2. Watcher.sqlite

Delete those first, then run it. I forgot to change the flags in the run command back to the original names. It was still using —userdata, which will use default names.

Just so you know, I very often tinker with my docker and break it. You may wish to use barbecuesauce instead.

If I make worthwhile changes they’ll be PR’d.

My docker is a bit like fedora. A tinker test bed that often breaks 😄 I’m pretty much not an expert with quite a lot so often get it wrong or notice I’ve typoed 15 times. My bash also sucks so this entry point script has been a PITA. I was trying to do something clever, but probably not a good idea for the first time at midnight :-)

keppo070 commented 4 years ago

its working!! and i still have all of my 2 movies. Great! Everything works, thanks for the effort. I thought your docker was the official docker because bbqsauce mentioned it.

I will make a separate issue about the import.