[Request] Importing Options

[jaketame]

Requesting that there are some other import options as Google will ban if too many requests for opening the file headers (It thinks its a full file download)

1) Import from Plex 2) Import from Radarr 3) Import from Couchpotato

Cheers

[nosmokingbandit]

I haven't used it myself so I don't know how thorough it is, but /u/mekon on Reddit made a Ruby script to import from CP.

https://gist.github.com/anonymous/5ce4df44efa338bb305cd3336e5ab7a7

Plex and Kodi imports are on the todo list, but I can't promise I'll get it done soon.

[seppi91]

Would be nice to get emby (https://emby.media/) on the todo list as well.

[nosmokingbandit]

Kodi and Plex added in 795c0fcbe8ffc4d270f2f0a9b65ceaad308bbfc7

I haven't had a chance to test Plex a ton, but it worked well enough when I did test it.

I'm leaving this open until Emby is added. Media center applications like this all seem to have poorly documented apis, so I'm going to work on a few other things before I dive into Emby.

[nosmokingbandit]

FYI, added CouchPotato importing in dc92db5ce6375f1b37b491f8471d52ec03a05d7b. I'll start work on Emby relatively soon.

[jaketame]

Nice work will give it a test for Plex. PlexPy has some good docs on the API available and even imports from Plex itself.

[nosmokingbandit]

I don't use plex, so I didn't add my entire library to it for testing -- only ~12 movies. Let me know how well it works for a larger library.

Plex is a little frustrating because it doesn't give an ID for movies when you query the whole library. In order to get an IMDB or TMDB number you have to ask for each movie individually, which can really bog things down on a huge library (especially if something goes wrong and you have to repeat it). I'm not super thrilled about the CSV method, but it is a whole lot easier on your server.

[jaketame]

Sorry for the delay, just got round to testing plex import; INFO 2017-05-25 11:23:44,567 cherrypy.error.139850005196248.error: [25/May/2017:11:23:44] HTTP sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: MOVIES [SQL: 'SELECT FROM MOVIES ORDER BY title ASC'] cursor.execute(statement, parameters) File "/app/watcher/lib/sqlalchemy/engine/default.py", line 462, in do_execute context) File "/app/watcher/lib/sqlalchemy/engine/base.py", line 1182, in _execute_context raise value.with_traceback(tb) File "/app/watcher/lib/sqlalchemy/util/compat.py", line 185, in reraise reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/app/watcher/lib/sqlalchemy/util/compat.py", line 202, in raise_from_cause exc_info File "/app/watcher/lib/sqlalchemy/engine/base.py", line 1393, in _handle_dbapi_exception context) File "/app/watcher/lib/sqlalchemy/engine/base.py", line 1189, in _execute_context statement, parameters File "/app/watcher/lib/sqlalchemy/engine/base.py", line 1097, in _execute_text return self._execute_text(object, multiparams, params) File "/app/watcher/lib/sqlalchemy/engine/base.py", line 939, in execute return connection.execute(statement, multiparams, params) File "/app/watcher/lib/sqlalchemy/engine/base.py", line 2058, in execute result = self.engine.execute(command) File "/app/watcher/core/sqldb.py", line 116, in execute result = self.execute(command) File "/app/watcher/core/sqldb.py", line 243, in get_user_movies library = [i['imdbid'] for i in sql.get_user_movies()] File "/app/watcher/core/library.py", line 263, in read_csv return json.dumps(library.ImportPlexLibrary.read_csv(csv_text)) File "/app/watcher/core/ajax.py", line 928, in upload_plex_csv return self.callable(*self.args, *self.kwargs) File "/app/watcher/lib/cherrypy/_cpdispatch.py", line 60, in call self.body = self.oldhandler(args, kwargs) File "/app/watcher/lib/cherrypy/lib/encoding.py", line 220, in call response.body = self.handler() File "/app/watcher/lib/cherrypy/_cprequest.py", line 670, in respond Traceback (most recent call last):

The above exception was the direct cause of the following exception:

sqlite3.OperationalError: no such table: MOVIES cursor.execute(statement, parameters) File "/app/watcher/lib/sqlalchemy/engine/default.py", line 462, in do_execute context) File "/app/watcher/lib/sqlalchemy/engine/base.py", line 1182, in _execute_context Traceback (most recent call last): ERROR 2017-05-25 11:23:44,566 cherrypy.error.139850005196248.error: [25/May/2017:11:23:44] HTTP

[nosmokingbandit]

Have you added movies via search? The Movies table is created before anything else happens, so I'm not sure what would cause it to not be found.

[jaketame]

Yes, couple of movies instead the DB, I opened up the database.sqlite too and it was there. I get Status 500 when I hit the import button

[nosmokingbandit]

That is the same method that is used to get the list of movies for the /library/status page, so it is odd that it wouldn't work when importing. I'm looking in to it. What version of Python are you using? There might be a namespace collision that is messing things up, which had changed somewhere in python 3's life.

[jaketame]

im using this https://github.com/linuxserver/docker-watcher

[nosmokingbandit]

I've never been able to get Docker working correctly (I always get errors that apparently nobody has ever seen) so I can't troubleshoot directly with that container.

In commit 2cd175b577b6d205dfbb7f1889d06369e14b93f2 I made a few major changes to the database module. Its a long shot but take a crack at it after updating to that commit and see if anything different happens.

[jaketame]

Updated however on updating it fails to start

root@27fa4fcc627a:/app/watcher$ python3 watcher.py --db /config/db/database.sqlite --conf /config/config.cfg --log /config/logs/ --plugins /config/plugins/ & [1] 421 root@27fa4fcc627a:/app/watcher$ Config file found, merging any new options. Database found. Traceback (most recent call last): File "/app/watcher/lib/sqlalchemy/engine/base.py", line 1182, in _execute_context context) File "/app/watcher/lib/sqlalchemy/engine/default.py", line 462, in do_execute cursor.execute(statement, parameters) sqlite3.OperationalError: no such table: MOVIES

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "watcher.py", line 113, in core.sql.update_tables() File "/app/watcher/core/sqldb.py", line 530, in update_tables for i in self.get_user_movies(): File "/app/watcher/core/sqldb.py", line 245, in get_user_movies result = self.execute(command) File "/app/watcher/core/sqldb.py", line 118, in execute result = self.engine.execute(command) File "/app/watcher/lib/sqlalchemy/engine/base.py", line 2058, in execute return connection.execute(statement, *multiparams, *params) File "/app/watcher/lib/sqlalchemy/engine/base.py", line 939, in execute return self._execute_text(object, multiparams, params) File "/app/watcher/lib/sqlalchemy/engine/base.py", line 1097, in _execute_text statement, parameters File "/app/watcher/lib/sqlalchemy/engine/base.py", line 1189, in _execute_context context) File "/app/watcher/lib/sqlalchemy/engine/base.py", line 1393, in _handle_dbapi_exception exc_info File "/app/watcher/lib/sqlalchemy/util/compat.py", line 202, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/app/watcher/lib/sqlalchemy/util/compat.py", line 185, in reraise raise value.with_traceback(tb) File "/app/watcher/lib/sqlalchemy/engine/base.py", line 1182, in _execute_context context) File "/app/watcher/lib/sqlalchemy/engine/default.py", line 462, in do_execute cursor.execute(statement, parameters) sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: MOVIES [SQL: 'SELECT FROM MOVIES ORDER BY title ASC']

[jaketame]

Ah ok,

moved database.sqlite to database.sqlite.test

python3 watcher.py --db /config/db/database.sqlite --conf /config/config.cfg --log /config/logs/ --plugins /config/plugins/ &

Ran the above command again

Watcher starts up successful but creates new database under /app/watcher/watcher.sqlite

*appears to be ignoring --db flag.

ln -s /config/db/watcher.sqlite /app/watcher/watcher.sqlite

Creating symlink to data mount works.

[jaketame]

DEBUG 2017-06-09 10:34:49,277 core.sqldb.get_user_movies: Retrieving list of user's movies. ERROR 2017-06-09 10:34:49,333 core.library.read_csv: Error parsing Plex CSV. Traceback (most recent call last): File "/app/watcher/core/library.py", line 301, in read_csv width = int(movie['Width']) ValueError: invalid literal for int() with base 10: '1920 - 1920'

[jaketame]

Ok all imported, around 1400 movies. A success!

[nosmokingbandit]

Thanks for reporting back, and sorry about the mess. I was cleaning up some old ugly code I wrote a long time ago and naturally that breaks more things than I expect.

I'm a little concerned about ValueError: invalid literal for int() with base 10: '1920 - 1920'. Did you figure out what caused that or did it just start working all on its own?