search

Flexget / Flexget

The official FlexGet repository
http://www.flexget.com
MIT License
1.73k stars 471 forks source link

sqlite3.IntegrityError. Flexget 3.1.1 #2565

Closed FeromonDE closed 4 years ago

FeromonDE commented 4 years ago

Hallo, after Update to 3.1.1 i have this in my log if i try to open movie tmdb site from webui entry list:


2020-01-10 14:49:07 ERROR    flask.app                     Exception on /tmdb/movies [GET]
Traceback (most recent call last):

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1246, in _execute_context
    cursor, statement, parameters, context
    │       │          │           └ <sqlalchemy.dialects.sqlite.base.SQLiteExecutionContext object at 0x6c7eb930>
    │       │          └ ('ночь в осаде', 641790)
    │       └ 'INSERT INTO tmdb_search_results (search, movie_id) VALUES (?, ?)'
    └ <sqlite3.Cursor object at 0x671415a0>

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute
    cursor.execute(statement, parameters)
    │      │       │          └ ('ночь в осаде', 641790)
    │      │       └ 'INSERT INTO tmdb_search_results (search, movie_id) VALUES (?, ?)'
    │      └ <method 'execute' of 'sqlite3.Cursor' objects>
    └ <sqlite3.Cursor object at 0x671415a0>

sqlite3.IntegrityError: UNIQUE constraint failed: tmdb_search_results.search

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap
    self._bootstrap_inner()
    │    └ <function Thread._bootstrap_inner at 0x758dbcd8>
    └ <WorkerThread(CP Server Thread-14, started 1818227808)>
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
    │    └ <function WorkerThread.run at 0x740fcd20>
    └ <WorkerThread(CP Server Thread-14, started 1818227808)>

  File "/usr/local/lib/python3.6/dist-packages/cheroot/workers/threadpool.py", line 125, in run
    keep_conn_open = conn.communicate()
                     │    └ <function HTTPConnection.communicate at 0x74087bb8>
                     └ <cheroot.server.HTTPConnection object at 0x6bd81850>

  File "/usr/local/lib/python3.6/dist-packages/cheroot/server.py", line 1280, in communicate
    req.respond()
    │   └ <function HTTPRequest.respond at 0x74087a08>
    └ <cherrypy._cpwsgi_server.CPWSGIHTTPRequest object at 0x6e5fc2d0>

  File "/usr/local/lib/python3.6/dist-packages/cheroot/server.py", line 1083, in respond
    self.server.gateway(self).respond()
    │    │      │       └ <cherrypy._cpwsgi_server.CPWSGIHTTPRequest object at 0x6e5fc2d0>
    │    │      └ <class 'cheroot.wsgi.Gateway_10'>
    │    └ <cherrypy._cpwsgi_server.CPWSGIServer object at 0x702529f0>
    └ <cherrypy._cpwsgi_server.CPWSGIHTTPRequest object at 0x6e5fc2d0>

  File "/usr/local/lib/python3.6/dist-packages/cheroot/wsgi.py", line 143, in respond
    response = self.req.server.wsgi_app(self.env, self.start_response)
               │    │   │      │        │    │    │    └ <function Gateway.start_response at 0x7015cc00>
               │    │   │      │        │    │    └ <cheroot.wsgi.Gateway_10 object at 0x6d1a7210>
               │    │   │      │        │    └ {'ACTUAL_SERVER_PROTOCOL': 'HTTP/1.1', 'PATH_INFO': '/flexget/api/tmdb/movies', 'QUERY_STRING': 'title=%D0%9D%D0%BE%D1%87%D1%...
               │    │   │      │        └ <cheroot.wsgi.Gateway_10 object at 0x6d1a7210>
               │    │   │      └ <cherrypy._cptree.Tree object at 0x740b1610>
               │    │   └ <cherrypy._cpwsgi_server.CPWSGIServer object at 0x702529f0>
               │    └ <cherrypy._cpwsgi_server.CPWSGIHTTPRequest object at 0x6e5fc2d0>
               └ <cheroot.wsgi.Gateway_10 object at 0x6d1a7210>

  File "/usr/local/lib/python3.6/dist-packages/cherrypy/_cptree.py", line 302, in __call__
    return app(environ, start_response)
           │   │        └ <bound method Gateway.start_response of <cheroot.wsgi.Gateway_10 object at 0x6d1a7210>>
           │   └ {'ACTUAL_SERVER_PROTOCOL': 'HTTP/1.1', 'PATH_INFO': '/tmdb/movies', 'QUERY_STRING': 'title=%D0%9D%D0%BE%D1%87%D1%8C%20%D0%92%...
           └ <Flask 'flexget.api.app'>

  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 2309, in __call__
    return self.wsgi_app(environ, start_response)
           │    │        │        └ <bound method Gateway.start_response of <cheroot.wsgi.Gateway_10 object at 0x6d1a7210>>
           │    │        └ {'ACTUAL_SERVER_PROTOCOL': 'HTTP/1.1', 'PATH_INFO': '/tmdb/movies', 'QUERY_STRING': 'title=%D0%9D%D0%BE%D1%87%D1%8C%20%D0%92%...
           │    └ <function Flask.wsgi_app at 0x73868198>
           └ <Flask 'flexget.api.app'>

  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
               │    └ <function Flask.full_dispatch_request at 0x73865cd8>
               └ <Flask 'flexget.api.app'>

> File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
         │    └ <function Flask.dispatch_request at 0x73865c90>
         └ <Flask 'flexget.api.app'>

  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           │    │              │    │           │   └ {}
           │    │              │    │           └ <Request 'https://xxxxxxx:3539/flexget/api/tmdb/movies?title=Ночь%20В%20Осаде' [GET]>
           │    │              │    └ 'tmdb_tmdb_movies_api'
           │    │              └ <Rule '/tmdb/movies/' (OPTIONS, HEAD, GET) -> tmdb_tmdb_movies_api>
           │    └ {'static': <bound method _PackageBoundObject.send_static_file of <Flask 'flexget.api.app'>>, 'specs': <function View.as_view....
           └ <Flask 'flexget.api.app'>

  File "/usr/local/lib/python3.6/dist-packages/flask_restplus/api.py", line 313, in wrapper
    resp = resource(*args, **kwargs)
           │         │       └ {}
           │         └ ()
           └ <function View.as_view.<locals>.view at 0x706d0300>

  File "/usr/local/lib/python3.6/dist-packages/flask/views.py", line 88, in view
    return self.dispatch_request(*args, **kwargs)
           │    │                 │       └ {}
           │    │                 └ ()
           │    └ <function Resource.dispatch_request at 0x7375d6f0>
           └ <flexget.components.tmdb.api.TMDBMoviesAPI object at 0x6bdb6770>

  File "/usr/local/lib/python3.6/dist-packages/flask_restplus/resource.py", line 44, in dispatch_request
    resp = meth(*args, **kwargs)
           │     │       └ {}
           │     └ ()
           └ <function with_session.<locals>.decorator.<locals>.wrapper at 0x6bdb0c00>

  File "/usr/local/lib/python3.6/dist-packages/flexget/api/app.py", line 74, in wrapped
    rv = f(*args, **kwargs)
         │  │       └ {}
         │  └ ()
         └ <function with_session.<locals>.decorator.<locals>.wrapper at 0x6bdb0468>

  File "/usr/local/lib/python3.6/dist-packages/flexget/utils/database.py", line 30, in wrapper
    return func(*args, **kwargs)
           │     │       └ {'session': <sqlalchemy.orm.session.ContextSession object at 0x6bdb65d0>}
           │     └ ()
           └ <bound method TMDBMoviesAPI.get of <flexget.components.tmdb.api.TMDBMoviesAPI object at 0x6bdb6770>>

  File "/usr/local/lib/python3.6/dist-packages/flexget/api/app.py", line 367, in wrapped
    rv = method(*args, **kwargs)
         │       │       └ {'session': <sqlalchemy.orm.session.ContextSession object at 0x6bdb65d0>}
         │       └ (<flexget.components.tmdb.api.TMDBMoviesAPI object at 0x6bdb6770>,)
         └ <function TMDBMoviesAPI.get at 0x706d0468>

  File "/usr/local/lib/python3.6/dist-packages/flexget/components/tmdb/api.py", line 136, in get
    movie = lookup(session=session, **args)
            │              │          └ {'title': 'Ночь В Осаде', 'tmdb_id': None, 'language': None, 'year': None, 'only_cached': None}
            │              └ <sqlalchemy.orm.session.ContextSession object at 0x6bdb65d0>
            └ <function with_session.<locals>.decorator.<locals>.wrapper at 0x7067ee40>

  File "/usr/local/lib/python3.6/dist-packages/flexget/utils/database.py", line 27, in wrapper
    return func(*args, **kwargs)
           │     │       └ {'session': <sqlalchemy.orm.session.ContextSession object at 0x6bdb65d0>, 'title': 'Ночь В Осаде', 'tmdb_id': None, 'language...
           │     └ ()
           └ <function ApiTmdb.lookup at 0x7066edf8>

  File "/usr/local/lib/python3.6/dist-packages/flexget/components/tmdb/api_tmdb.py", line 405, in lookup
    movie = session.merge(movie)
            │       │     └ <flexget.components.tmdb.api_tmdb.TMDBMovie object at 0x66f31d70>
            │       └ <function Session.merge at 0x74702d20>
            └ <sqlalchemy.orm.session.ContextSession object at 0x6bdb65d0>

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/session.py", line 2093, in merge
    self._autoflush()
    │    └ <function Session._autoflush at 0x74702780>
    └ <sqlalchemy.orm.session.ContextSession object at 0x6bdb65d0>

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/session.py", line 1588, in _autoflush
    util.raise_from_cause(e)
    │    └ <function raise_from_cause at 0x74c6d0c0>
    └ <module 'sqlalchemy.util' from '/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/__init__.py'>

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
    │            │           │             │             └ None
    │            │           │             └ <traceback object at 0x6bdbf4b8>
    │            │           └ IntegrityError('(sqlite3.IntegrityError) UNIQUE constraint failed: tmdb_search_results.search',)
    │            └ IntegrityError('(sqlite3.IntegrityError) UNIQUE constraint failed: tmdb_search_results.search',)
    └ <function reraise at 0x74c4ee88>

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/compat.py", line 153, in reraise
    raise value
          └ IntegrityError('(sqlite3.IntegrityError) UNIQUE constraint failed: tmdb_search_results.search',)

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/session.py", line 1577, in _autoflush
    self.flush()
    │    └ <function Session.flush at 0x747000c0>
    └ <sqlalchemy.orm.session.ContextSession object at 0x6bdb65d0>

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/session.py", line 2470, in flush
    self._flush(objects)
    │    │      └ None
    │    └ <function Session._flush at 0x74700198>
    └ <sqlalchemy.orm.session.ContextSession object at 0x6bdb65d0>

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/session.py", line 2608, in _flush
    transaction.rollback(_capture_exception=True)
    │           └ <function SessionTransaction.rollback at 0x746fef60>
    └ <sqlalchemy.orm.session.SessionTransaction object at 0x6bd93750>

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
    │      │       │         │          └ <traceback object at 0x701eea80>
    │      │       │         └ IntegrityError('(sqlite3.IntegrityError) UNIQUE constraint failed: tmdb_search_results.search',)
    │      │       └ <class 'sqlalchemy.exc.IntegrityError'>
    │      └ <function reraise at 0x74c4ee88>
    └ <module 'sqlalchemy.util.compat' from '/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/compat.py'>

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/compat.py", line 153, in reraise
    raise value
          └ IntegrityError('(sqlite3.IntegrityError) UNIQUE constraint failed: tmdb_search_results.search',)

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/session.py", line 2568, in _flush
    flush_context.execute()
    │             └ <function UOWTransaction.execute at 0x746dd108>
    └ <sqlalchemy.orm.unitofwork.UOWTransaction object at 0x6bd935f0>

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
    rec.execute(self)
    │   │       └ <sqlalchemy.orm.unitofwork.UOWTransaction object at 0x6bd935f0>
    │   └ <function SaveUpdateAll.execute at 0x746dd540>
    └ SaveUpdateAll(mapped class TMDBSearchResult->tmdb_search_results)

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/unitofwork.py", line 589, in execute
    uow,
    └ <sqlalchemy.orm.unitofwork.UOWTransaction object at 0x6bd935f0>

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj
    insert,
    └ <generator object _collect_insert_commands at 0x67105e10>

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/persistence.py", line 1084, in _emit_insert_statements
    c = cached_connections[connection].execute(statement, multiparams)
        │                  │                   │          └ [{'search': 'ночь в осаде', 'movie_id': 641790}]
        │                  │                   └ <sqlalchemy.sql.dml.Insert object at 0x7019d4b0>
        │                  └ <sqlalchemy.engine.base.Connection object at 0x701f7c10>
        └ {<sqlalchemy.engine.base.Connection object at 0x701f7c10>: <sqlalchemy.engine.base.Connection object at 0x6c7eb550>}

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 982, in execute
    return meth(self, multiparams, params)
           │    │     │            └ {}
           │    │     └ ([{'search': 'ночь в осаде', 'movie_id': 641790}],)
           │    └ <sqlalchemy.engine.base.Connection object at 0x6c7eb550>
           └ <bound method ClauseElement._execute_on_connection of <sqlalchemy.sql.dml.Insert object at 0x7019d4b0>>

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
           │          │                      │     │            └ {}
           │          │                      │     └ ([{'search': 'ночь в осаде', 'movie_id': 641790}],)
           │          │                      └ <sqlalchemy.sql.dml.Insert object at 0x7019d4b0>
           │          └ <function Connection._execute_clauseelement at 0x7495b300>
           └ <sqlalchemy.engine.base.Connection object at 0x6c7eb550>

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1101, in _execute_clauseelement
    distilled_params,
    └ [{'search': 'ночь в осаде', 'movie_id': 641790}]

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1250, in _execute_context
    e, statement, parameters, cursor, context
       │          │           │       └ <sqlalchemy.dialects.sqlite.base.SQLiteExecutionContext object at 0x6c7eb930>
       │          │           └ <sqlite3.Cursor object at 0x671415a0>
       │          └ ('ночь в осаде', 641790)
       └ 'INSERT INTO tmdb_search_results (search, movie_id) VALUES (?, ?)'

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1476, in _handle_dbapi_exception
    util.raise_from_cause(sqlalchemy_exception, exc_info)
    │    │                │                     └ (<class 'sqlite3.IntegrityError'>, IntegrityError('UNIQUE constraint failed: tmdb_search_results.search',), <traceback object...
    │    │                └ IntegrityError('(sqlite3.IntegrityError) UNIQUE constraint failed: tmdb_search_results.search',)
    │    └ <function raise_from_cause at 0x74c6d0c0>
    └ <module 'sqlalchemy.util' from '/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/__init__.py'>

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
    │            │           │             │             └ IntegrityError('UNIQUE constraint failed: tmdb_search_results.search',)
    │            │           │             └ <traceback object at 0x701a4c38>
    │            │           └ IntegrityError('(sqlite3.IntegrityError) UNIQUE constraint failed: tmdb_search_results.search',)
    │            └ IntegrityError('(sqlite3.IntegrityError) UNIQUE constraint failed: tmdb_search_results.search',)
    └ <function reraise at 0x74c4ee88>

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/compat.py", line 152, in reraise
    raise value.with_traceback(tb)
          │     │              └ <traceback object at 0x701a4c38>
          │     └ <method 'with_traceback' of 'BaseException' objects>
          └ IntegrityError('(sqlite3.IntegrityError) UNIQUE constraint failed: tmdb_search_results.search',)

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1246, in _execute_context
    cursor, statement, parameters, context
    │       │          │           └ <sqlalchemy.dialects.sqlite.base.SQLiteExecutionContext object at 0x6c7eb930>
    │       │          └ ('ночь в осаде', 641790)
    │       └ 'INSERT INTO tmdb_search_results (search, movie_id) VALUES (?, ?)'
    └ <sqlite3.Cursor object at 0x671415a0>

  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute
    cursor.execute(statement, parameters)
    │      │       │          └ ('ночь в осаде', 641790)
    │      │       └ 'INSERT INTO tmdb_search_results (search, movie_id) VALUES (?, ?)'
    │      └ <method 'execute' of 'sqlite3.Cursor' objects>
    └ <sqlite3.Cursor object at 0x671415a0>

sqlalchemy.exc.IntegrityError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely)
(sqlite3.IntegrityError) UNIQUE constraint failed: tmdb_search_results.search
[SQL: INSERT INTO tmdb_search_results (search, movie_id) VALUES (?, ?)]
[parameters: ('ночь в осаде', 641790)]
(Background on this error at: http://sqlalche.me/e/gkpj)
``
liiight commented 4 years ago

is this consistent to the same movie every time? ночь в осаде in this case

FeromonDE commented 4 years ago

no, it was only one time, when i tried to open tmdb site

liiight commented 4 years ago

Probably a race condition. Our API is not thread safe unfortunately. This is bound to happen from time to time.

Reopen if you feel its relevant.