werwolfby / monitorrent

Automatic torrents downloader
Do What The F*ck You Want To Public License
504 stars 69 forks source link

Начала сыпаться база #375

Closed zalessky closed 1 year ago

zalessky commented 1 year ago

в логах:

DatabaseError('(sqlite3.DatabaseError) database disk image is malformed') Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1245, in _execute_context self.dialect.do_execute( File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 581, in do_execute cursor.execute(statement, parameters) sqlite3.DatabaseError: database disk image is malformed The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/cheroot/server.py", line 1252, in communicate req.respond() File "/usr/local/lib/python3.8/site-packages/cheroot/server.py", line 1056, in respond self.server.gateway(self).respond() File "/usr/local/lib/python3.8/site-packages/cheroot/wsgi.py", line 142, in respond response = self.req.server.wsgi_app(self.env, self.start_response) File "/usr/local/lib/python3.8/site-packages/falcon/api.py", line 244, in __call__ responder(req, resp, **params) File "/var/www/monitorrent/monitorrent/rest/execute_logs.py", line 17, in on_get executes, count = self.log_manager.get_log_entries(skip, take) File "/var/www/monitorrent/monitorrent/engine.py", line 454, in get_log_entries for execute, downloads, fails in result_query.all(): File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3211, in all return list(self) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3367, in __iter__ return self._execute_and_instances(context) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3392, in _execute_and_instances result = conn.execute(querycontext.statement, self._params) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 982, in execute return meth(self, multiparams, params) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection return connection._execute_clauseelement(self, multiparams, params) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1095, in _execute_clauseelement ret = self._execute_context( File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context self._handle_dbapi_exception( File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1476, in _handle_dbapi_exception util.raise_from_cause(sqlalchemy_exception, exc_info) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 152, in reraise raise value.with_traceback(tb) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1245, in _execute_context self.dialect.do_execute( File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 581, in do_execute cursor.execute(statement, parameters) sqlalchemy.exc.DatabaseError: (sqlite3.DatabaseError) database disk image is malformed   [SQL: SELECT execute.id AS execute_id, execute.start_time AS execute_start_time, execute.finish_time AS execute_finish_time, execute.status AS execute_status, execute.failed_message AS execute_failed_message, anon_1.count AS anon_1_count, anon_2.count AS anon_2_count FROM execute LEFT OUTER JOIN (SELECT execute_log.execute_id AS execute_id, count(execute_log.id) AS count FROM execute_log GROUP BY execute_log.execute_id, execute_log.level HAVING execute_log.level = ?) AS anon_2 ON execute.id = anon_2.execute_id LEFT OUTER JOIN (SELECT execute_log.execute_id AS execute_id, count(execute_log.id) AS count FROM execute_log GROUP BY execute_log.execute_id, execute_log.level HAVING execute_log.level = ?) AS anon_1 ON execute.id = anon_1.execute_id ORDER BY execute.finish_time DESC LIMIT ? OFFSET ?] [parameters: ('failed', 'downloaded', 10, 80)] (Background on this error at: http://sqlalche.me/e/4xp6) Основное тут вот: sqlalchemy.exc.DatabaseError: (sqlite3.DatabaseError) database disk image is malformed

Логи в вебморде не видны, файл с базой при этом раздут - более 3Гб

пробовал подлечить по рецепту из интернетов: `sqlite3 ./monitorrent.db ".dump" | sqlite3 ./monitorrent.db.fix

Error: near line 23313889: UNIQUE constraint failed: execute_log.id Error: near line 23313890: UNIQUE constraint failed: execute_log.id .... Error: near line 23314016: CHECK constraint failed: execute_log Error: near line 23314017: CHECK constraint failed: execute_log .... Error: near line 23593098: UNIQUE constraint failed: execute_log.id Error: near line 23593099: UNIQUE constraint failed: execute_log.id `

Таких ошибок несколько десятков.

Как почистить базу от логов, но не убить имеющуюся библиотеку?

werwolfby commented 1 year ago

Закрываю за давностью

zalessky commented 11 months ago

Когда база доросла до 5+ Гб, пришлось таки что-то решать Спасла простейшая инструкция отсюда: http://itsphera.ru/1c/vosstanovlenie-fajla-baz-dannyx-sqlite.html

Теперь она весит 99Мб и логи снова работают