internal 500 error when trying to use /author/

[aroundmyroom]

get this error in the log

Probably an issue in the db, but no idea how to solve this. A number of features (/hot/ /author (when pressing the author name) gives this error [2018-07-06 19:36:51,020] {/home/dennis/calibre-web/cps.py:25} INFO - Attempting to start gevent [2018-07-06 19:37:00,863] {/usr/local/lib/python2.7/dist-packages/flask/app.py:1761} ERROR - Exception on /author/23 [GET] Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 2292, in wsgi_app response = self.full_dispatch_request() File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1815, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1718, in handle_user_exception reraise(exc_type, exc_value, tb) File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1813, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1799, in dispatch_request return self.view_functionsrule.endpoint File "/home/dennis/calibre-web/cps/web.py", line 393, in decorated_view return login_required(func)(*args, *kwargs) File "/usr/local/lib/python2.7/dist-packages/flask_login/utils.py", line 261, in decorated_view return func(args, **kwargs) File "/home/dennis/calibre-web/cps/web.py", line 1218, in author db.Books.timestamp.desc()) File "/home/dennis/calibre-web/cps/web.py", line 543, in fill_indexpage .filter(db_filter).filter(common_filters()).all())) File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2836, in all return list(self) File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2987, in iter self.session._autoflush() File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1444, in _autoflush util.raise_from_cause(e) File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 265, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1434, in _autoflush self.flush() File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2254, in flush self._flush(objects) File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2380, in _flush transaction.rollback(_capture_exception=True) File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 66, in exit compat.reraise(exc_type, exc_value, exc_tb) File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2344, in _flush flush_context.execute() File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 391, in execute rec.execute(self) File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 556, in execute uow File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 177, in save_obj mapper, table, update) File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 768, in _emit_update_statements execute(statement, multiparams) File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 948, in execute return meth(self, multiparams, params) File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection return connection._execute_clauseelement(self, multiparams, params) File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1060, in _execute_clauseelement compiled_sql, distilled_params File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context context) File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception exc_info File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 265, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1170, in _execute_context context) File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 506, in do_executemany cursor.executemany(statement, parameters) IntegrityError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely) (sqlite3.Integrit yError) UNIQUE constraint failed: authors.name [SQL: u'UPDATE authors SET name=? WHERE authors.id = ?'] [parameters: ((u'Jones, Diana Wynne', 5485), (u'Tamas, Geller t', 5486), (u'Broscie, Jack', 5487), (u'Raay, Carla van', 5488), (u'Smeesters, Hilde', 5489), (u'Zimler, Richard', 5490), (u'Gogh, Vincent van', 5491), (u'Hulsker, J an', 5492) ... displaying 10 of 11 total bound parameter sets ... (u'Michel van Egmond,Jan Hillenius', 5499), (u'Michael Robotham,Joost Mulder', 5508))] (Backgroun d on this error at: http://sqlalche.me/e/gkpj)

[aroundmyroom]

just fixed the issue. I had 10 author names where there was a pipe in the name. I was trigged by an issue reported long time ago where it was stated it was 'fixed') .. but I rather have a good db. so I did some magic with access (kuch) and a sqllite odbc connector and was able to see the issue quickly. It seems to work fine now after replacing the db.

[OzzieIsaacs]

The error itself points towards double ids in the author field? The output also shows no pipes in the name? Could you please tell me which field in which table you changed, so I could try to figure out a solution for the future?

[aroundmyroom]

I'll check this tonight. I had messed up my db again trying to remove duplicates within calibre itself (windows and SMB shares and calibre are not working well together (calibre tells you that the db is not ok anymore and you can nothing do other than to rebuild it again... ) .. Even after the fix of the db calibre tells me that there are still some issues with unique constraints with +13000 books in the db I am afraid I still have some maintenance to do. Come back in this thread this evening. (CET)

[OzzieIsaacs]

I had also problems with windos. If you set your SMB server to SMB version 1.0 you can access it via calibre without any problem (I usually face file locked problems). For this usecase calibre-web is perfect (if there are no errors of course).

[aroundmyroom]

Yeah I tried that.. 200 books ok than boem .. calibre, version 3.26.1 CantOpenError: CantOpenError: unable to open database file pfff ..

update forced server in NT1 mode (Samba) enabled SMB1 in Windows 10, disabled oplocks (messed up the db directly) now it is ok again. but will see what will happen if Calibre reads and writes the db and than look what calibre-web will do. This morning with oplocks on it could only read the db from memory (it seems) cause new added books where not seen and it could not write to the db anymore (I believe it was privately in use) OperationalError: (sqlite3.OperationalError) disk I/O error (Background on this error at: http://sqlalche.me/e/e3q

400 books now in the DB .. manageable when there is an issue.

[aroundmyroom]

for historical reasons I configured my SMB vm with calibre-web to use SMB1 only smb.conf [global] min protocol = NT1 max protocol = NT1 client min protocol = NT1 client max protocol = NT1

[books] force user = useryoudesire path = /var/www/html/books writeable = yes browseable = yes guest ok = yes guest account = root create mask = 0777 directory mask = 0777 oplocks = false level2 oplocks = false

Yeah 0777 possibly not needed but its in my private network and I want to be sure that this kind of stuff will not bother anything. Finetuning is possible afterwards

Than under windows 10, add SMB 1.0 support through features. make sure the server is started and mounted. Now it seems Calibre and Calibre-web can share the db without issue (Just added 56 books in series and single books) .. it seems to work .. (fingers crossed)

[aroundmyroom]

issue closed as a lot of errors is caused by a broken DB of calibre

[aroundmyroom]

update, using SMB1 with the oplocks on false solved the issue. Added 526 books today and no db error. not 1 #yeah. Thanks for the tip !