janeczku / calibre-web

:books: Web app for browsing, reading and downloading eBooks stored in a Calibre database
GNU General Public License v3.0
12.72k stars 1.33k forks source link

calibre-web can not update calibre-library-data after changing meta data in calibre-web #1133

Closed abubaman closed 4 years ago

abubaman commented 4 years ago

It is same issue I spend #522. But it works as same as what I explained to you. I closed Calibre-Progrm when I modify metadata on calibre-web, but it doesn't apply to the real metadata on metadata.opf file in Calibre-library.

I am using Synology nas with docker(technosoft2000). and I added PUID, PGID following technosoft's instruction(because I think the problem is related to permission problem.). But it doesn't work as same as before.

Below is my calibre-web.log file. It was an info level so it doesn't contain much information which we want but please look into them. and I changed it to debug level now.

[2020-01-05 09:26:01,129]  INFO {cps:89} Starting Calibre Web...
[2020-01-05 09:26:01,370]  INFO {cps.server:140} Starting Gevent server on [::]:8083
[2020-01-05 09:26:11,746]  INFO {cps.web:1164} Login failed for user "admin123" IP-adress: 220.93.115.218
[2020-01-05 09:28:31,735]  INFO {cps.admin:691} Update status requested
[2020-01-05 09:28:45,662]  INFO {cps.server:194} webserver stop (restart=True)
[2020-01-05 09:29:45,371]  INFO {cps.server:182} Performing restart of Calibre-Web
[2020-01-05 09:29:48,226] {/calibre-web/app/cps/web.py:188} INFO - Starting Calibre Web...
[2020-01-05 09:29:48,239] {/calibre-web/app/cps/comic.py:32} WARNING - cannot import comicapi, extracting comic metadata will not work: No module named comicapi.comicarchive
[2020-01-05 09:29:48,694] {/calibre-web/app/cps/server.py:84} INFO - Starting Gevent server
[2020-01-05 12:53:27,994] {/calibre-web/app/cps/web.py:2061} INFO - 051 Luxury 2019 01 - al su eobseum
[2020-01-05 12:53:28,227] {/calibre-web/app/cps/web.py:2061} INFO - 051 Luxury 2019 01 - al su eobseum
[2020-01-05 12:53:28,304] {/calibre-web/app/cps/web.py:2061} INFO - 051 Luxury 2019 01 - al su eobseum
[2020-01-05 12:53:28,363] {/calibre-web/app/cps/web.py:2061} INFO - 051 Luxury 2019 01 - al su eobseum
[2020-01-05 12:53:28,461] {/calibre-web/app/cps/web.py:2061} INFO - 051 Luxury 2019 01 - al su eobseum
[2020-01-05 12:53:28,611] {/calibre-web/app/cps/web.py:2061} INFO - 051 Luxury 2019 01 - al su eobseum
[2020-01-05 12:53:28,729] {/calibre-web/app/cps/web.py:2061} INFO - 051 Luxury 2019 01 - al su eobseum
[2020-01-05 12:53:28,820] {/calibre-web/app/cps/web.py:2061} INFO - 051 Luxury 2019 01 - al su eobseum
[2020-01-05 12:53:29,057] {/calibre-web/app/cps/web.py:2061} INFO - 051 Luxury 2019 01 - al su eobseum
[2020-01-05 12:53:29,176] {/calibre-web/app/cps/web.py:2061} INFO - 051 Luxury 2019 01 - al su eobseum
[2020-01-06 07:47:58,708] {/calibre-web/app/cps/web.py:188} INFO - Starting Calibre Web...
[2020-01-06 07:47:58,746] {/calibre-web/app/cps/comic.py:32} WARNING - cannot import comicapi, extracting comic metadata will not work: No module named comicapi.comicarchive
[2020-01-06 07:48:00,085] {/calibre-web/app/cps/server.py:84} INFO - Starting Gevent server
[2020-01-06 07:48:30,243] {/usr/lib/python2.7/site-packages/flask/app.py:1560} ERROR - Exception on / [GET]
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/calibre-web/app/cps/web.py", line 349, in decorated_view
    return login_required(func)(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/flask_login/utils.py", line 261, in decorated_view
    return func(*args, **kwargs)
  File "/calibre-web/app/cps/web.py", line 1235, in index
    title=_(u"Recently Added Books"), page="root", config_authors_max=config.config_authors_max)
  File "/calibre-web/app/cps/web.py", line 715, in render_title_template
    return render_template(instance=config.config_calibre_web_title, accept=EXTENSIONS_UPLOAD, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/flask/templating.py", line 134, in render_template
    context, ctx.app)
  File "/usr/lib/python2.7/site-packages/flask/templating.py", line 116, in _render
    rv = template.render(context)
  File "/usr/lib/python2.7/site-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/lib/python2.7/site-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/calibre-web/app/cps/templates/index.html", line 1, in top-level template code
    {% extends "layout.html" %}
  File "/calibre-web/app/cps/templates/layout.html", line 39, in top-level template code
    <a class="navbar-brand" href="{{url_for('index')}}">aaa<img src="{{ url_for('static', finename='favicon.ico') }}"></a>
  File "/usr/lib/python2.7/site-packages/flask/helpers.py", line 308, in url_for
    appctx.app.inject_url_defaults(endpoint, values)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1786, in inject_url_defaults
    func(endpoint, values)
  File "/calibre-web/app/cps/cache_buster.py", line 64, in reverse_to_cache_busted_url
    file_hash = bust_filename(values["filename"])
KeyError: 'filename'
[2020-01-06 07:48:46,279] {/usr/lib/python2.7/site-packages/flask/app.py:1560} ERROR - Exception on / [GET]
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/calibre-web/app/cps/web.py", line 349, in decorated_view
    return login_required(func)(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/flask_login/utils.py", line 261, in decorated_view
    return func(*args, **kwargs)
  File "/calibre-web/app/cps/web.py", line 1235, in index
    title=_(u"Recently Added Books"), page="root", config_authors_max=config.config_authors_max)
  File "/calibre-web/app/cps/web.py", line 715, in render_title_template
    return render_template(instance=config.config_calibre_web_title, accept=EXTENSIONS_UPLOAD, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/flask/templating.py", line 134, in render_template
    context, ctx.app)
  File "/usr/lib/python2.7/site-packages/flask/templating.py", line 116, in _render
    rv = template.render(context)
  File "/usr/lib/python2.7/site-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/lib/python2.7/site-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/calibre-web/app/cps/templates/index.html", line 1, in top-level template code
    {% extends "layout.html" %}
  File "/calibre-web/app/cps/templates/layout.html", line 39, in top-level template code
    <a class="navbar-brand" href="{{url_for('index')}}">aaa<img src="{{ url_for('static', finename='favicon.ico') }}"></a>
  File "/usr/lib/python2.7/site-packages/flask/helpers.py", line 308, in url_for
    appctx.app.inject_url_defaults(endpoint, values)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1786, in inject_url_defaults
    func(endpoint, values)
  File "/calibre-web/app/cps/cache_buster.py", line 64, in reverse_to_cache_busted_url
    file_hash = bust_filename(values["filename"])
KeyError: 'filename'
[2020-01-06 07:52:36,616] {/calibre-web/app/cps/web.py:188} INFO - Starting Calibre Web...
[2020-01-06 07:52:36,621] {/calibre-web/app/cps/comic.py:32} WARNING - cannot import comicapi, extracting comic metadata will not work: No module named comicapi.comicarchive
[2020-01-06 07:52:37,024] {/calibre-web/app/cps/server.py:84} INFO - Starting Gevent server
[2020-01-06 10:44:55,508] {/calibre-web/app/cps/web.py:188} INFO - Starting Calibre Web...
[2020-01-06 10:44:55,554] {/calibre-web/app/cps/comic.py:32} WARNING - cannot import comicapi, extracting comic metadata will not work: No module named comicapi.comicarchive
[2020-01-06 10:44:56,843] {/calibre-web/app/cps/server.py:84} INFO - Starting Gevent server
[2020-01-06 10:54:08,942] {/calibre-web/app/cps/web.py:188} INFO - Starting Calibre Web...
[2020-01-06 10:54:08,946] {/calibre-web/app/cps/comic.py:32} WARNING - cannot import comicapi, extracting comic metadata will not work: No module named comicapi.comicarchive
[2020-01-06 10:54:09,339] {/calibre-web/app/cps/server.py:84} INFO - Starting Gevent server
[2020-01-06 10:59:32,844] {/calibre-web/app/cps/web.py:188} INFO - Starting Calibre Web...
[2020-01-06 10:59:32,848] {/calibre-web/app/cps/comic.py:32} WARNING - cannot import comicapi, extracting comic metadata will not work: No module named comicapi.comicarchive
[2020-01-06 10:59:33,230] {/calibre-web/app/cps/server.py:84} INFO - Starting Gevent server
[2020-01-06 11:07:24,140] {/calibre-web/app/cps/web.py:188} INFO - Starting Calibre Web...
[2020-01-06 11:07:24,144] {/calibre-web/app/cps/comic.py:32} WARNING - cannot import comicapi, extracting comic metadata will not work: No module named comicapi.comicarchive
[2020-01-06 11:07:24,495] {/calibre-web/app/cps/server.py:84} INFO - Starting Gevent server
[2020-01-06 11:09:48,743] {/calibre-web/app/cps/web.py:188} INFO - Starting Calibre Web...
[2020-01-06 11:09:48,750] {/calibre-web/app/cps/comic.py:32} WARNING - cannot import comicapi, extracting comic metadata will not work: No module named comicapi.comicarchive
[2020-01-06 11:09:49,364] {/calibre-web/app/cps/server.py:84} INFO - Starting Gevent server
[2020-01-06 11:23:40,927] {/calibre-web/app/cps/web.py:188} INFO - Starting Calibre Web...
[2020-01-06 11:23:40,931] {/calibre-web/app/cps/comic.py:32} WARNING - cannot import comicapi, extracting comic metadata will not work: No module named comicapi.comicarchive
[2020-01-06 11:23:41,328] {/calibre-web/app/cps/server.py:84} INFO - Starting Gevent server
[2020-01-06 11:25:35,613] {/calibre-web/app/cps/web.py:188} INFO - Starting Calibre Web...
[2020-01-06 11:25:35,619] {/calibre-web/app/cps/comic.py:32} WARNING - cannot import comicapi, extracting comic metadata will not work: No module named comicapi.comicarchive
[2020-01-06 11:25:36,055] {/calibre-web/app/cps/server.py:84} INFO - Starting Gevent server
[2020-01-06 11:26:45,030] {/calibre-web/app/cps/web.py:188} INFO - Starting Calibre Web...
[2020-01-06 11:26:45,034] {/calibre-web/app/cps/comic.py:32} WARNING - cannot import comicapi, extracting comic metadata will not work: No module named comicapi.comicarchive
[2020-01-06 11:26:45,419] {/calibre-web/app/cps/server.py:84} INFO - Starting Gevent server
[2020-01-06 11:29:18,617] {/calibre-web/app/cps/web.py:188} INFO - Starting Calibre Web...
[2020-01-06 11:29:18,621] {/calibre-web/app/cps/comic.py:32} WARNING - cannot import comicapi, extracting comic metadata will not work: No module named comicapi.comicarchive
[2020-01-06 11:29:19,010] {/calibre-web/app/cps/server.py:84} INFO - Starting Gevent server
[2020-01-06 11:34:25,784] {/calibre-web/app/cps/web.py:188} INFO - Starting Calibre Web...
[2020-01-06 11:34:25,788] {/calibre-web/app/cps/comic.py:32} WARNING - cannot import comicapi, extracting comic metadata will not work: No module named comicapi.comicarchive
[2020-01-06 11:34:26,179] {/calibre-web/app/cps/server.py:84} INFO - Starting Gevent server
[2020-01-06 11:50:12,817] {/calibre-web/app/cps/web.py:188} INFO - Starting Calibre Web...
[2020-01-06 11:50:12,832] {/calibre-web/app/cps/comic.py:32} WARNING - cannot import comicapi, extracting comic metadata will not work: No module named comicapi.comicarchive
[2020-01-06 11:50:13,663] {/calibre-web/app/cps/server.py:84} INFO - Starting Gevent server
[2020-01-06 12:35:31,139] {/calibre-web/app/cps/web.py:188} INFO - Starting Calibre Web...
[2020-01-06 12:35:31,241] {/calibre-web/app/cps/comic.py:32} WARNING - cannot import comicapi, extracting comic metadata will not work: No module named comicapi.comicarchive
[2020-01-06 12:35:33,133] {/calibre-web/app/cps/server.py:84} INFO - Starting Gevent server
[2020-01-06 23:33:02,754] {/calibre-web/app/cps/web.py:188} INFO - Starting Calibre Web...
[2020-01-06 23:33:02,791] {/calibre-web/app/cps/comic.py:32} WARNING - cannot import comicapi, extracting comic metadata will not work: No module named comicapi.comicarchive
[2020-01-06 23:33:04,032] {/calibre-web/app/cps/server.py:84} INFO - Starting Gevent server
[2020-01-06 23:35:19,969] {/calibre-web/app/cps/web.py:188} INFO - Starting Calibre Web...
[2020-01-06 23:35:19,973] {/calibre-web/app/cps/comic.py:32} WARNING - cannot import comicapi, extracting comic metadata will not work: No module named comicapi.comicarchive
[2020-01-06 23:35:20,349] {/calibre-web/app/cps/server.py:84} INFO - Starting Gevent server
[2020-01-08 22:50:06,637] {/calibre-web/app/cps/server.py:120} INFO - Performing restart of Calibre-Web
[2020-01-08 22:50:20,439] {/calibre-web/app/cps/web.py:188} INFO - Starting Calibre Web...
[2020-01-08 22:50:20,491] {/calibre-web/app/cps/comic.py:32} WARNING - cannot import comicapi, extracting comic metadata will not work: No module named comicapi.comicarchive
[2020-01-08 22:50:21,716] {/calibre-web/app/cps/server.py:84} INFO - Starting Gevent server

522

I have updated all book's meta data(tags, author, series, rating) and it works fine.

But after I tried to update metadata(description) for one book, I found that I can see any changes in calibre-web but it doesn't apply to Calibre-Library.

So after that, I tried to add new book to Calibre-Library. but it didn't work. Calibre-web didn't change anything even though I clicked 'Re-Connect to Calibre DB' in admin page.

So do you have any idea to solve this problem? If you need further information please let me know.

abubaman commented 4 years ago

I think It can be a timezone problem because I found that log's date and time are different to my system's date and time.

OzzieIsaacs commented 4 years ago

doesn't apply to the real metadata on metadata.opf file in Calibre-library

This is correct, as calibre-web doesn't generate, modify and use the opf files generated by calibre as backup.

abubaman commented 4 years ago

And how calibre-web can modify metadata of books in the library without modift metadata.opf file?

OzzieIsaacs commented 4 years ago

My knowledge how calibre handles books: Upon adding books to the calibre program metadata is extracted from the ebooks and stored in a database (metadata.db) in the root folder of the database. The books are identified by an id. Furthermore the books are stored in a files structure with [authorname]/[bookname]([book_id]). In parallel a file metadata.opf is generated with the current metadata stored in the library (I think this is done as backup if the metadata.db is crashed somehow). Calibre-web handles books as follow: Upon uploading books to the calibre-web program metadata is extracted from certain file types (epub, pdf, cbz files) the ebooks and stored in a database (metadata.db) in the root folder of the database. The books are identified by an id. Furthermore the books are stored in a files structure with [authorname]/[bookname]([book_id]). No metadata.opf file is generated and used. If you edit metadata of files in calibre, this is stored in the metadata.db and in parallel in the metadata.opf file, if author/booktitle is edited, the files are moved arond on hdd. If you edit metadata of files in calibre-web, this is stored in the metadata.db, if author/booktitle is edited, the files are moved arond on hdd.

OzzieIsaacs commented 4 years ago

Regarding the errors in the stacktrace:

    file_hash = bust_filename(values["filename"])
KeyError: 'filename'

I think you changed some filecontents, and you reverted your change and now is it working again and has nothing to do with the problem. So I don't do anythink about this part.

abubaman commented 4 years ago

Now, I solved this problem. But I don't know what was the exact problem becuase i tried lots of things. So, now i am sharing my setting of docker(technosoft2000) in synology nas. And i will also share suspious thing which it can be a cause of this problem.

My Docker ENV & Owner Setting

So, It is working well now. thank you for your kind support.

OzzieIsaacs commented 4 years ago

@technosoft2000: Maybe this is helpful for you, as several users have reported similar problems in the past to me.