Fail to start...

[mycanaletto]

Hello,

I'm desperately stuck on this page, although in docker I can see the /books directory, he doesn't want to access it.

` ,

[INFO] Docker image version: 1.4.0

[INFO] Alpine Linux version: 3.11.5

[WARNING] A group with id 65539 exists already [in use by calibre] and will be modified.

[WARNING] The group calibre will be renamed to calibre

[WARNING] A user with id 1029 exists already [in use by calibre].

[WARNING] Create user calibre with temporary user id 999.

adduser: user 'calibre' in use

[WARNING] Assign non-unique user id 1029 to created user calibre

usermod: no changes

[INFO] Current active timezone is CEST

Sat Oct  3 23:47:29 CEST 2020

[INFO] Container timezone is changed to: Europe/Paris

[INFO] Change the ownership of /calibre-web (including subfolders) to calibre:calibre

[INFO] Current git version is:

git version 2.24.1

[INFO] Checkout the latest Calibre-Web version ...

[INFO] Autoupdate is active, try to pull the latest sources for Calibre-Web ...

[INFO] ... current git status is

On branch master

Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

1cb640e51e52bb6a02e2cecaf6cb3e9bd2b1349e

FO] ... pulling sources

Already up to date.

[INFO] ... git status after update is

On branch master

Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

1cb640e51e52bb6a02e2cecaf6cb3e9bd2b1349e

[INFO] Config directory option is ACTIVATED

> due this the directory /calibre-web/config will be used to store the configuration

[INFO] Change the ownership of /calibre-web/config (including subfolders) to calibre:calibre

[INFO] Checking permissions of the config directory: /calibre-web/config

> Output is: 755 calibre 65539 calibre 1029

> Permissions: 755

> Assigned group: calibre

Assigned group ID: 65539

> Assigned owner: calibre

> Assigned owner ID: 1029

> Using permissions for checks: 0755

> The user calibre:1029 is the owner and has write access at /calibre-web/config

[INFO] 'app.db' and 'gdrive.db' will be linked into /calibre-web/config

> 'app.db' link /calibre-web/app/app.db exists already and won't be recreated

> 'gdrive.db' link /calibre-web/app/gdrive.db exists already and won't be recreated

[INFO] Checking permissions of the books directory: /books

> Output is: 777 users 100 UNKNOWN 1024

> Permissions: 777

> Assigned group: users

> Assigned group ID: 100

> Assigned owner: UNKNOWN

> Assigned owner ID: 1024

> Using permissions for checks: 0777

> Everyone has write access at /books

[INFO] The mapped volume for /books contains a Calibre database file 'metadata.db' which will be used

[INFO] The kindlegen directory exist already and will be used: /calibre-web/kindlegen

[INFO] Downloading kindlegen from http://kindlegen.s3.amazonaws.com/kindlegen_linux_2.6_i386_v2_9.tar.gz into directory: /calibre-web/kindlegen/kindlegen_linux_2.6_i386_v2_9.tar.gz

--2020-10-03 23:47:33--  http://kindlegen.s3.amazonaws.com/kindlegen_linux_2.6_i386_v2_9.tar.gz

Resolving kindlegen.s3.amazonaws.com... 52.216.112.251

Connecting to kindlegen.s3.amazonaws.com|52.216.112.251|:80... connected.

HTTP request sent, awaiting response... 403 Forbidden

2020-10-03 23:47:33 ERROR 403: Forbidden.

[INFO] Extracting kindlegen_linux_2.6_i386_v2_9.tar.gz into directory: /calibre-web/kindlegen

tar (child): /calibre-web/kindlegen/kindlegen_linux_2.6_i386_v2_9.tar.gz: Cannot open: No such file or directory

tar (child): Error is not recoverable: exiting now

tar: Child returned status 2

tar: Error is not recoverable: exiting now

[INFO] Change the ownership of /calibre-web/kindlegen (including subfolders) to calibre:calibre

[INFO] kindlegen (Amazon Kindle Generator) will be linked into /calibre-web/app/vendor

> create kindlegen link /calibre-web/app/vendor/kindlegen assigned to source /calibre-web/kindlegen/kindlegen

ln: failed to create symbolic link '/calibre-web/app/vendor/kindlegen': File exists

> change the ownership of /calibre-web/app/vendor/kindlegen to calibre:calibre

[INFO] Launching Calibre-Web ...

`

[varequan]

Me too

[Technosoft2000]

So it seems that kindlegen is not available anymore from the default place at Amazon ... I've to check if there is another place or if I've to remove the support.

[mycanaletto]

I'm not sure that's what's stopping us from going further?

[Technosoft2000]

Okay it isn't an issue of kindlegen now, but I've to remove it because it isn't supported anymore by Amazon and most of the users only use calibre ebook-convert tool inside this image.

I've tested the last version 1.4.0 and I'm able to configure all parts as described in my README.md

Configuration at first launch

1. Point your browser to http://hostname: e.g. http://hostname:8083

2. Set Location of your Calibre books folder to the path of the folder where you mounted your Calibre folder in the container, which is by default \books. So enter at the field Location of Calibre database the mapped volume \books. Hit Submit then Login.

3. Default admin login: Username: admin Password: admin123

Important Hint: in your picture I can't see that you've entered the directory \books

[mycanaletto]

I am using /books...

[Technosoft2000]

Could you please check your calibre-web.log in your app directory , maybe there is an error visible. Because the docker logs doesn't show any issue why Calibre-Web is not able to use your Calibre DB file metadata.db, could be an issue with the Calibre-Web application.

[mycanaletto]

`[2020-10-03 04:27:51,241]  WARN {cps.config_sql:293} Log path None not valid, falling back to default
[2020-10-03 04:27:55,927]  INFO {cps:98} Starting Calibre Web...
[2020-10-03 04:27:55,951]  WARN {cps.config_sql:316} invalidating configuration
[2020-10-03 04:27:56,224]  INFO {cps.server:146} Starting Gevent server on [::]:8083
[2020-10-03 04:35:11,851] ERROR {cps.config_sql:315} (sqlite3.OperationalError) database is locked
[SQL: attach database '/books/metadata.db' as calibre;]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2020-10-03 04:35:11,852]  WARN {cps.config_sql:316} invalidating configuration
[2020-10-03 04:41:33,035] ERROR {cps.config_sql:315} (sqlite3.OperationalError) database is locked
[SQL: attach database '/books/metadata.db' as calibre;]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2020-10-03 04:41:33,036]  WARN {cps.config_sql:316} invalidating configuration
[2020-10-03 04:59:40,523] ERROR {cps.config_sql:315} (sqlite3.OperationalError) database is locked
[SQL: attach database '/books/metadata.db' as calibre;]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2020-10-03 04:59:40,523]  WARN {cps.config_sql:316} invalidating configuration
[2020-10-03 05:09:36,980]  INFO {cps:98} Starting Calibre Web...
[2020-10-03 05:09:36,984]  WARN {cps.config_sql:316} invalidating configuration
[2020-10-03 05:09:37,240]  INFO {cps.server:146} Starting Gevent server on [::]:8083
[2020-10-03 05:18:16,330]  WARN {cps.config_sql:316} invalidating configuration
[2020-10-03 05:22:51,329]  INFO {cps:98} Starting Calibre Web...
[2020-10-03 05:22:51,332]  WARN {cps.config_sql:316} invalidating configuration
[2020-10-03 05:22:51,519]  INFO {cps.server:146} Starting Gevent server on [::]:8083
[2020-10-03 05:26:44,516]  INFO {cps:98} Starting Calibre Web...
[2020-10-03 05:26:44,518]  WARN {cps.config_sql:316} invalidating configuration
[2020-10-03 05:26:44,702]  INFO {cps.server:146} Starting Gevent server on [::]:8083
[2020-10-03 05:33:15,755] ERROR {cps.config_sql:315} (sqlite3.OperationalError) database is locked
[SQL: attach database '/books/metadata.db' as calibre;]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2020-10-03 05:33:15,756]  WARN {cps.config_sql:316} invalidating configuration
[2020-10-03 05:47:15,691] ERROR {cps.config_sql:315} (sqlite3.OperationalError) database is locked
[SQL: attach database '/books/metadata.db' as calibre;]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2020-10-03 05:47:15,692]  WARN {cps.config_sql:316} invalidating configuration
[2020-10-03 23:23:05,963] ERROR {cps.config_sql:315} (sqlite3.OperationalError) database is locked
[SQL: attach database '/books/metadata.db' as calibre;]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2020-10-03 23:23:05,964]  WARN {cps.config_sql:316} invalidating configuration
[2020-10-03 23:47:45,098]  INFO {cps:98} Starting Calibre Web...
[2020-10-03 23:47:45,106]  WARN {cps.config_sql:316} invalidating configuration
[2020-10-03 23:47:45,362]  INFO {cps.server:146} Starting Gevent server on [::]:8083
[2020-10-03 23:56:56,555] ERROR {cps.config_sql:315} (sqlite3.OperationalError) database is locked
[SQL: attach database '/books/metadata.db' as calibre;]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2020-10-03 23:56:56,556]  WARN {cps.config_sql:316} invalidating configuration
[2020-10-04 00:03:17,740] ERROR {cps.config_sql:315} (sqlite3.OperationalError) database is locked
[SQL: attach database '/books/books/metadata.db' as calibre;]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2020-10-04 00:03:17,741]  WARN {cps.config_sql:316} invalidating configuration
[2020-10-05 17:56:39,128]  WARN {cps.config_sql:316} invalidating configuration
[2020-10-05 18:03:11,851] ERROR {cps.config_sql:315} (sqlite3.OperationalError) database is locked
[SQL: attach database '/books/metadata.db' as calibre;]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2020-10-05 18:03:11,852]  WARN {cps.config_sql:316} invalidating configuration
[2020-10-05 18:05:31,548]  WARN {cps.config_sql:316} invalidating configuration
[2020-10-05 18:08:02,132]  INFO {cps:98} Starting Calibre Web...
[2020-10-05 18:08:02,135]  WARN {cps.config_sql:316} invalidating configuration
[2020-10-05 18:08:02,317]  INFO {cps.server:146} Starting Gevent server on [::]:8083
[2020-10-05 18:08:02,468]  WARN {cps.config_sql:316} invalidating configuration
[2020-10-05 18:08:55,758]  WARN {cps.config_sql:316} invalidating configuration
[2020-10-05 18:13:07,134]  INFO {cps:98} Starting Calibre Web...
[2020-10-05 18:13:07,137]  WARN {cps.config_sql:316} invalidating configuration
[2020-10-05 18:13:07,323]  INFO {cps.server:146} Starting Gevent server on [::]:8083
[2020-10-05 18:17:43,463]  INFO {cps:98} Starting Calibre Web...
[2020-10-05 18:17:43,467]  WARN {cps.config_sql:316} invalidating configuration
[2020-10-05 18:17:43,761]  INFO {cps.server:146} Starting Gevent server on [::]:8083`

[Technosoft2000]

[2020-10-05 18:03:11,851] ERROR {cps.config_sql:315} (sqlite3.OperationalError) database is locked [SQL: attach database '/books/metadata.db' as calibre;]

SQLite is meant to be a lightweight database, and thus can't support a high level of concurrency. OperationalError: database is locked errors indicate that your application is experiencing more concurrency than sqlite can handle in default configuration. This error means that one thread or process has an exclusive lock on the database connection and another thread timed out waiting for the lock the be released.

Therefore for me it looks like that you have an process / application running which already exclusively uses /books/metadata.db and Calibre-Web can't open it.

[mycanaletto]

This is curious because I made a copy of my Calibre database, which is not the one used by Calibre, and therefore is not used by any other application....

[mycanaletto]

If I rename my db, caliber web creates a blank one... but the result is the same, it doesn't go further...

[Technosoft2000]

Okay that is interesting - maybe the database file has now an issue, maybe it includes some kind of lock internally, I'm not sure.

I've found the following article to repair an SQLite database How To Repair An SQLite Database

So you can try to fix the file via - here an example: # echo ".dump" | sqlite metadata.db | sqlite metadata-fixed.db

then rename metadata-fixed.db back to metadata.db.

If I rename my db, caliber web creates a blank one... but the result is the same, it doesn't go further...

Yes if no metadata.db is found then a default one will be created - I've tested this approach and also my existing metadata.db now, and both are working here.

You can try to get into the docker container and to check if you're able to change into your mounted /books directory which should of course work.

[mycanaletto]

yes !

[mycanaletto]

[Technosoft2000]

Inside the docker image is no sqlite tool installed, you've to download / install it on your host.

Btw. at the moment I've no idea & clue why it is not working in your case, sorry :-|

As alternative you can try the docker container from Linuxserver https://github.com/linuxserver/docker-calibre-web maybe you've more success with their version.

[Technosoft2000]

If I get a new idea what to check or what the issue could be I let you know it.

[mycanaletto]

I tried and I have the same problem...

I wonder if the problem isn't due to the fact that /books is an NFS setup? Yet I can write from the container...

[Technosoft2000]

Ohhh okay I've never used a mounted network share as directory for the books - both the docker container and the books are here on the same machine (Synology NAS)

Yet I can write from the container...

The container was also able to create the metadata.db inside your share.

Nevertheless please try-out how it behaves when you create all on the same host, start with an empty /books directory, only to check if you are able to come through this config page.

[mycanaletto]

Well, I got around the problem by mounting the NFS volume locally on Ubuntu... And now it works...

Now I have to figure out how to get my configuration (users, etc.) from what was running on Synology and doesn't want to boot anymore... It's possible that the version that is on Synology (1.3.1) doesn't support the Caliber 5.x database?

[Technosoft2000]

Well, I got around the problem by mounting the NFS volume locally on Ubuntu... And now it works...

great to hear that it works now 👍

It's possible that the version that is on Synology (1.3.1) doesn't support the Caliber 5.x database?

Yes that could be of course - the last time I've tested v1.4.0 with Calibre 4.x which worked well. So it could be the case that Calibre 5.x introduces changes which aren't backward compatible.

[mycanaletto]

Is there a way to get the configuration out of my old web-gauge that doesn't fail?

[Technosoft2000]

The configuration of Calibre-Web is stored at app.db and is independent from the metadata.db of Calibre - I think you can re-use the old app.db to don't loose the users and rights, but I think you have to adapt then some rights which were added in later versions of Calibre-Web.

[mycanaletto]

Yes, I had thought of that, except that on the old Docker Synology installation there was only /books outside the container, everything else is in the container... Who doesn't want to be wasted anymore...

[mycanaletto]

When I am connected with my domain name via a reverse proxy (HAProxy) it is impossible to create users.... It always falls on the admin account... On the other hand, live on the IP it works....

Everything else works now (uploads and convert).

Thanks for your help and patience !

[Technosoft2000]

When I am connected with my domain name via a reverse proxy (HAProxy)

I've tested the user creation here with a NGINX reverse proxy and it works for me; so please check your configuration and logs, hopefully you'll find some hints to fix the issue.

Everything else works now (uploads and convert).

Great 👍

Thanks for your help and patience !

You're welcome and thanks for your cooperation.

[mycanaletto]

Impossible to create users, if I use the ssl domain name + reverse proxy. I always fall on the Admin account...

On the other hand if I connect directly with the IP address it works well...

#PROXY-START/
location  ~* \.(php|jsp|cgi|asp|aspx)$
{
    proxy_bind $server_addr;
    proxy_pass http://172.17.0.3:8083;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header REMOTE-HOST $remote_addr;
}
location /
{
    proxy_bind $server_addr;
    proxy_pass http://172.17.0.3:8083;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header REMOTE-HOST $remote_addr;

    #Persistent connection related configuration

    add_header X-Cache $upstream_cache_status;
    #Set Nginx Cache

    proxy_ignore_headers Set-Cookie Cache-Control expires;
    proxy_cache cache_one;
    proxy_cache_key $host$uri$is_args$args;
    proxy_cache_valid 200 304 301 302 360m;
    expires 12h;
}

#PROXY-END/

[NullReferenceError]

I also was having this issue in Calibre-web, using the linuxserver image:

apt update && apt install sqlite
echo ".dump" | sqlite3 metadata.db | sqlite3 metadata-fixed.db
mv metadata.db metadata-old-backup.db
mv metadata-fixed.db metadata.db
chown abc:abc metadata.db

Resolved my issue after reading through this issue (Able to select the metadata.db directory from the web page and login) Not sure when the issue originally started, haven't logged in to the calibre web in a few months, but I also have watchtower running and auto updating the image. The first occurrence in the logs of the error happening was back in 9/20 (Watchtower pulled the updated image around that time):

[2020-09-29 23:57:56,458]  INFO {cps:98} Starting Calibre Web...
[2020-09-29 23:57:56,460]  WARN {cps.config_sql:316} invalidating configuration
[2020-09-29 23:57:56,561]  INFO {cps.server:144} Starting Gevent server on [::]:8083
[2020-09-30 00:07:51,991]  INFO {cps.server:207} webserver stop (restart=False)
[2020-09-30 00:07:51,992]  INFO {cps.server:184} Performing shutdown of Calibre-Web