seblucas / cops

Calibre OPDS (and HTML) PHP Server : web-based light alternative to Calibre content server / Calibre2OPDS to serve ebooks (epub, mobi, pdf, ...)
http://blog.slucas.fr/en/oss/calibre-opds-php-server
GNU General Public License v2.0
1.42k stars 229 forks source link

Gray page #499

Open errecielle opened 3 years ago

errecielle commented 3 years ago

I have Cops installed on my web server but the only thing I see is a gray page. My server log reports these errors which, you will excuse me, I am absolutely unable to decipher. The metadata.db file is from Calibre 5.9. The PHP version is 7.4

17/01/2021 14:54:35 [error] [client ip_blank] - site_blank - AH01215: PHP Fatal error: Uncaught Error: Call to a member function fetchColumn() on bool in /web/htdocs/site_blank/home/cops/base.php:1295: /web/htdocs/site_blank/home/cops/getJSON.php, referer http://site_blank/cops/
17/01/2021 14:54:35 [error] [client ip_blank] - site_blank - AH01215: Stack trace:: /web/htdocs/site_blank/home/cops/getJSON.php, referer http://site_blank/cops/
17/01/2021 14:54:35 [error] [client ip_blank] - site_blank - AH01215: #0 /web/htdocs/site_blank/home/cops/base.php(1302): Base::executeQuerySingle(): /web/htdocs/site_blank/home/cops/getJSON.php, referer http://site_blank/cops/
17/01/2021 14:54:35 [error] [client ip_blank] - site_blank - AH01215: #1 /web/htdocs/site_blank/home/cops/author.php(43): Base::getCountGeneric(): /web/htdocs/site_blank/home/cops/getJSON.php, referer http://site_blank/cops/
17/01/2021 14:54:35 [error] [client ip_blank] - site_blank - AH01215: #2 /web/htdocs/site_blank/home/cops/base.php(613): Author::getCount(): /web/htdocs/site_blank/home/cops/getJSON.php, referer http://site_blank/cops/
17/01/2021 14:54:35 [error] [client ip_blank] - site_blank - AH01215: #3 /web/htdocs/site_blank/home/cops/JSON_renderer.php(192): Page->InitializeContent(): /web/htdocs/site_blank/home/cops/getJSON.php, referer http://site_blank/cops/
17/01/2021 14:54:35 [error] [client ip_blank] - site_blank - AH01215: #4 /web/htdocs/site_blank/home/cops/getJSON.php(16): JSONRenderer::getJson(): /web/htdocs/site_blank/home/cops/getJSON.php, referer http://site_blank/cops/
17/01/2021 14:54:35 [error] [client ip_blank] - site_blank - AH01215: #5 {main}: /web/htdocs/site_blank/home/cops/getJSON.php, referer http://site_blank/cops/
17/01/2021 14:54:35 [error] [client ip_blank] - site_blank - AH01215: thrown in /web/htdocs/site_blank/home/cops/base.php on line 1295: /web/htdocs/site_blank/home/cops/getJSON.php, referer http://site_blank/cops/
17/01/2021 14:54:30 [error] [client ip_blank] - site_blank - AH01215: PHP Fatal error: Uncaught Error: Call to a member function fetchColumn() on bool in /web/htdocs/site_blank/home/cops/base.php:1295: /web/htdocs/site_blank/home/cops/getJSON.php, referer http://site_blank/cops/index.php
17/01/2021 14:54:30 [error] [client ip_blank] - site_blank - AH01215: Stack trace:: /web/htdocs/site_blank/home/cops/getJSON.php, referer http://site_blank/cops/index.php
17/01/2021 14:54:30 [error] [client ip_blank] - site_blank - AH01215: #0 /web/htdocs/site_blank/home/cops/base.php(1302): Base::executeQuerySingle(): /web/htdocs/site_blank/home/cops/getJSON.php, referer http://site_blank/cops/index.php
17/01/2021 14:54:30 [error] [client ip_blank] - site_blank - AH01215: #1 /web/htdocs/site_blank/home/cops/author.php(43): Base::getCountGeneric(): /web/htdocs/site_blank/home/cops/getJSON.php, referer http://site_blank/cops/index.php
17/01/2021 14:54:30 [error] [client ip_blank] - site_blank - AH01215: #2 /web/htdocs/site_blank/home/cops/base.php(613): Author::getCount(): /web/htdocs/site_blank/home/cops/getJSON.php, referer http://site_blank/cops/index.php
17/01/2021 14:54:30 [error] [client ip_blank] - site_blank - AH01215: #3 /web/htdocs/site_blank/home/cops/JSON_renderer.php(192): Page->InitializeContent(): /web/htdocs/site_blank/home/cops/getJSON.php, referer http://site_blank/cops/index.php
17/01/2021 14:54:30 [error] [client ip_blank] - site_blank - AH01215: #4 /web/htdocs/site_blank/home/cops/getJSON.php(16): JSONRenderer::getJson(): /web/htdocs/site_blank/home/cops/getJSON.php, referer http://site_blank/cops/index.php
17/01/2021 14:54:30 [error] [client ip_blank] - site_blank - AH01215: #5 {main}: /web/htdocs/site_blank/home/cops/getJSON.php, referer http://site_blank/cops/index.php
17/01/2021 14:54:30 [error] [client ip_blank] - site_blank - AH01215: thrown in /web/htdocs/site_blank/home/cops/base.php on line 1295: /web/htdocs/site_blank/home/cops/getJSON.php, referer http://site_blank/cops/index.php

Thank you for your help to solve this issue.

Regards

Roberto

marioscube commented 3 years ago

A grey screen is usually caused by an error in the settings in config_local.php

please post your config_local.php

NB: edit out your domain name and external IP from your post above. You do not want the whole world be able to access you webserver!

errecielle commented 3 years ago

First of all thank you... here the config_local.php

/*
 * The directory containing calibre's metadata.db file, with sub-directories
 * containing all the formats.
 * BEWARE : it has to end with a /
 */
$config['calibre_directory'] = '../files/libri/bibliotecadicalibre/';

/*
 * Catalog's title
 */
$config['cops_title_default'] = "RCL";

/*
 * use URL rewriting for downloading of ebook in HTML catalog
 * See README for more information
 *  1 : enable
 *  0 : disable
 */
$config['cops_use_url_rewriting'] = "1";
marioscube commented 3 years ago

Your config_local.php file should start with:

<?php

if (!isset($config)) {                                                                                                                                                                         
    $config = array();                                                                                                                                                                         
}

maybe yours does or doesn’t.

errecielle commented 3 years ago

config_local.php.txt

marioscube commented 3 years ago

That looks correct.

you could try the absolute path to the calibre library

http://........../cops/checkconfig.php gives no error?

errecielle commented 3 years ago

Immagine 2021-01-17 193525

All seems to be ok...

marioscube commented 3 years ago

Did you install a release zip or git pull?

use the zip.

php 7.4 should work.

Just for testing: Either install the calibre library in ../cops ( and adjust config_local.php ;-) )

or

chmod 777 your existing library to make sure permissions do not mess things up. You can always change it back to what it was.

errecielle commented 3 years ago

I've installed the release zip: cops-1.1.3.zip The library is very large so I've changed the permissions to the library directory and all the sub to 777 Also the metadata.db have 777 as permissions.

Thanx again for your time...

rcl

marioscube commented 3 years ago

10000 books should be no problem, even on a slow (not very old and slow) server. Memory could be a problem. Cops starts with reading the metadata.db in memory (In my experience).

errecielle commented 3 years ago

The books are around 4500 so this should not be a problem

marioscube commented 3 years ago

Sometimes if something does not work a reinstall and new setup of config files fixes the problem.

the error with the base.php file suggest that something with a php file does not work as it should. What? I do not know.

errecielle commented 3 years ago

πŸ˜‚ πŸ˜‚ πŸ˜‚ Ok. Tomortow I'll reinstall and keep you updated.

errecielle commented 3 years ago

Nope :-( I've reinstall from scratch, I've tried to move the calibre directory under /cops but nothing happens and the log errors are the same...

marioscube commented 3 years ago

What OS (Linux debian, ubuntu, centos, etc) do you use? And what webserver (if Apache I can try to help)?

Maybe I can recreate and test your setup in a VM.

Calibre 5.9 works (just tested), so not the problem.

errecielle commented 3 years ago

I'm using a server provided by a server farm here in Italy. Aruba. So I'm not able to answer this type of questions. It's an hosted solution.

I'll check with the provider.

Thank you again.

Roberto

Il giorno lun 18 gen 2021 alle ore 11:37 marioscube < notifications@github.com> ha scritto:

What OS (Linux debian, ubuntu, centos, etc) do you use? And what webserver (if Apache I can try to help)?

Maybe I can recreate and test your setup in a VM.

β€” You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/seblucas/cops/issues/499#issuecomment-762156421, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACD7SDPPROFO44EUSPW4GXDS2QFNDANCNFSM4WGETGXA .

marioscube commented 3 years ago

I did some testing (just for fun), but it did not solve your problem.

My findings are that it is NOT:

What could be the problem:

errecielle commented 3 years ago

First of all thank you very much for your support Secondly I've tried this:

In addition I've posted a question to the tech of the provider but I've little hope they answer this type of questions

PHP 7.4.10 - phpinfo().zip

marioscube commented 3 years ago

Aaahhrrrggg I should have thought of this earlier (I must be getting old and forgetfull). It's this #48 problem all over again with.centOS 7!

The calibre database made with Calibre 5 does not work with the php-sqlite3 version included in centOS 7. I think, based on their website, that Aruba uses centOS 7 as OS.

This thread in #487 gives a nice solution to your problem if you are comfortable with changing some of the COPS code. If not, let me know.

errecielle commented 3 years ago

If it's a simple CTRl-C - Ctrl-V, yes I'm able to do that.

// -------------------------------------------------------------------------------- // -- Fix for incompatibility older sqlite3 & Calibre v5 database format // true will create metadata.dbv4 and use that copy $config['sqlite_fix'] = true; $config['dumpfile'] = '/tmp/metadata.dump'; $config['sqlite3pgm'] = '/opt/bin/sqlite3'; // leave this false unless debugging is required with url /getJSON.php?complete=1 $config['sqlite_fix_debug'] = false;
// --------------------------------------------------------------------------------

But... where?

marioscube commented 3 years ago

See: https://github.com/jvanderzande/cops/commit/9abeffa462b3c72984931ac3e44300931f387767

The above part in config_local.php

The larger part in ../cops/lib/base.php

I have not tried this last iteration of this solution.

errecielle commented 3 years ago

we, you, are close, I feel it :-)

Now the error is in the path for the db: " File ../files/libri/calibre/metadata.dbv4 not found, Please check I've tried to modify the path in $config['dumpfile'] = '/tmp/metadata.dump'; because I don't have a tmp directory so I've put the same of the db but it seems not working

Il giorno gio 21 gen 2021 alle ore 19:32 marioscube < notifications@github.com> ha scritto:

See: https://github.com/jvanderzande/cops/commit/9abeffa462b3c72984931ac3e44300931f387767 http://url

The above part in config_local.php

The larger part in ../cops/lib/base.php

I have not tried this last iteration of this solution.

β€” You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/seblucas/cops/issues/499#issuecomment-764851155, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACD7SDOUSSC4YZVRKSAFJD3S3BXNPANCNFSM4WGETGXA .

marioscube commented 3 years ago

What worx for me:

1 - make e tmp directory in ../cops

2 - copy the config_local.php file in ../cops (or after evaluating my changes copy the changes to your config_local.php config_local.php.txt

3 - replace the existing ../cops/lib/Base.php (or rename first to Base.php.orig)with this Base.php Base.php.txt

4 - check if it works.

There is one problem with this fix. It only works if you use just one calibre library in COPS. I use multiple, so I do not use this fix.

errecielle commented 3 years ago

I checked your files with mine and I don't see any particular differences but some must be there because now I have this error message:

Check if Calibre database file contains at least some of the needed tablesNot all Calibre tables were found. Are you sure you're using the correct database. It actually created the .dbv4 file but created it empty. I also tried to move the calbre folder under /cops but the error remains the same.

I've checked the permissions and all 777 [image: Immagine 2021-01-22 075226.png]

Il giorno ven 22 gen 2021 alle ore 00:13 marioscube < notifications@github.com> ha scritto:

What worx for me:

1 - make e tmp directory in ../cops

2 - copy the config_local.php file in ../cops (or after evaluating my changes copy the changes to your config_local.php config_local.php.txt https://github.com/seblucas/cops/files/5852607/config_local.php.txt

3 - replace the existing ../cops/lib/Base.php (or rename first to Base.php.orig)with this Base.php Base.php.txt https://github.com/seblucas/cops/files/5852610/Base.php.txt

4 - check if it works.

There is one problem with this fix. It only works if you use just one calibre library in COPS. I use multiple, so I do not use this fix.

β€” You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/seblucas/cops/issues/499#issuecomment-765001129, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACD7SDIXFEPOXIDMJC5GP7LS3CYKXANCNFSM4WGETGXA .

marioscube commented 3 years ago

Is there a metadata.dump file in ../cops/tmp/ ?

errecielle commented 3 years ago

Yes. There is

[image: Immagine 2021-01-22 141748.png]

Il giorno ven 22 gen 2021 alle ore 08:24 marioscube < notifications@github.com> ha scritto:

Is there a metadata.dump file in ../cops/tmp/ ?

β€” You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/seblucas/cops/issues/499#issuecomment-765193781, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACD7SDKL4SJ6KZ6T3YH3673S3ER2LANCNFSM4WGETGXA .

marioscube commented 3 years ago

The image does not work, but I believe you. Size of metadata.dump is > 0?

I will test again with my antique setup.

errecielle commented 3 years ago

Yes...

Immagine 2021-01-22 141748

errecielle commented 3 years ago

I don't know if this matters but the sizes of metadata.db and metadata.dump are different The first one is 405.504.

marioscube commented 3 years ago

That is ok. The size of metadata.dump seems a bit small though.....

You have tried to delete metadata.dump and metadata.dbv4 and then tried it again

errecielle commented 3 years ago

Yes, more or less five times switching the calibre folder up and down... I'll retry again now

Il giorno ven 22 gen 2021 alle ore 16:42 marioscube < notifications@github.com> ha scritto:

That is ok. The size of metadata.dump seems a bit small though.....

You have tried to delete metadata.dump and metadata.dbv4 and then tried it again

β€” You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/seblucas/cops/issues/499#issuecomment-765494590, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACD7SDKQTWGRD3FH7AZMFHLS3GMFNANCNFSM4WGETGXA .

errecielle commented 3 years ago

same as before

Immagine 2021-01-22 164538 Immagine 2021-01-22 164624

marioscube commented 3 years ago

What are the "Permessi" of folder../files/libri/bibliotecadicalibre/. ?

errecielle commented 3 years ago

777 and the same for all the sub, the files Only metadata_db_prefs_backup.json have 755

marioscube commented 3 years ago

I'm searching my disks for a batch file I made that you could run, but I can't find it.........

I will try again this evening.

marioscube commented 3 years ago

I have no clue why it does not work for you, but here is plan B!

Plan B 1 - open a terminal on the server 2 - go to ../files/libri/bibliotecadicalibre 3 - type: nano convert.sh (or any other name you like) 4 - copy this into the nano editor

#!/bin/bash rm -f metadata.dbv4 sqlite3 metadata.db .dump > metadata.dump sqlite3 metadata.dbv4 < metadata.dump rm -f metadata.dump echo "done ....."

save with x and yes 5 - make executable: chmod +x convert.sh 6 - run this executable: ./convert.sh 7 - there should be a metadata.dbv4 file in the calibre directory. Is there? 8 - got cops in a browser. 9 - report back

errecielle commented 3 years ago

Unfortunately I can't access neither terminal nor sqlite πŸ˜– . Is it possibile to run this batch as php?

Il giorno ven 22 gen 2021 alle ore 23:21 marioscube < notifications@github.com> ha scritto:

I have no clue why it does not work for you, but here is plan B!

Plan B 1 - open a terminal on the server 2 - go to ../files/libri/bibliotecadicalibre 3 - type: nano convert.sh (or any other name you like) 4 - copy this into the nano editor

#!/bin/bash rm -f metadata.dbv4 sqlite3 metadata.db .dump > metadata.dump sqlite3 metadata.dbv4 < metadata.dump rm -f metadata.dump echo "done ....."

save with x and yes 5 - make executable: chmod +x convert.sh 6 - run this executable: ./convert.sh 7 - there should be a metadata.dbv4 file in the calibre directory. Is there? 8 - got cops in a browser. 9 - report back

β€” You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/seblucas/cops/issues/499#issuecomment-765716848, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACD7SDMPHFIJXNV74UIAWLTS3H25LANCNFSM4WGETGXA .

marioscube commented 3 years ago

You cannot use ssh? You only use ftp?

Maybe tomorrow or Monday I can hack something together that is php.

errecielle commented 3 years ago

Yep... Only ftp

Il giorno sab 23 gen 2021 alle ore 19:25 marioscube < notifications@github.com> ha scritto:

You cannot use ssh? You only use ftp?

Maybe tomorrow or Monday I can hack something together that is php.

β€” You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/seblucas/cops/issues/499#issuecomment-766156615, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACD7SDJPN7J2UXG42RNRV5DS3MIDJANCNFSM4WGETGXA .

marioscube commented 3 years ago

OK, plan C: ;-)

1 - clear of files ../cops/tmp 2 - copy metadata.db from your calibre directory to ../cops/tmp 3 - copy this php file to ../cops/tmp (without the .txt :-) convert-metadata.php.txt 4 - in browser http:///cops/tmp/convert-metadata.php 5 - you should see: Start ..... end! 6 - copy the metadata.dbv4 to your calibre library 7 - then cops in browser. 8 - did this help?

errecielle commented 3 years ago

Plan C probably show us which is the problem... All the steps form 1 to 5 were ok but... the dbv4 is empty :-(

Immagine 2021-01-24 172951

errecielle commented 3 years ago

Could the problem be in sqlite 3 on my server? If you think this should help in understanding the problem I can send you my metadata.db

marioscube commented 3 years ago

The problem is the splite3, or when making the metadata.dbv4 file your metadata.db file.

You could try sending it to me and see if I can convert it.

errecielle commented 3 years ago

Here or where?

Il giorno dom 24 gen 2021 alle ore 18:19 marioscube < notifications@github.com> ha scritto:

The problem is the splite3, or when making the metadata.dbv4 file your metadata.db file.

You could try sending it to me and see if I can convert it.

β€” You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/seblucas/cops/issues/499#issuecomment-766397506, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACD7SDMGTFQOJ5WDSJ6HQX3S3RJCFANCNFSM4WGETGXA .

marioscube commented 3 years ago

Mail to my username at gmail.com

if you want to mail metadata.db to me can you send me:

1 metadata.db from the server

and

2 metadatadb from calibre5

?

errecielle commented 3 years ago

Done.

In parallel I've installed sqlite3 on my pc and, searching the web, I've found some help on how to use it.

sqlite> .open metadata.db sqlite> .output metadata.dbv4 sqlite> .dump sqlite> .output stdout

Probably in the commands I'm missing something because the dbv4 file is not working

Il giorno dom 24 gen 2021 alle ore 18:32 marioscube < notifications@github.com> ha scritto:

Mail to my username at gmail.com

β€” You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/seblucas/cops/issues/499#issuecomment-766399448, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACD7SDIMIBDDYRRQMK5VOJLS3RKUJANCNFSM4WGETGXA .

marioscube commented 3 years ago

I have an empty in-box on xxxxxx at gmail.com?

to run sqlite3 and try for yourself:

1 - make dumpfile: sqlite3 metadata.db .dump > metadata.dump

2 - read txt dump in database: sqlite3 metadata.dbv4 < metadata.dump

DB99 commented 2 years ago

I have a somewhat datted QNAP NAS TS-421 running the latest COPS version installed from source from Jos jvanderzande's fork which incorporate the fixes to lib/Base.php and config_local.php. Ther process works flawlessly and createts a correct metadata.dbv4 file. However, the grey page persists and checkconfig.php cannot verify the database. A check of the v4 database shows the annotations tables still present. Removing the new annotations tables results in proper functioning. Is there any step in the process which is intended to delete these tables or should COPS be able to function with them in the dbv4 database? Thanks in anticipation of assistance.

jvanderzande commented 2 years ago

I am running it on a QNAP TS269L (Also pretty dated) and do not have issues and checkconfig.php works fine. Maybe you can share your config so I can compare it with mine?

DB99 commented 2 years ago

I've attached a zip file with my config_local.php, the debug log and the resulting metadata.dbv4 file. The metadata.dbv4 contains the fts annotations tables and does not work. However, removing these tables does result in it working with COPS. It seems that on my \ QNAP the dump is creating the dbv4 metadata with all of the fts annotations tables without actually fixing them.

I'd like to know why. If not I might try modifying your code to simply copy the metadate.db to metadata.dbv4 rather than creating the latter via the dump command, then dropping the offending tables in the dbv4 file and using that for COPS. As the dbv4 file is used only for COPS I don't see any real downside to this approach. Thanks for your hard work on this. I'd love to know what's causing the problem.

configs.zip

jvanderzande commented 2 years ago

I had to create a symlink to be able to access the library from the webserver. This is what I documented in my config:

//// from command prompt: Create link in website directory for COBS to the Calibre directorie(s) to be able to view them
//~   cd /share/Web/calibre
//~   ln -s /share/Public/ebooks /share/Web/calibre/ebooks
// I have 2 directories hence the array:
$config['calibre_directory'] = array ("Nederlands" => "./ebooks/Calibrebibliotheek-NL-NAS/", "English" => "./ebooks/Calibrebibliotheek-EN-NAS/");