search

seblucas / cops

Calibre OPDS (and HTML) PHP Server : web-based light alternative to Calibre content server / Calibre2OPDS to serve ebooks (epub, mobi, pdf, ...)
http://blog.slucas.fr/en/oss/calibre-opds-php-server
GNU General Public License v2.0
1.43k stars 229 forks source link

metadata.db not found #533

Open pjoc opened 1 month ago

pjoc commented 1 month ago

I know this is a FAQ, but I've checked:

$config['calibre_directory'] = '/storage/Media/Books/';

File /storage/Media/Books/metadata.db not found, Please check

ls -lZ /storage/Media/Books/metadata.db

-rw-r--r--. 1 poc poc unconfined_u:object_r:unlabeled_t:s0 4128768 Aug 6 11:58 /storage/Media/Books/metadata.db

Suggestions welcome.

marioscube commented 1 month ago
  1. What linux distribution are you running?
  2. This seems a SELinux restriction (to test, switch it off. If confirmed it's SELinux, then switch it on again)
  3. No longer use COPS. Go to https://github.com/mikespub-org/seblucas-cops for a maintained distribution.
  4. for more testing: put your Books directory as a subdirectory of ./cops/ (and change config_local.php accordingly). $config['calibre_directory'] = './Books/';
pjoc commented 1 month ago
  1. What linux distribution are you running?

Fedora 40

2. This seems a SELinux restriction (to test, switch it off. If confirmed it's SELinux, then switch it on again)

Tried that. Made no difference.

3. No longer use COPS. Go to https://github.com/mikespub-org/seblucas-cops for a maintained distribution.

That's what I'm using.

4. for more testing: put your Books directory as a subdirectory of ./cops/ (and change config_local.php accordingly). $config['calibre_directory'] = './Books/';

Did that as a symbolic link. No difference.

Tried again but copying the directory (recursively). This time it doesn't give the metadata.db error, but the check still fails with:

You've been redirected because COPS is not configured properly Database error

marioscube commented 1 month ago

Some progress.

To test: chmod 777 -R ./Books

Refresh browser cache and try again.

pjoc commented 1 month ago

Some progress.

To test: chmod 777 -R ./Books

Refresh browser cache and try again.

That did it, thanks. Does this mean that Books can't be outside the COPS tree?

marioscube commented 1 month ago

chmod 755 -R ./Books & refresh browser cache

./Books should be able to be outside ./cops

Just play around with the SELinux settings, permissions and owners of the path /storage/Media/Books

setenforce 0 chown poc:poc -R /storage chmod 755 -R /storage $config['calibre_directory'] = '/storage/Media/Books/';

This should work until the next (re)boot.

If it does, then play around with the SELinux setting to give access to /storage/Media/Books and it's subdirectories. For a personal server I hate SELinux. Therefore I use Debian. But in version 12 I can no longer place ./Books in a /home/user/ diirectory (I always did). I now have to use /storage/ as well.

pjoc commented 1 month ago

OK. I actually did have it working on the old version of COPS, using SElinux, but then did a fresh install and am trying to figure out what bits are missing. Still haven´t got Apache access working from outside my local net (all this has been on localhost:80), but thatś another issue.

Thanks anyway.

marioscube commented 1 month ago

To make apache2 access from the WAN working (for IPv4):

I personally would not open port 80 or 443 to the www. I would use a random port (eg 12380 and 12443) and forward these ports in your router to port 80 and 443.

pjoc commented 1 month ago

To make apache2 access from the WAN working (for IPv4):

* use portforwarding in your router: open port 80 external to 80 internal with the IP from your local PC

* use a free domain like duckdns or freemyip of no-ip and point that to the external WAN IP of your router

* look into let's encrypt for SSL/https on port 443

I personally would not open port 80 or 443 to the www. I would use a random port (eg 12380 and 12443) and forward these ports in your router to port 80 and 443.

Yes, I understand that. including LetsEncrypt, and I have a domain already. I just need to go over all the details and make sure the firewall settings are correct.

marioscube commented 1 month ago

Happy reading!

Don't forget to set this in config_local.php /*

pjoc commented 1 month ago

/*

* Choose preferred epub reader when viewing epub files online:

* 'monocle' (default)

* 'epubjs'
  */
  $config['cops_epub_reader'] = 'epubjs';

Didn't know about that, thanks.

pjoc commented 1 month ago

I personally would not open port 80 or 443 to the www. I would use a random port (eg 12380 and 12443) and forward these ports in your router to port 80 and 443.

Then I would have to announce those ports to my users, so not sure what good it would do. I already have access control and fail2ban.

marioscube commented 1 month ago

If you have users, then port 80 and 443 will have to do.

Just test your security.

pjoc commented 1 month ago

I'm afraid I still don't have this working. I continue to get the 'metadata.db not found' error, despite the file clearly existing, the config_local script pointing to the right place (including the terminating '/') and the permissions aligned (execute and read on every element of the path). FollowSymLinks is on by default. Turning SElinux off makes no difference, so that isn't it. I've tried accessing the file as the Apache user using 'sudo', and it's accessible. It would help if there were a debugging option saying exactly what the access error is, but there doesn't appear to be.

marioscube commented 1 month ago

Back to basics then:

What is the exact error given by cops/checkconfig.php ? Have you installed COPS from the zip file? (not git clone etc.)

pjoc commented 1 month ago

Back to basics then:

* can you open the metadata.db and books in calibre?

Yes. Tested from the command-line on with both the /storage location and with /var/www/html/Books

* to test put all in ./cops/Books/

I assume you mean /var/www/html/Books. If not, please clarify.

* SELinux off

Makes no difference.

* no symlinks

Off.

* any fancy virthosts in apache2? Keep apache2 vanilla for now.

Just the one pointing to my server, which is very simple.

What is the exact error given by cops/checkconfig.php ? Have you installed COPS from the zip file? (not git clone etc.)

When the db is in /storage/... I get "metadata.db not found".

When I put the db in /var/www/html/Books, I no longer get that (this is a change), but the config test still says:

You've been redirected because COPS is not configured properly Database error

I installed COPS from Git. I didn't realise it mattered.

One more thing: if I run "php config_local.php" from the command line, it says:

    _<article class="frontpage">
        <h2>Check if Calibre database file exists and is readable</h2>
        <h4>
         File ./metadata.db not found,

Please check_

This is after restarting Apache. Nevertheless, the web GUI still says it can find metadata.db but there is a database error.

marioscube commented 1 month ago

After some troubles*) installing fedora server 40 (command line only) in virtualbox, installing apache (httpd) and php (and the php packages for COPS) I installed COPS from the zip file in /var/www/html/cops Then I installed a calibre library in /var/www/html/cops/Books Configured cops with config_local.php

And it works.

*) to make apache2 (httpd) work I had to enable and start apache sudo systemctl enable httpd sudo systemctl start httpd I clearly do not know what I'm doing here. I'm a simple debian guy. ;-)

SELINUX set to "disabled" in /etc/sysconfig/selinux

edit: I opened the firewall as well (do not know if it's needed, but it can't hurt).

pjoc commented 1 month ago

I moved the library to /var/www/html/cops/Books but no difference.

(you can combine those two apache commands into one: systemctl -now start httpd)

Did you create an 'apache' user? What user owns the /var/www/html/cops directory?

This is the full output of the config test:

_You've been redirected because COPS is not configured properly Database error Check if PHP version is correct OK (8.3.10) Check if GD is properly installed and loaded OK Check if Sqlite is properly installed and loaded OK Check if libxml is properly installed and loaded OK Check if DOM is properly installed and loaded OK Check if XMLWriter is properly installed and loaded OK Check if Json is properly installed and loaded OK Check if mbstring is properly installed and loaded OK Check if intl is properly installed and loaded OK Check if Normalizer class is properly installed and loaded OK Check if zlib is properly installed and loaded OK Check if the base URL looks OK Base URL detected by the script: / Full URL specified in $config['cops_full_url']: REMOTE_ADDR: 127.0.0.1

SCRIPT_NAME: /checkconfig.php HTTP_HOST: localhost SERVER_NAME: localhost SERVER_ADDR: 127.0.0.1 SERVER_PORT: 80 REQUEST_SCHEME: http REQUEST_URI: /checkconfig.php?err=1 Check if the rendering will be done on client side or server side Client side rendering User agent detected Mozilla/5.0 (X11; Linux x8664; rv:128.0) Gecko/20100101 Firefox/128.0 Check if Calibre database path is not an URL OK Check if Calibre database file exists and is readable OK Check if Calibre database file can be opened with PHP OK Check if Calibre database file contains at least some of the needed tables OK

I'm going to try again but installing from the zip file. Can;t think of anything else to do.

marioscube commented 1 month ago

Hmmmmmmm........ I do not see much wrong here.

I did not create an apache user, the install script should do that (in debian) The owner is either root or fedora (can't check, noisy server is off for the evening and night)

What is your documentroot? Mine is /var/www/html

Cops installed in /var/www/html/cops Books in /var/www/html/cops/Books

Differences I see with my output (in debian)

I installed apache2 like this (but you probably did the same...): sudo dnf install httpd php php-xml php-mbstring php-zip php-gd php-sqlite3 php-intl php-json

pjoc commented 1 month ago

I've wiped everything and am starting again. My DocumentRoot is /var/www/html. To clarify, you unzipped the cops zip file directly into /var/www/html/cops? Note that my .../html directory is currently empty, i.e. I have no other web content, not even an index.html file.

marioscube commented 1 month ago

I made a directory somewhere in /home/user/ cd /home/user/dir downloaded the zip wget https://github.com/mikespub-org/seblucas-cops/releases/download/2.7.2/cops-2.7.1.zip unzip cops-2.7.1.zip copy the extracted files and directories to /var/www/html/cops

made directory Books in /var/www/html/cops ls -l Books/ drwxrwxrwx 4 fedora fedora 4096 Jun 25 2018 'Alexandre Dumas' drwxrwxrwx 10 fedora fedora 4096 Jun 25 2018 'Arthur Conan Doyle' drwxrwxrwx 3 fedora fedora 4096 Jun 25 2018 'H. G. Wells' drwxrwxrwx 3 fedora fedora 4096 Jun 25 2018 'Jack London' drwxrwxrwx 4 fedora fedora 4096 Jun 25 2018 'Lewis Carroll' -rwxrw-rw- 1 fedora fedora 368640 Jun 25 2018 metadata.db -rwxrw-rw- 1 fedora fedora 13569 Jun 25 2018 metadata_db_prefs_backup.json

$config['calibre_directory'] = './Books/'; $config['cops_full_url'] = '/cops/';

Put an index.html file in /var/www/html so you know the apache2 (httpd) server is working.

marioscube commented 1 month ago

Make sure you clean all browser caches before browsing to http://localhost/cops quit browser (firefox) start browser (firefox)

If I do not do that, I sometimes get "strange/unexpected" results, not only with cops.

pjoc commented 1 month ago

OK, I did all that and it works for localhost. I don't know why it didn't work before.

However I still have the original issue: it doesn't work when Books is not under /var/www/html/cops, or when it's a symlink to somewhere outside the tree (even with SElinux turned off). It does work when it's a symlink to a directory within the tree. So it looks like the problem isn't with the symlink per se but with the location of the Books directory.

marioscube commented 1 month ago

At least a partial success!

In debian I can place the calibre library wherever I want, with or without a symlink, except in a /home/user directory. (apparmor prevents that = like SELinux).

Maybe this link provides a solution?

marioscube commented 1 month ago

If I set SELINUX=disabled in /etc/selinux/config (and reboot) I can place a calibre directory in /storage/Books This works.

Placing a directory in /home/fedora/Books does NOT work (for me)

It in my opinion it all comes down to SELinux. It's way to strict and complicated (for me) to set-up and use in a simple LAN environment.

With SELINUX=enforcing in /etc/selinux/config and sudo setenforce 0 COPS works for me.

pjoc commented 1 month ago

As I said, turning off SElinux (setenforcing 0) makes no difference, so it's hard to see how it can be an SElinux issue. I'll look over that link you sent anyway.

pjoc commented 1 month ago

Just a quick update: I still don't know why symlinks won't work, but I've worked around the problem by using a bind mount of the out-of tree directory. I've also made it read-only for safety. This seems to work, touch wood:

mount -o bind,ro /storage/Media/Books Books

PS Possibly my issue is caused by the "Directory" blocks in /etc/httpd/conf/httpd.conf which control access to parts of the filesystem, but this hasn't changed since my previous working system so it seems unlikely.

marioscube commented 1 month ago

Thank you for your update.

A symlink should work, but lately I also use "mount bind" more and more. Usually they just work.

mikespub commented 2 weeks ago

For future reference, when your Calibre library is outside the web directory, it's always worth checking if your PHP installation has "open_basedir" restrictions configured or not: https://www.php.net/manual/en/ini.core.php#ini.open-basedir

pjoc commented 2 weeks ago

Looks like that's it. in /etc/php.ini I find:

open_basedir = /var/www

I assume that must be a recent change in the default setup, since I've never touched any PHP config settings. I'll try turning it off as a test, but may just leave it set since I've worked round the problem with a bind mount.

Good call, thanks. It might be worth adding something about this to the COPS README file.

marioscube commented 2 weeks ago

@pjoc

In my version of fedora 40, in /etc/php.ini

;open_basedir =

That is different from your setup.

Good idea from @mikespub to check this as well. I never do unless COPS is on a synology.

pjoc commented 2 weeks ago

@pjoc

In my version of fedora 40, in /etc/php.ini

;open_basedir =

That is different from your setup.

Good idea from @mikespub to check this as well. I never do unless COPS is on a synology.

Interesting. I've no idea how this could have happened, but 'rpm -V php-common' says the file has been modified, so I guess I must have done it at one point.