search

ajslater / codex

Codex is a web based comic archive browser and reader
GNU General Public License v3.0
202 stars 6 forks source link

Synology Docker: Default comicbox config not read when PUID set after Codex 1.5 #344

Open abillauer opened 6 months ago

abillauer commented 6 months ago

Hi,

My docker upgraded to 1.5 version and after the upgrade no comics showed up in the main screen. I tried re-scanning all the comics in the library from the admin screen. I get the following errors for all comics in the library.

2024-03-18 21:27:33 GMT ERROR   Failed to import: XXXX.cbr
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/codex/librarian/importer/aggregate_metadata.py", line 97, in _get_path_metadata
    with Comicbox(path) as cb:
         ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/comicbox/box/init.py", line 63, in __init__
    self._config: AttrDict = get_config(config)
                             ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/comicbox/config.py", line 212, in get_config
    ad = config.get(_TEMPLATE)
         ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/core.py", line 293, in get
    return templates.as_template(template).value(self, template)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/templates.py", line 146, in value
    out[key] = typ.value(view[key], self)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/templates.py", line 146, in value
    out[key] = typ.value(view[key], self)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/templates.py", line 53, in value
    return self.get_default_value(view.name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/templates.py", line 62, in get_default_value
    raise exceptions.NotFoundError(u"{} not found".format(key_name))
confuse.exceptions.NotFoundError: comicbox.compute_pages not found

Am I doing something wrong? Did I miss a step in the upgrade? Any help would be appreciated. Thanks!

ajslater commented 6 months ago

This looks like a bug. The trace will be very helpful. Thanks. Taking a look.

abillauer commented 6 months ago

I installed a fresh docker config added a new library and got the same behavior.

ajslater commented 6 months ago

This is puzzling me. I can't seem to replicate this myself. At least not yet.

What's happening is the comicbox library is trying to find the default comicbox value of "count_pages" since none was explicitly set by codex. Which is all perfectly normal. It should be getting that value from the config_default.yaml file included in the comicbox package. I'm not sure why it can't find it.

Assuming the container name of your codex docker container is 'codex' could you run this command from your docker host and post the result?

docker exec -ti codex grep compute_pages /usr/local/lib/python3.12/site-packages/comicbox/config_default.yaml

It should return:

  compute_pages: True

The correct config_default.yaml file looks like this

I'm trying to think what else it could be. If there were a rogue config_default.yaml file somewhere, that it could pick up somewhere it might cause this, but codex and comicbox shouldn't really search for that unless explicitly instructed to.

Is there a WARNING just before:

2024-03-18 21:27:33 GMT ERROR   Failed to import: XXXX.cbr

that refers to failing to read the default config or anything?

abillauer commented 6 months ago

I spun up a new clean docker. I ran the following: docker exec -ti codex grep compute_pages /usr/local/lib/python3.12/site-packages/comicbox/config_default.yaml compute_pages: True

I also ran this just to check:

docker exec -ti codex cat /usr/local/lib/python3.12/site-packages/comicbox/config_default.yaml

comicbox:
  # Options
  compute_pages: True
  config: null
  delete: False
  delete_orig: False
  dest_path: .
  dry_run: False
  loglevel: INFO
  metadata: {}
  metadata_cli: null
  read:
    - cli
    - comet
    - metroninfo
    - comicbookinfo
    - comicinfo
    - comictagger
    - filename
    - json
    - pdf
    - yaml

  read_ignore: []
  recurse: False
  stamp_notes: True
  tagger: null
  # API Options
  close_fd: True
  # Actions
  cbz: False
  cover: False
  export: []
  import_paths: []
  index_from: null
  index_to: null
  print: ""
  rename: False
  write: []
  # Targets
  paths: null

I created a library with one cbz file in it and added the library and got the following:


2024-03-19 01:35:46 GMT INFO    Started watching library /comics/test with poll
2024-03-19 01:35:46 GMT INFO    Polling /comics/test again in a second.
2024-03-19 01:35:48 GMT INFO    Polled /comics/test
2024-03-19 01:35:48 GMT INFO    Polling /comics/test again in 5 days.
2024-03-19 01:35:48 GMT INFO    Reading tags from 1 comics in /comics/test...
2024-03-19 01:35:48 GMT ERROR   Failed to import: XXXXX.cbz
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/codex/librarian/importer/aggregate_metadata.py", line 97, in _get_path_metadata
    with Comicbox(path) as cb:
         ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/comicbox/box/init.py", line 63, in __init__
    self._config: AttrDict = get_config(config)
                             ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/comicbox/config.py", line 212, in get_config
    ad = config.get(_TEMPLATE)
         ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/core.py", line 293, in get
    return templates.as_template(template).value(self, template)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/templates.py", line 146, in value
    out[key] = typ.value(view[key], self)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/templates.py", line 146, in value
    out[key] = typ.value(view[key], self)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/templates.py", line 53, in value
    return self.get_default_value(view.name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/templates.py", line 62, in get_default_value
    raise exceptions.NotFoundError(u"{} not found".format(key_name))
confuse.exceptions.NotFoundError: comicbox.compute_pages not found
2024-03-19 01:35:48 GMT INFO    Aggregated tags from 1 comics.
2024-03-19 01:35:48 GMT INFO    Added 1 comics to failed imports.
2024-03-19 01:35:48 GMT INFO    No updates neccissary.
2024-03-19 01:35:49 GMT INFO    Modified 1 folders
2024-03-19 01:35:49 GMT INFO    Updated library /comics/test in a moment. No comics to import.
2024-03-19 01:35:52 GMT INFO    Updating search index with all comics...
abillauer commented 6 months ago

If it helps here is my docker-compose.yml

services:
  codex:
      env_file: .env
      image: docker.io/ajslater/codex
      container_name: codex
      volumes:
        - </PATH/TO/CONFIG>:/config
        - </PATH/TO/COMICS>:/comics:ro
      ports:
        - 9810:9810
      environment:
        - PUID=1024
        - PGID=100
        - TZ=<TIMEZONE>
      restart: on-failure
ajslater commented 6 months ago

Yeah I reckon it likely has something to do with your environment. So the first thing I’d try is changing the locations for volume mounts to eliminate that. Then if that still doesn’t work, try commenting out the PUID & PGID venv vars Other than that I’d try commenting out the env_file:

On Mon, Mar 18, 2024 at 5:03 PM abillauer @.***> wrote:

If it helps here is my docker-compose.yml

services: codex: env_file: .env image: docker.io/ajslater/codex container_name: codex volumes:

  • </PATH/TO/CONFIG>:/config
  • </PATH/TO/COMICS>:/comics:ro ports:
  • 9810:9810 environment:
  • PUID=1024
  • PGID=100
  • TZ= restart: on-failure

— Reply to this email directly, view it on GitHub https://github.com/ajslater/codex/issues/344#issuecomment-2005373672, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACAKRWYJUSSSUEIZS7CJ4TYY56FHAVCNFSM6AAAAABE4OZRK2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBVGM3TGNRXGI . You are receiving this because you commented.Message ID: @.***>

abillauer commented 6 months ago

That could be but the same environment worked on the previous version.

I'll try experimenting on the environment side but it is odd the external docker environment would cause the internal python to be unable to read a docker file inside the image.

Thanks for all the help so far. Any additional suggestions for debugging would be welcome.

ajslater commented 6 months ago

I'll bet there is a comicbox or codex issue to fix here, I'm just not sure how to isolate what that is yet.

abillauer commented 6 months ago

No surprise, but same issue in 1.5.1. I also did a docker-compose down and docker system prune -a to force a clean download of the image. Same error.

abillauer commented 6 months ago

I rolled back to 1.4.3 and it works as expected. 

2024-03-19 06:20:42 UTC INFO    Started watching library /comics/test with poll
2024-03-19 06:20:42 UTC INFO    Polling /comics/test again in a second.
2024-03-19 06:20:43 UTC INFO    Polled /comics/test
2024-03-19 06:20:43 UTC WARNING Library /comics/test update in progress.
2024-03-19 06:20:43 UTC INFO    Polling /comics/test again in 15 minutes.
2024-03-19 06:20:43 UTC INFO    Reading tags from 1 comics in /comics/test...
2024-03-19 06:20:43 UTC INFO    Aggregated tags from 1 comics.
2024-03-19 06:20:43 UTC INFO    Prepared 1 new Publishers.
2024-03-19 06:20:43 UTC INFO    Prepared 1 new Imprints.
2024-03-19 06:20:43 UTC INFO    Prepared 1 new Seriess.
2024-03-19 06:20:43 UTC INFO    Prepared 1 new Volumes.
2024-03-19 06:20:43 UTC INFO    Prepared 1 new Folders.
2024-03-19 06:20:43 UTC INFO    Created 1 Publishers.
2024-03-19 06:20:43 UTC INFO    Created 1 Imprints.
2024-03-19 06:20:43 UTC INFO    Created 1 Seriess.
2024-03-19 06:20:43 UTC INFO    Created 1 Volumes.
2024-03-19 06:20:43 UTC INFO    Created 1 comics.
2024-03-19 06:20:44 UTC INFO    Created 1 new folders relations for altered comics.
2024-03-19 06:20:44 UTC INFO    Linked 1 comics to tags.
2024-03-19 06:20:44 UTC INFO    Updated library /comics/test in a moment. Imported 1 comics at 1.4 comics per second.
2024-03-19 06:20:44 UTC INFO    Modified 1 folders
2024-03-19 06:20:44 UTC INFO    Updated library /comics/test in a moment. No comics to import.
2024-03-19 06:20:47 UTC INFO    Updating search index with all comics...
2024-03-19 06:20:47 UTC INFO    Search engine updated 1 comics in a moment at 1 comics per second.
2024-03-19 06:20:47 UTC INFO    Search index updated in a moment.
2024-03-19 06:20:48 UTC INFO    Updating search index since 2024-03-19 06:20:43.943132+00:00...
2024-03-19 06:20:48 UTC INFO    Search index updated in a moment.

I am trying on a different host machine. I will post an update soon.

abillauer commented 6 months ago

I tried it on a different host machine (a raspberry pi 4) and it works as expected. My initial machine is a Synology NAS with an intel chip. I am not sure what to make of it. I will keep poking around.

abillauer commented 6 months ago

Same issue still in 1.5.2. I know no reason to assume it would be fixed but I figures it was worth a shot.

2024-03-19 20:00:04 UTC INFO    Started watching library /comics/test with poll
2024-03-19 20:00:04 UTC INFO    Library /comics/test waiting for manual poll.
2024-03-19 20:00:14 UTC DEBUG   Poll Library /comics/test: ?/?
2024-03-19 20:00:14 UTC DEBUG   Polling /comics/test...
2024-03-19 20:00:14 UTC DEBUG   Force modify path with missing db stat: /comics/test
2024-03-19 20:00:14 UTC DEBUG   Poller sending unfiltered files: 0 deleted, 0 modified, 1 created, 0 moved.
2024-03-19 20:00:14 UTC DEBUG   Poller sending comic folders: 0 deleted, 1 modified, 0 moved.
2024-03-19 20:00:14 UTC INFO    Polled /comics/test
2024-03-19 20:00:14 UTC DEBUG   Told search indexers to stop for db updates.
2024-03-19 20:00:14 UTC INFO    Library /comics/test waiting for manual poll.
2024-03-19 20:00:14 UTC DEBUG   Updating library /comics/test...
2024-03-19 20:00:14 UTC DEBUG     Comics: 1 created
2024-03-19 20:00:14 UTC DEBUG   Read Book Tags /comics/test: 0/1
2024-03-19 20:00:14 UTC DEBUG   Discover Missing Tags: ?/?
2024-03-19 20:00:14 UTC DEBUG   Create Missing Tags: ?/?
2024-03-19 20:00:14 UTC DEBUG   Create Books: ?/1
2024-03-19 20:00:14 UTC DEBUG   Link Books to Tags: ?/?
2024-03-19 20:00:14 UTC DEBUG   Search Index Update: 0/1
2024-03-19 20:00:14 UTC DEBUG   Search Index Remove Stale Records: ?/?
2024-03-19 20:00:14 UTC DEBUG   Read Book Tags: 0/1
2024-03-19 20:00:14 UTC INFO    Reading tags from 1 comics in /comics/test...
2024-03-19 20:00:14 UTC ERROR   Failed to import: /comics/test/XXXX.cbz
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/codex/librarian/importer/aggregate_metadata.py", line 97, in _get_path_metadata
    with Comicbox(path) as cb:
         ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/comicbox/box/init.py", line 63, in __init__
    self._config: AttrDict = get_config(config)
                             ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/comicbox/config.py", line 212, in get_config
    ad = config.get(_TEMPLATE)
         ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/core.py", line 293, in get
    return templates.as_template(template).value(self, template)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/templates.py", line 146, in value
    out[key] = typ.value(view[key], self)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/templates.py", line 146, in value
    out[key] = typ.value(view[key], self)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/templates.py", line 53, in value
    return self.get_default_value(view.name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/templates.py", line 62, in get_default_value
    raise exceptions.NotFoundError(u"{} not found".format(key_name))
confuse.exceptions.NotFoundError: comicbox.compute_pages not found
2024-03-19 20:00:15 UTC INFO    Aggregated tags from 1 comics.
2024-03-19 20:00:15 UTC DEBUG   Querying existing foreign keys for comics in /comics/test
2024-03-19 20:00:15 UTC DEBUG   Discover Missing Tags: 0/0
2024-03-19 20:00:15 UTC DEBUG   Create Missing Tags: 0/0
2024-03-19 20:00:15 UTC INFO    Added 1 comics to failed imports.
2024-03-19 20:00:15 UTC INFO    No updates neccissary.
2024-03-19 20:00:15 UTC DEBUG   Told search indexers to stop for db updates.
2024-03-19 20:00:15 UTC DEBUG   Updating library /comics/test...
2024-03-19 20:00:15 UTC DEBUG     Folders: 1 modified
2024-03-19 20:00:15 UTC DEBUG   Search Index Update: 0/0
2024-03-19 20:00:15 UTC DEBUG   Search Index Remove Stale Records: ?/?
2024-03-19 20:00:15 UTC DEBUG   Modify Folders: ?/1
2024-03-19 20:00:15 UTC INFO    Modified 1 folders
2024-03-19 20:00:15 UTC INFO    Updated library /comics/test in a moment. No comics to import.
2024-03-19 20:00:18 UTC DEBUG   Search Index Update: ?/?
2024-03-19 20:00:18 UTC DEBUG   Search Index Remove Stale Records: ?/?
2024-03-19 20:00:18 UTC INFO    Updating search index with all comics...
2024-03-19 20:00:18 UTC INFO    Search index updated in a moment.
2024-03-19 20:00:19 UTC DEBUG   No stale records to remove from the search index.
ajslater commented 6 months ago

I tried setting my PUID and GUID in my docker environment to the ones you specified, but it still worked worked for me. Which as far as environment goes leaves whatever's in your .env file or something curious in your /config dir or even more remotely /comics dir.

I'd try loading this docker-compose.yaml on your synology and see what happens:

services:
  codex:
      image: docker.io/ajslater/codex
      container_name: codex
      volumes:
        - </PATH/TO/NEW_EMPTY_DIR>:/config
        - </PATH/TO/NEW_DIR_WITH_ONE_COMIC>:/comics:ro
      ports:
        - 9810:9810
      restart: on-failure
abillauer commented 6 months ago

I tried the docker-compose above and it works! The issue only happens if running the docker as a non-root user.

If I set the PUID/GUID to 1024/100 on the synology (The admin user) the library scan kicks out the errors above. If I run the process as root it works 100% as expected.

The 1.4.3 version works as a non-root user.

I can replicate/resolve the issue consistently now.

tehguitarist commented 6 months ago

Just to stick my nose in, what are the permissions for the comics/folders in question? Does it work non-root if the'yre 755'd or set to the user? I know I had issues with that in the past.

abillauer commented 6 months ago

In the initial setup the permissions were 775 and the admin user was the owner of both the config and comics directory. I did a chmod -R and a chown -R to ensure the permissions and ownership were consistent. This setup worked on the 1.4.x versions.

I even changed the permissions to 777 and it still made no difference.

ajslater commented 6 months ago

The most likely culprit now is my change of the docker image from being based on python3.11.0-alpine to python3.12.2-slim-bookworm I reckon debian is handling something differently than alpine as far as permissions go. I'm still not totally sure what's going on though because the file in question that codex/comicbox is looking for is mode 644, so should be world readable.

Also I can run codex though it's paces on macos and ubuntu linux hosts with PUID=1024,GUID=100 and it works fine. I'm stumped why a synology host would make a difference here.

abillauer commented 6 months ago

Synology is a custom linux distro. It can behave weirdly.

There maybe something in the docker implementation or the image that is trying to make a system call that only root can make.

I rulled out permissions to the docker.sock. The admin user can't call docker commands. It needs to sudo them on synology since the admin user doesn't have permissions to the docker.sock. I updated permissions to the docker.sock so the admin user would have permissions and I still get the same error.

It get's even weirder. I created a container as root and imported one comic and it works fine. I shutdown the instance (docker-compose down) and edited the docker-compose.yml to set the PUID PGUD to the admin user and fired up the container. It starts fine. When I try to import a new comic I get the same error. What was new is that when I tried to read the existing comic (as the admin user) I get the following:

2024-03-20 21:29:41 UTC WARNING comicbox.compute_pages not found
2024-03-20 21:29:42 UTC WARNING comicbox.compute_pages not found
2024-03-20 21:29:42 UTC WARNING comicbox.compute_pages not found
2024-03-20 21:29:44 UTC WARNING comicbox.compute_pages not found

If I switch the user back to root it works as expected.

I am not sure if this helps but it was new for me.

ajslater commented 6 months ago

I've added an experimental feature to Codex v1.5.7 in an attempt to address this issue. Try starting the new Codex with this environment variable:

CHOWN_PYTHON_SITE_PACKAGES=1

When it starts the container it will chown all the codex installed python packages, like comicbox to the PUID user, and maybe that will help.

abillauer commented 6 months ago

No luck. This is the docker-compose.yml

services:
  codex:
      image: docker.io/ajslater/codex
      container_name: codex
      volumes:
        - /volume1/Data/codex:/config
        - /volume1/Comics/server:/comics
      ports:
        - 9810:9810
      environment:
        - PUID=1024
        - PGID=100
        - LOGLEVEL=DEBUG
        - CHOWN_PYTHON_SITE_PACKAGES=1
      restart: on-failure

Here is the log. I don't see any mention of the new "CHOWN_PYTHON_SITE_PACKAGES" switch.

2024-03-26 07:38:51 UTC DEBUG   sqlite journal_mode=wal
2024-03-26 07:38:56 UTC DEBUG   Loaded json choices from /usr/local/lib/python3.12/site-packages/codex/static_root/js choices
2024-03-26 07:38:56 UTC DEBUG   Parsed browser choices
2024-03-26 07:38:56 UTC DEBUG   Parsed reader choices
2024-03-26 07:38:56 UTC DEBUG   Parsed websockets choices
2024-03-26 07:38:56 UTC DEBUG   Loaded json choices from /usr/local/lib/python3.12/site-packages/codex/static_root/js choices-admin
2024-03-26 07:38:58 UTC INFO    Running Codex v1.5.7
2024-03-26 07:38:58 UTC INFO    Ensuring database is correct and up to date...
2024-03-26 07:38:58 UTC DEBUG   Reparing database integrity...
2024-03-26 07:38:59 UTC INFO    Database integrity confirmed.
2024-03-26 07:38:59 UTC INFO    Database up to date.
2024-03-26 07:38:59 UTC INFO    Database ready.
2024-03-26 07:39:00 UTC DEBUG   Reset 0 Library's update_in_progress flag
2024-03-26 07:39:00 UTC INFO    root_path: 
2024-03-26 07:39:00 UTC DEBUG   Started LibrarianDaemon.
2024-03-26 07:39:00 UTC DEBUG   Creating Librarian threads...
2024-03-26 07:39:00 UTC DEBUG   Active threads before thread creation: 2
2024-03-26 07:39:00 UTC DEBUG   Created notifier_thread thread.
2024-03-26 07:39:00 UTC DEBUG   Created delayed_tasks_thread thread.
2024-03-26 07:39:00 UTC DEBUG   Created cover_contributor_thread thread.
2024-03-26 07:39:00 UTC DEBUG   Created search_indexer_thread thread.
2024-03-26 07:39:00 UTC DEBUG   Created comic_importer_thread thread.
2024-03-26 07:39:00 UTC DEBUG   Lifespan application started.
2024-03-26 07:39:00 UTC DEBUG   Lifespan startup started.
2024-03-26 07:39:00 UTC DEBUG   Lifespan startup complete.
2024-03-26 07:39:00 UTC INFO    BroadcastListener started.
2024-03-26 07:39:00 UTC DEBUG   Created watchdog_event_batcher_thread thread.
2024-03-26 07:39:00 UTC DEBUG   Created library_event_observer thread.
2024-03-26 07:39:00 UTC DEBUG   Created library_polling_observer thread.
2024-03-26 07:39:00 UTC DEBUG   Created janitor_thread thread.
2024-03-26 07:39:00 UTC DEBUG   Threads created
2024-03-26 07:39:00 UTC DEBUG   LibrarianDaemon starting all threads.
2024-03-26 07:39:00 UTC DEBUG   Started NotifierThread
2024-03-26 07:39:00 UTC DEBUG   Started DelayedTasksThread
2024-03-26 07:39:00 UTC DEBUG   Started CoverContributorThread
2024-03-26 07:39:00 UTC DEBUG   Started SearchIndexerThread
2024-03-26 07:39:00 UTC DEBUG   Started ComicImporterThread
2024-03-26 07:39:00 UTC DEBUG   Started WatchdogEventBatcherThread
2024-03-26 07:39:00 UTC DEBUG   Started JanitorThread
2024-03-26 07:39:00 UTC INFO    LibrarianDaemon started all threads.
2024-03-26 07:39:00 UTC INFO    LibrarianDaemon ready for tasks.
2024-03-26 07:39:00 UTC DEBUG   Not watching library /comics/test with events, disabled.
2024-03-26 07:39:00 UTC INFO    Started watching library /comics/test with poll
2024-03-26 07:39:00 UTC INFO    Waiting 16 hours until next maintenance.
2024-03-26 07:39:00 UTC INFO    Library /comics/test waiting for manual poll.
2024-03-26 07:39:00 UTC INFO    Database matches search index.

When I rerun a full import I get.

2024-03-26 07:39:00 UTC INFO    Database matches search index.
2024-03-26 07:43:10 UTC DEBUG   Poll Library /comics/test: ?/?
2024-03-26 07:43:10 UTC DEBUG   Polling /comics/test...
2024-03-26 07:43:10 UTC DEBUG   Poller sending unfiltered files: 0 deleted, 2 modified, 0 created, 0 moved.
2024-03-26 07:43:10 UTC DEBUG   Poller sending comic folders: 0 deleted, 1 modified, 0 moved.
2024-03-26 07:43:10 UTC DEBUG   Told search indexers to stop for db updates.
2024-03-26 07:43:10 UTC INFO    Polled /comics/test
2024-03-26 07:43:10 UTC DEBUG   Updating library /comics/test...
2024-03-26 07:43:10 UTC DEBUG     Comics: 1 modified
2024-03-26 07:43:11 UTC INFO    Library /comics/test waiting for manual poll.
2024-03-26 07:43:11 UTC DEBUG   Modify Folders: ?/0
2024-03-26 07:43:11 UTC DEBUG   Read Book Tags /comics/test: 0/1
2024-03-26 07:43:11 UTC DEBUG   Discover Missing Tags: ?/?
2024-03-26 07:43:11 UTC DEBUG   Create Missing Tags: ?/?
2024-03-26 07:43:11 UTC DEBUG   Modify Books: ?/1
2024-03-26 07:43:11 UTC DEBUG   Link Books to Tags: ?/?
2024-03-26 07:43:11 UTC DEBUG   Search Index Update: 0/1
2024-03-26 07:43:11 UTC DEBUG   Search Index Remove Stale Records: ?/?
2024-03-26 07:43:11 UTC DEBUG   Read Book Tags: 0/1
2024-03-26 07:43:11 UTC INFO    Reading tags from 1 comics in /comics/test...
2024-03-26 07:43:11 UTC ERROR   Failed to import: /comics/test/The Peacemaker #001 (1967).cbz
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/codex/librarian/importer/aggregate_metadata.py", line 97, in _get_path_metadata
    with Comicbox(path) as cb:
         ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/comicbox/box/init.py", line 63, in __init__
    self._config: AttrDict = get_config(config)
                             ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/comicbox/config.py", line 212, in get_config
    ad = config.get(_TEMPLATE)
         ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/core.py", line 293, in get
    return templates.as_template(template).value(self, template)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/templates.py", line 146, in value
    out[key] = typ.value(view[key], self)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/templates.py", line 146, in value
    out[key] = typ.value(view[key], self)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/templates.py", line 53, in value
    return self.get_default_value(view.name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/templates.py", line 62, in get_default_value
    raise exceptions.NotFoundError(u"{} not found".format(key_name))
confuse.exceptions.NotFoundError: comicbox.compute_pages not found
2024-03-26 07:43:11 UTC INFO    Aggregated tags from 1 comics.
2024-03-26 07:43:11 UTC DEBUG   Querying existing foreign keys for comics in /comics/test
2024-03-26 07:43:11 UTC DEBUG   Discover Missing Tags: 0/0
2024-03-26 07:43:11 UTC DEBUG   Create Missing Tags: 0/0
2024-03-26 07:43:11 UTC INFO    Updated 1 old failed imports.
2024-03-26 07:43:11 UTC INFO    No updates neccissary.
2024-03-26 07:43:11 UTC DEBUG   Told search indexers to stop for db updates.
2024-03-26 07:43:12 UTC DEBUG   Updating library /comics/test...
2024-03-26 07:43:12 UTC DEBUG     Comics: 1 modified
2024-03-26 07:43:12 UTC DEBUG     Folders: 1 modified
2024-03-26 07:43:12 UTC DEBUG   Modify Folders: ?/1
2024-03-26 07:43:12 UTC DEBUG   Read Book Tags /comics/test: 0/1
2024-03-26 07:43:12 UTC DEBUG   Discover Missing Tags: ?/?
2024-03-26 07:43:12 UTC DEBUG   Create Missing Tags: ?/?
2024-03-26 07:43:12 UTC DEBUG   Modify Books: ?/1
2024-03-26 07:43:12 UTC DEBUG   Link Books to Tags: ?/?
2024-03-26 07:43:12 UTC DEBUG   Search Index Update: 0/1
2024-03-26 07:43:12 UTC DEBUG   Search Index Remove Stale Records: ?/?
2024-03-26 07:43:12 UTC DEBUG   Modify Folders: ?/1
2024-03-26 07:43:12 UTC INFO    Modified 1 folders
2024-03-26 07:43:12 UTC DEBUG   Read Book Tags: 0/1
2024-03-26 07:43:12 UTC INFO    Reading tags from 1 comics in /comics/test...
2024-03-26 07:43:12 UTC ERROR   Failed to import: /comics/test/AAA.cbz
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/codex/librarian/importer/aggregate_metadata.py", line 97, in _get_path_metadata
    with Comicbox(path) as cb:
         ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/comicbox/box/init.py", line 63, in __init__
    self._config: AttrDict = get_config(config)
                             ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/comicbox/config.py", line 212, in get_config
    ad = config.get(_TEMPLATE)
         ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/core.py", line 293, in get
    return templates.as_template(template).value(self, template)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/templates.py", line 146, in value
    out[key] = typ.value(view[key], self)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/templates.py", line 146, in value
    out[key] = typ.value(view[key], self)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/templates.py", line 53, in value
    return self.get_default_value(view.name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/confuse/templates.py", line 62, in get_default_value
    raise exceptions.NotFoundError(u"{} not found".format(key_name))
confuse.exceptions.NotFoundError: comicbox.compute_pages not found
2024-03-26 07:43:12 UTC INFO    Aggregated tags from 1 comics.
2024-03-26 07:43:12 UTC DEBUG   Querying existing foreign keys for comics in /comics/test
2024-03-26 07:43:12 UTC DEBUG   Discover Missing Tags: 0/0
2024-03-26 07:43:12 UTC DEBUG   Create Missing Tags: 0/0
2024-03-26 07:43:12 UTC INFO    Added 1 comics to failed imports.
2024-03-26 07:43:12 UTC INFO    Updated library /comics/test in a moment. No comics to import.
2024-03-26 07:43:15 UTC DEBUG   Search Index Update: ?/?
2024-03-26 07:43:15 UTC DEBUG   Search Index Remove Stale Records: ?/?
2024-03-26 07:43:15 UTC INFO    Updating search index since 2024-03-20 09:49:32.395875+00:00...
2024-03-26 07:43:15 UTC INFO    Search index updated in a moment.
2024-03-26 07:43:15 UTC DEBUG   No stale records to remove from the search index.

Am I doing something wrong?

ajslater commented 6 months ago

No. you're fine. Dagnabbit. I'm still not sure what's going on with synology here.

tehguitarist commented 6 months ago

Whilst this is largely unhelpful in this context, it may be an intel+synology thing. I have a 923+ which is an AMD Ryzen CPU and haven't experienced this. Hopefully this helps? They're both x86_64 (I assume) but you never know.

abillauer commented 6 months ago

Mine in an x86_64 Intel(R) Atom(TM) CPU D2701 (Synology DS1813+) running DSM 7.1.1-42962 Update 6