ajslater / codex

Codex is a web based comic archive browser and reader
GNU General Public License v3.0
184 stars 6 forks source link

[Feature Request] No Metadata import on read #366

Open abillauer opened 3 months ago

abillauer commented 3 months ago

With the "Import Metadata on Library Scan" unchecked, codex will read metadata of the 3 comics per read and update the search index.

This process can takes a significant amount of time on older hardware (log snippet below).

For faster performance on older servers can there be a flag that fully disables reading "additional" tags on comics when they are read? Having metadata only on read comics is of limited value anyway. The performance boost would be useful.

codex v1.5.12 - docker

2024-03-28 10:13:29 UTC DEBUG   Read Book Tags: 0/3
2024-03-28 10:13:29 UTC INFO    Reading tags from 3 comics in /comics/ZZZ...
2024-03-28 10:13:29 UTC INFO    Aggregated tags from 3 comics.
2024-03-28 10:13:29 UTC DEBUG   Querying existing foreign keys for comics in /comics/ZZZ
2024-03-28 10:13:29 UTC DEBUG   Discover Missing Tags: 0/11
2024-03-28 10:13:29 UTC INFO    Prepared 3 Series for creation.
2024-03-28 10:13:29 UTC INFO    Prepared 3 Volumes for creation.
2024-03-28 10:13:30 UTC DEBUG   Prepared 0 new Folders.
2024-03-28 10:13:30 UTC DEBUG   Create Missing Tags: 0/6
2024-03-28 10:13:30 UTC INFO    Created 3 Series.
2024-03-28 10:13:30 UTC INFO    Created 3 Volumes.
2024-03-28 10:13:30 UTC DEBUG   Modify Books: ?/3
2024-03-28 10:13:30 UTC DEBUG   Preparing 3 comics for update in library /comics/ZZZ.
2024-03-28 10:13:30 UTC DEBUG   Bulk updating 3 comics.
2024-03-28 10:13:30 UTC DEBUG   Purging covers for 3 updated comics.
2024-03-28 10:13:30 UTC INFO    Updated 3 comics.
2024-03-28 10:13:30 UTC DEBUG   Removing 3 possible cover thumbnails...
2024-03-28 10:13:30 UTC DEBUG   Link Books to Tags: ?/3
2024-03-28 10:13:30 UTC DEBUG   Remove Covers: 0/3
2024-03-28 10:13:30 UTC INFO    Removed 3 cover thumbnails.
2024-03-28 10:13:30 UTC INFO    Updated library /comics/ZZZ in a moment. Imported 3 comics at 5.6 comics per second.
2024-03-28 10:13:33 UTC DEBUG   Search Index Update: ?/?
2024-03-28 10:13:33 UTC DEBUG   Search Index Remove Stale Records: ?/?
2024-03-28 10:14:25 UTC DEBUG   Told search indexers to stop for db updates.
2024-03-28 10:14:25 UTC DEBUG   Updating library /comics/ZZZ...
2024-03-28 10:14:25 UTC DEBUG     Comics: 1 modified
2024-03-28 10:14:25 UTC DEBUG   Modify Folders: ?/0
2024-03-28 10:14:25 UTC DEBUG   Read Book Tags /comics/ZZZ: 0/1
2024-03-28 10:14:25 UTC DEBUG   Discover Missing Tags: ?/?
2024-03-28 10:14:25 UTC DEBUG   Create Missing Tags: ?/?
2024-03-28 10:14:25 UTC DEBUG   Modify Books: ?/1
2024-03-28 10:14:25 UTC DEBUG   Link Books to Tags: ?/?
2024-03-28 10:14:25 UTC DEBUG   Search Index Update: 0/1
2024-03-28 10:14:25 UTC DEBUG   Search Index Remove Stale Records: ?/?
2024-03-28 10:14:25 UTC DEBUG   Read Book Tags: 0/1
2024-03-28 10:14:25 UTC INFO    Reading tags from 1 comics in /comics/ZZZ...
2024-03-28 10:15:57 UTC INFO    Read Book Tags: 1/1
2024-03-28 10:15:57 UTC INFO    Aggregated tags from 1 comics.
2024-03-28 10:15:57 UTC DEBUG   Querying existing foreign keys for comics in /comics/ZZZ
2024-03-28 10:15:57 UTC DEBUG   Discover Missing Tags: 0/5
2024-03-28 10:15:57 UTC INFO    Prepared 1 Series for creation.
2024-03-28 10:15:57 UTC INFO    Prepared 1 Volumes for creation.
2024-03-28 10:15:57 UTC DEBUG   Prepared 0 new Folders.
2024-03-28 10:15:57 UTC DEBUG   Create Missing Tags: 0/2
2024-03-28 10:15:57 UTC INFO    Created 1 Series.
2024-03-28 10:15:57 UTC INFO    Created 1 Volumes.
2024-03-28 10:15:57 UTC DEBUG   Modify Books: ?/1
2024-03-28 10:15:57 UTC DEBUG   Preparing 1 comics for update in library /comics/ZZZ.
2024-03-28 10:15:58 UTC DEBUG   Bulk updating 1 comics.
2024-03-28 10:15:58 UTC DEBUG   Purging covers for 1 updated comics.
2024-03-28 10:15:58 UTC INFO    Updated 1 comics.
2024-03-28 10:15:58 UTC DEBUG   Removing 1 possible cover thumbnails...
2024-03-28 10:15:58 UTC DEBUG   Remove Covers: 0/1
2024-03-28 10:15:58 UTC INFO    Removed 1 cover thumbnails.
2024-03-28 10:15:58 UTC DEBUG   Link Books to Tags: ?/1
2024-03-28 10:15:58 UTC INFO    Updated library /comics/ZZZ in a minute. Imported 1 comics at 0.0 comics per second.
2024-03-28 10:16:11 UTC INFO    Updating search index since 2024-03-28 10:04:47.712708+00:00...
2024-03-28 10:16:11 UTC INFO    Search index updated in 2 minutes.
2024-03-28 10:16:11 UTC DEBUG   Search Index Update: ?/?
2024-03-28 10:16:11 UTC DEBUG   Search Index Remove Stale Records: ?/?
2024-03-28 10:18:47 UTC INFO    Updating search index since 2024-03-28 10:04:47.712708+00:00...
2024-03-28 10:18:48 UTC DEBUG   Search Index Update: 0/4
2024-03-28 10:18:48 UTC DEBUG   MMAP_RATIO=240, WRITER_MEMORY_PERCENT=0.6, CPU_MULTIPLIER=1.25, CHUNK_PER_GB=250, MAX_CHUNK_SIZE=1000
2024-03-28 10:18:48 UTC DEBUG   Search Index update opts: {'comics': 4, 'memgb': 15.522666931152344, 'procs': 4, 'batch_size': 1}
2024-03-28 10:18:48 UTC DEBUG   Search index updating 4 batches, attempt 1
2024-03-28 10:18:48 UTC DEBUG   Search index update queued 4 batches...
2024-03-28 10:18:48 UTC DEBUG   Search index starting final commit for batch 1.
2024-03-28 10:18:48 UTC DEBUG   Search index starting final commit for batch 0.
2024-03-28 10:18:48 UTC DEBUG   Search index batch 0/4 complete: 0/4 comics
2024-03-28 10:18:48 UTC DEBUG   Search index batch 1/4 complete: 1/4 comics
2024-03-28 10:18:49 UTC DEBUG   Search index starting final commit for batch 3.
2024-03-28 10:18:49 UTC DEBUG   Search index starting final commit for batch 2.
2024-03-28 10:18:49 UTC DEBUG   Search index batch 2/4 complete: 2/4 comics
2024-03-28 10:18:49 UTC DEBUG   Search index batch 3/4 complete: 3/4 comics
2024-03-28 10:18:49 UTC DEBUG   Search Index attempt 1 batch success ratio: 100%
2024-03-28 10:18:49 UTC INFO    Search engine updated 4 comics in a second at 2 comics per second.
2024-03-28 10:18:49 UTC INFO    Search index updated in 2 minutes.
2024-03-28 10:19:17 UTC DEBUG   No stale records to remove from the search index.
2024-03-28 10:19:55 UTC DEBUG   No stale records to remove from the search index.