search

Difegue / LANraragi

Web application for archival and reading of manga/doujinshi. Lightweight and Docker-ready for NAS/servers.
https://lrr.tvc-16.science
MIT License
2.27k stars 160 forks source link

Unable to scan files in the content folder #916

Closed Fmajor closed 1 year ago

Fmajor commented 1 year ago

LRR Version and OS

Linux NAS 6.1.21-gentoo-x86_64

docker version
Client:
 Version:           23.0.3
 API version:       1.42
 Go version:        go1.20.3
 OS/Arch:           linux/amd64
 Context:           default

Server:
 Engine:
  Version:          23.0.3

LRR docker image
a33fc8c0d90d   difegue/lanraragi

LRR version: Version 0.9.0

docker-compose.yml
version: "2.3"
services:
  lanraragi:
    image: difegue/lanraragi
    container_name: lanraragi
    environment:
      - LRR_UID=1000 # also test 0 (root)
      - LRR_GID=297  # also test 0 (root)
      - LRR_DATA_DIRECTORY=/mnt/data
      - LRR_THUMB_DIRECTORY=/mnt/thumb
      - LRR_TEMP_DIRECTORY=/mnt/temp
      - LRR_FORCE_DEBUG=true
    volumes:
      - /somewhere/content:/mnt/data
      - /somewhere/thumb:/mnt/thumb
      - /somewhere/temp:/mnt/temp
      - /somewhere/database:/home/koyomi/lanraragi/database
    ports:
      - 3001:3000 # HTTP port
    restart: on-failure

Bug Details LRR unable to find files.

LANraragi$ docker exec -it lanraragi sh                                                                                                                                                                                                                                      
/home/koyomi/lanraragi # env | grep LRR
LRR_GID=297
LRR_DATA_DIRECTORY=/mnt/data
LRR_AUTOFIX_PERMISSIONS=1
LRR_UID=1000
LRR_THUMB_DIRECTORY=/mnt/thumb
LRR_FORCE_DEBUG=true
LRR_TEMP_DIRECTORY=/mnt/temp
LRR_NETWORK=http://*:3000
/home/koyomi/lanraragi # cd /mnt/data
/mnt/data # ls -alh
total 105K
drwxr-xr-x    5 koyomi   koyomi         6 Nov 25 07:25 .
drwxr-xr-x    6 root     root           6 Nov 25 08:01 ..
-rw-r--r--    1 koyomi   koyomi         8 Nov 25 07:08 oshino
drwxr-xr-x    2 koyomi   koyomi        22 Nov 25 07:24 test1
drwxrwxr-x    2 koyomi   koyomi        22 Mar  2  2020 test2
drwxrwxr-x    2 koyomi   koyomi        28 Nov 23 15:52 test3
/mnt/data # ls test1
00000001.jpg  00000002.jpg  00000003.jpg  00000004.jpg  00000005.jpg  00000006.jpg  00000007.jpg  00000008.jpg  00000009.jpg  00000010.jpg  00000011.jpg  00000012.jpg  00000013.jpg  00000014.jpg  00000015.jpg  00000016.jpg  00000017.jpg  00000018.jpg  00000019.jpg  00000020.jpg
/mnt/data #

The above information shows that

But when I Rescan Archive Directory and go to see the logs, It can not search any files.

## Shinobu logs
## startup
[2023-11-25 08:01:52] [Shinobu] [info] Shinobu File Watcher started.
[2023-11-25 08:01:52] [Shinobu] [info] Content folder is /mnt/data.
[2023-11-25 08:01:52] [Shinobu] [info] Scanning content folder for changes...
[2023-11-25 08:01:52] [Shinobu] [info] Found 0 new files.
[2023-11-25 08:01:52] [Shinobu] [info] 0 files were found on the filemap but not on the filesystem.
[2023-11-25 08:01:52] [Shinobu] [debug] Number of available cores for processing: 8
[2023-11-25 08:01:52] [Shinobu] [info] Initial scan complete! Adding watcher to content folder to monitor for further file edits.
[2023-11-25 08:01:52] [Shinobu] [debug] Watcher class is File::ChangeNotify::Watcher::Inotify
[2023-11-25 08:01:52] [Shinobu] [info] All done! Now dutifully watching your files. 
## select `Rescan Archive Directory`
[2023-11-25 08:02:52] [Shinobu] [info] Shinobu File Watcher started.
[2023-11-25 08:02:52] [Shinobu] [info] Content folder is /mnt/data.
[2023-11-25 08:02:52] [Shinobu] [info] Scanning content folder for changes...
[2023-11-25 08:02:52] [Shinobu] [info] Found 0 new files.
[2023-11-25 08:02:52] [Shinobu] [info] 0 files were found on the filemap but not on the filesystem.
[2023-11-25 08:02:52] [Shinobu] [debug] Number of available cores for processing: 8
[2023-11-25 08:02:52] [Shinobu] [info] Initial scan complete! Adding watcher to content folder to monitor for further file edits.
[2023-11-25 08:02:52] [Shinobu] [debug] Watcher class is File::ChangeNotify::Watcher::Inotify
[2023-11-25 08:02:52] [Shinobu] [info] All done! Now dutifully watching your files.

I test it in two linux servers (Gentoo os, zfs file system), some result

Matching Logs

[2023-11-25 08:01:51] [LANraragi] [info] LANraragi 0.9.0 (re-)started. (Debug Mode)
[2023-11-25 08:01:51] [LANraragi] [info] Plugin Detected: Chaika.moe
[2023-11-25 08:01:51] [LANraragi] [info] Plugin Detected: Chaika.moe api.json
[2023-11-25 08:01:51] [LANraragi] [info] Plugin Detected: ComicInfo
[2023-11-25 08:01:51] [LANraragi] [info] Plugin Detected: Tag Copier
[2023-11-25 08:01:51] [LANraragi] [info] Plugin Detected: Add Timestamp Tag
[2023-11-25 08:01:51] [LANraragi] [info] Plugin Detected: E-Hentai
[2023-11-25 08:01:51] [LANraragi] [info] Plugin Detected: eze
[2023-11-25 08:01:51] [LANraragi] [info] Plugin Detected: FAKKU
[2023-11-25 08:01:51] [LANraragi] [info] Plugin Detected: Hdoujin
[2023-11-25 08:01:51] [LANraragi] [info] Plugin Detected: Hentag
[2023-11-25 08:01:51] [LANraragi] [info] Plugin Detected: Hentag Online Lookups
[2023-11-25 08:01:51] [LANraragi] [info] Plugin Detected: Hitomi
[2023-11-25 08:01:51] [LANraragi] [info] Plugin Detected: koromo
[2023-11-25 08:01:51] [LANraragi] [info] Plugin Detected: Koushoku.yaml
[2023-11-25 08:01:51] [LANraragi] [info] Plugin Detected: Mayriad's EH Master Script
[2023-11-25 08:01:51] [LANraragi] [info] Plugin Detected: Filename Parsing
[2023-11-25 08:01:51] [LANraragi] [info] Plugin Detected: nHentai
[2023-11-25 08:01:51] [LANraragi] [info] Script Detected: Subfolders to Categories
[2023-11-25 08:01:51] [LANraragi] [info] Script Detected: Source Finder
[2023-11-25 08:01:51] [LANraragi] [info] Script Detected: nHentai Source Converter
[2023-11-25 08:01:51] [LANraragi] [info] Downloader Detected: Chaika.moe Downloader
[2023-11-25 08:01:51] [LANraragi] [info] Downloader Detected: E*Hentai Downloader
[2023-11-25 08:01:51] [LANraragi] [info] Successfully connected to Minion database.
[2023-11-25 08:01:51] [LANraragi] [debug] Registered tasks with Minion.
[2023-11-25 08:01:51] [LANraragi] [debug] Shinobu Worker new PID is 119
[2023-11-25 08:01:51] [LANraragi] [info] Routing done! Ready to receive requests.
[2023-11-25 08:01:51] [Tag Stats] [info] Building stat indexes... (0 archives)
[2023-11-25 08:01:51] [Tag Stats] [info] Stat indexes built! (0 archives)
[2023-11-25 08:02:08] [Search Cache] [debug] Search request: --date_added-1-1-
[2023-11-25 08:02:08] [Search Engine] [debug] No cache available, doing a full DB parse.
[2023-11-25 08:02:08] [Categories] [debug] No category ID provided.
[2023-11-25 08:02:08] [Tag Stats] [debug] Serving tag statistics with a minimum weight of 2
[2023-11-25 08:02:08] [Search Cache] [debug] Search request: --title-0-0-0
[2023-11-25 08:02:08] [Search Engine] [debug] No cache available, doing a full DB parse.
[2023-11-25 08:02:08] [Search Cache] [debug] Search request: --date_added-1-1-
[2023-11-25 08:02:08] [Search Cache] [debug] Using cache for this query.
[2023-11-25 08:02:08] [Categories] [debug] No category ID provided.
[2023-11-25 08:02:08] [Search Cache] [debug] Search request: --title-0-0-0
[2023-11-25 08:02:08] [Search Cache] [debug] Using cache for this query.
[2023-11-25 08:02:22] [LANraragi] [info] Successful login attempt from 192.168.130.88
[2023-11-25 08:02:22] [Search Cache] [debug] Search request: --date_added-1-1-
[2023-11-25 08:02:22] [Search Cache] [debug] Using cache for this query.
[2023-11-25 08:02:22] [Tag Stats] [debug] Serving tag statistics with a minimum weight of 2
[2023-11-25 08:02:22] [Search Cache] [debug] Search request: --date_added-1-1-
[2023-11-25 08:02:22] [Search Cache] [debug] Using cache for this query.
[2023-11-25 08:02:22] [Search Cache] [debug] Search request: --title-0-0-0
[2023-11-25 08:02:22] [Search Cache] [debug] Search request: --title-0-0-0
[2023-11-25 08:02:22] [Search Cache] [debug] Using cache for this query.
[2023-11-25 08:02:22] [Search Cache] [debug] Using cache for this query.
[2023-11-25 08:02:27] [Tag Stats] [debug] Serving tag statistics with a minimum weight of 2
[2023-11-25 08:02:29] [Search Cache] [debug] Search request: --date_added-1-1-
[2023-11-25 08:02:29] [Search Cache] [debug] Using cache for this query.
[2023-11-25 08:02:29] [Tag Stats] [debug] Serving tag statistics with a minimum weight of 2
[2023-11-25 08:02:29] [Search Cache] [debug] Search request: --date_added-1-1-
[2023-11-25 08:02:29] [Search Cache] [debug] Search request: --title-0-0-0
[2023-11-25 08:02:29] [Search Cache] [debug] Using cache for this query.
[2023-11-25 08:02:29] [Search Cache] [debug] Using cache for this query.
[2023-11-25 08:02:29] [Search Cache] [debug] Search request: --title-0-0-0
[2023-11-25 08:02:29] [Search Cache] [debug] Using cache for this query.
[2023-11-25 08:02:45] [Search Cache] [debug] Search request: --date_added-1-1-
[2023-11-25 08:02:45] [Search Cache] [debug] Using cache for this query.
[2023-11-25 08:02:45] [Tag Stats] [debug] Serving tag statistics with a minimum weight of 2
[2023-11-25 08:02:45] [Search Cache] [debug] Search request: --title-0-0-0
[2023-11-25 08:02:45] [Search Cache] [debug] Using cache for this query.
[2023-11-25 08:02:45] [Search Cache] [debug] Search request: --title-0-0-0
[2023-11-25 08:02:45] [Search Cache] [debug] Using cache for this query.
[2023-11-25 08:02:45] [Search Cache] [debug] Search request: --date_added-1-1-
[2023-11-25 08:02:45] [Search Cache] [debug] Using cache for this query.
[2023-11-25 08:02:51] [LANraragi] [debug] Shinobu Worker new PID is 162
[2023-11-25 08:02:54] [Search Cache] [debug] Search request: --date_added-1-1-
[2023-11-25 08:02:54] [Search Cache] [debug] Using cache for this query.
[2023-11-25 08:02:54] [Search Cache] [debug] Search request: --date_added-1-1-
[2023-11-25 08:02:54] [Search Cache] [debug] Search request: --title-0-0-0
[2023-11-25 08:02:54] [Search Cache] [debug] Using cache for this query.
[2023-11-25 08:02:54] [Search Cache] [debug] Using cache for this query.
[2023-11-25 08:02:54] [Search Cache] [debug] Search request: --title-0-0-0
[2023-11-25 08:02:54] [Search Cache] [debug] Using cache for this query.
[2023-11-25 08:02:54] [Tag Stats] [debug] Serving tag statistics with a minimum weight of 2

How can I debug that?

Fmajor commented 1 year ago

My fault. According to the document

The following formats are supported by LRR for Archives:
zip/cbz
rar/cbr (up to RAR4 only)
tar.gz/cbt
lzma
7z/cb7
xz
pdf
epub (images only if viewed in the Web Client or through the Client API, potentially out of order)

so I should make tars or zips for my files.