search

Kareadita / Kavita

Kavita is a fast, feature rich, cross platform reading server. Built with the goal of being a full solution for all your reading needs. Setup your own server and share your reading collection with your friends and family.
http://www.kavitareader.com
GNU General Public License v3.0
5.9k stars 304 forks source link

ComicVine type library not scanning properly when using folders by Publisher #3110

Open Roshri opened 4 weeks ago

Roshri commented 4 weeks ago

What happened?

From Discord

My comicvine type library is not working properly when doing a normal scan. It's currently set up like this Comics/Publisher/SeriesName (Year)

with Comics being the root folder for the library. When doing a normal scan it doesn't pick up new files, so I have to do a Force Scan every time I add something new

majora's answer

Yeah, it's work built upon that. When we scan, we take series paths and check them. But with this enhancement, when you use a Publisher folder and it has modifications, instead of saying "Scan Publisher", it will now do a "dirty check" to check all the known series within.

But now that I'm talkinga bout it, there might be an edge case where there are new folders that Kavita might miss.

What did you expect?

The normal scan should pick up new files (and folders)

Kavita Version Number - If you don not see your version number listed, please update Kavita and see if your issue still persists.

0.8.2 - Stable

What operating system is Kavita being hosted from?

Windows

If the issue is being seen on Desktop, what OS are you running where you see the issue?

Windows

If the issue is being seen in the UI, what browsers are you seeing the problem on?

Firefox

If the issue is being seen on Mobile, what OS are you running where you see the issue?

Android

If the issue is being seen on the Mobile UI, what browsers are you seeing the problem on?

Firefox

Relevant log output

No response

Additional Notes

No response

DieselTech commented 4 weeks ago

Here is the notes that I have on the scanner for the CV library type I haven't gotten down to figuring out what is going on yet, just in the note taking phase:

[Kavita] [2024-07-16 18:23:41.234 -04:00  96] [Information] API.Services.TaskScheduler Enqueuing library scan for: 1
[Kavita] [2024-07-16 18:23:41.268 -04:00  96] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP POST /api/library/scan?libraryId=1&force=false responded 200 in 67.0881 ms
[Kavita] [2024-07-16 18:23:41.360 -04:00  93] [Information] API.Services.Tasks.ScannerService [ScannerService] Beginning file scan on Comics
[Kavita] [2024-07-16 18:23:41.388 -04:00  93] [Debug] API.Services.Tasks.ScannerService [ScannerService] Library Comics Step 1: Scan Files
[Kavita] [2024-07-16 18:23:41.667 -04:00  93] [Debug] API.Services.Tasks.ScannerService [ScannerService] Library Comics Step 1.A: Process 1 folders
[Kavita] [2024-07-16 18:23:41.667 -04:00  93] [Debug] API.Services.Tasks.ScannerService     [ScannerService] Library Comics Step 1.B: Scan files in /comics
[Kavita] [2024-07-16 18:23:41.673 -04:00  96] [Debug] API.Controllers.LibraryController Caching libraries for library_lsiotest
[Kavita] [2024-07-16 18:23:41.675 -04:00  96] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET /api/library/libraries responded 200 in 2.5990 ms
[Kavita] [2024-07-16 18:23:42.983 -04:00  89] [Debug] API.Controllers.LibraryController Caching libraries for library_lsiotest
[Kavita] [2024-07-16 18:23:50.046 -04:00  93] [Debug] API.Services.Tasks.ScannerService     [ScannerService] Library Comics Step 1.C: Process files in /comics

Almost 2 full days later the scan completes:

[Kavita] [2024-07-18 16:18:05.883 -04:00  93] [Debug] API.Services.Tasks.ScannerService [ScannerService] Library Comics Step 4: Save Library
[Kavita] [2024-07-18 16:18:15.798 -04:00  93] [Information] API.Services.Tasks.ScannerService [ScannerService] Finished library scan of 81365 files and 9672 series in 165274547 milliseconds for Comics
[Kavita] [2024-07-18 16:18:15.798 -04:00  93] [Debug] API.Services.Tasks.ScannerService [ScannerService] Library Comics Step 5: Remove Deleted Series
[Kavita] [2024-07-18 16:18:16.284 -04:00  93] [Debug] API.Services.Tasks.ScannerService [ScannerService] Removing Series that were not found during the scan
[Kavita] [2024-07-18 16:19:41.747 -04:00  93] [Debug] API.Services.Tasks.ScannerService [ScannerService] Found 15 series that needs to be removed: ["Star Trek: The Next Generation: The Gorn Crisis (2001)","Star Trek: The Next Generation: Forgiveness (2001)","Albion (2005)","Majestic (2005)","Countdown (2000)","Manifest Eternity (2006)","Star Trek: The Next Generation: Embrace the Wolf (2000)","Star Trek: The Next Generation -- The Killing Shadows (2000)","Captain Atom: Armageddon (2005)","21 Down (2002)","Planetary/The Authority: Ruling the World (2000)","The X-Files/30 Days of Night (1)","Planetary: Crossing Worlds (2004)","Star Trek: The Next Generation - Enemy Unseen (2001)","Planetary (1999)"]
[Kavita] [2024-07-18 16:19:41.756 -04:00  93] [Debug] API.Services.Tasks.ScannerService [ScannerService] Removing Series that were not found during the scan - complete
[Kavita] [2024-07-18 16:22:35.991 -04:00  29] [Information] API.Services.Tasks.CleanupService Performing cleanup of Cache directories
[Kavita] [2024-07-18 16:22:35.991 -04:00  29] [Information] API.Services.Tasks.CleanupService Cache directory purged

Once the scan was done I started a new scan the next day:

[Kavita] [2024-07-19 17:49:51.471 -04:00  80] [Information] API.Services.TaskScheduler Enqueuing library scan for: 1
[Kavita] [2024-07-19 17:49:51.531 -04:00  80] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP POST /api/library/scan?libraryId=1&force=false responded 200 in 98.0060 ms
[Kavita] [2024-07-19 17:49:51.556 -04:00  53] [Information] API.Services.Tasks.ScannerService [ScannerService] Beginning file scan on Comics
[Kavita] [2024-07-19 17:49:51.572 -04:00  53] [Debug] API.Services.Tasks.ScannerService [ScannerService] Library Comics Step 1: Scan Files
[Kavita] [2024-07-19 17:49:51.783 -04:00  53] [Debug] API.Services.Tasks.ScannerService [ScannerService] Library Comics Step 1.A: Process 1 folders
[Kavita] [2024-07-19 17:49:51.783 -04:00  53] [Debug] API.Services.Tasks.ScannerService     [ScannerService] Library Comics Step 1.B: Scan files in /comics
[Kavita] [2024-07-19 17:49:52.632 -04:00  53] [Debug] API.Services.Tasks.ScannerService [ProcessFiles] /comics/Boom Studios is dirty and has multiple series folders, checking if we can avoid a full scan
[Kavita] [2024-07-19 17:50:32.181 -04:00  53] [Debug] API.Services.Tasks.ScannerService     [ScannerService] Library Comics Step 1.C: Process files in /comics
[Kavita] [2024-07-19 18:08:51.561 -04:00  53] [Debug] API.Services.Tasks.ScannerService [ScannerService] Library Comics Step 2: Track Found Series
[Kavita] [2024-07-19 18:08:51.570 -04:00  53] [Debug] API.Services.Tasks.ScannerService [ScannerService] Library Comics Step 3: Process Parsed Series

I can't see it in the logs, but it said there was over 6000 changes. I just stopped the container at that point instead of letting it scan for 3 more days. I know this isn't down to how the file system behaves because my manga library in a different container does not do this.