search

Kareadita / Kavita

Kavita is a fast, feature rich, cross platform reading server. Built with the goal of being a full solution for all your reading needs. Setup your own server and share your reading collection with your friends and family.
http://www.kavitareader.com
GNU General Public License v3.0
5.62k stars 282 forks source link

[Nightly] Scheduled scan happening simultaneously with manual scan #3015

Open Cducharme84 opened 2 weeks ago

Cducharme84 commented 2 weeks ago

What happened?

On the latest nightly build if there's currently a scan going the scheduled scan will kick off and the two scans will alternate processing, outlined in both the gui (screens here first is an example of new/updated series being scanned into the DB out of order, there was a scan going for #-F already that didn't finish so the M-S was unexpected and right after the new day started) and in logs composite two scans.log here.

The scan for #-F was kicked off approximately 4 hours before the scheduled scan, there were thousands of changes as this was during the retagging expedition for genre data from GCD so it was still working it when the scheduled came up. Morsok (from Discord) and I both had noticed it the night before but hadn't captured good logs/screens yet so I was intentionally trying to replicate.

It does seem to slow down the scans, which was also noticed by us both some of my series of 1 or 2 issues took 10s+ and I suspect the root cause is something to do with the change in the root folder updating, both him and I had improper paths set still from the last nightly and I noticed anecdotally that the paths that needed to get updated in the logs were series that took a longer duration to scan most of the usual operation of Kavita since we use Linux hosts.

What did you expect?

The scheduled scan to queue but not be performed concurrently with the manual.

Kavita Version Number - If you don not see your version number listed, please update Kavita and see if your issue still persists.

Nightly Testing Branch

What operating system is Kavita being hosted from?

Linux

If the issue is being seen on Desktop, what OS are you running where you see the issue?

None

If the issue is being seen in the UI, what browsers are you seeing the problem on?

No response

If the issue is being seen on Mobile, what OS are you running where you see the issue?

None

If the issue is being seen on the Mobile UI, what browsers are you seeing the problem on?

No response

Relevant log output

Much too large see attached file in main section.

Additional Notes

No response

majora2007 commented 2 weeks ago

This can be a bit tricky as it behaves much different if you don't have massive libraries. I will first try with some code that kills the current request and requeues it for 10 mins in the future. That is extremely cheap and should work well on both smaller and larger systems.