search

MediaBrowser / Emby

Emby Server is a personal media server with apps on just about every device.
https://emby.media
GNU General Public License v2.0
4.18k stars 810 forks source link

Library Scanning Halts #284

Closed ginjaninja1 closed 11 years ago

ginjaninja1 commented 11 years ago

I have reproduced this when scanning tv librarys from scratch, fresh install, no metadata.

this one was for movies. https://dl.dropboxusercontent.com/u/84611964/Server-635043936879520470.rar

the log windows just stops, no timeout errors, nothing. sysinternals just shows mediabrowser doing thread create/close indefinitely

image

solution is to stop and start scan library task. only the scan task itself has paused, if new content is added or ammend, these changes are reflected in additional library activity.

ebr11 commented 11 years ago

Does the task % number on the dashboard ever change?

ginjaninja1 commented 11 years ago

i havent watched it for longer than a few minutes, but yes the % seems to never change.

ginjaninja1 commented 11 years ago

I was looking for 0 byte corrupt jpgs and xmls, (they used to cause a problem before i seem to remember)..none found. I was looking through the last tv to be scanned before the last halt. I found a season without any content, when i deleted it, some seemingly 'pent up' tv scanning activity came through and the scan task which had been halted for 20mins, was finished (though this might have been coincidence). will try and reproduce.

ginjaninja1 commented 11 years ago

might be getting somewhere, just halted again 'near to' the good wife...found an empty season folder in the good wife...deleted...the scan task completed straight after (with a flurry of activity in the logs) https://dl.dropboxusercontent.com/u/84611964/Server-635043983062141965.rar

ginjaninja1 commented 11 years ago

and again for true blood season 6 (log above updated). Seems the fetchers dont like blank seasons. My Management program creates season folders as soon as they are published on TVDB, dont know if others will see this issue. Now i think about it as i tested music, i moved over to tv. and i was still on tv when i moved to movies, so perhaps it was just tv that it was hanging on...

ebr11 commented 11 years ago

Looks like good detective work. Thanks.

mrwebsmith commented 11 years ago

i may be hitting this as well as part of my slower media scans... ill also give and fyi to jon to see if it helps him also

On Fri, May 17, 2013 at 12:23 PM, Eric Reed notifications@github.comwrote:

Looks like good detective work. Thanks.

— Reply to this email directly or view it on GitHubhttps://github.com/MediaBrowser/MediaBrowser/issues/284#issuecomment-18071555 .

ginjaninja1 commented 11 years ago

Its not guaranteed that an empty folder halts the scan but since i removed them all, the scan task finishing and the lines saying 'sql delayed writer X commands'' seem more frequent and assured.

ginjaninja1 commented 11 years ago

if others can't reproduce this, i am developing a theory that there is some sort of upper error limit for tv fetching. As i remove series with 4 digit seasons, and blank folders, more of the library is indexed properly. Its like MB3 is not moving onto the good bits of the library after it hits X problem areas.

ginjaninja1 commented 11 years ago

Think this issue might be peculiar to myself. I was using old series.xmls from another fetching utility.. My series are all suffixed with Seriesname (YYYY).

I had thought that these were ok since MB3 was fetching the xml for the episodes within, but on reflection this might have been causing MB3 an issue. The above still stands, but it may be peculiar to me as i was causing MB3 undue hassle (as it currently doesnt like 'Seriesname (YYYY)')

ebr11 commented 11 years ago

I don't think this issue is just isolated to you. I find my scans on my server routinely stuck at about 65% (always seems to be about that level). The activity in the log and everywhere else just seems to stop. The task never reports it is completed and you can't stop it. Pressing stop on the dash just puts it into a perpetual "Stopping" state.

Something is definitely up here but I can't tell what it is.

LukePulverenti commented 11 years ago

I see one potential problem here. Near the end of the log, the movie db provider saves json within the movie directory, and then the directory watchers are firing in response to that. I verified the path of the json file is getting passed into TemporarilyIgnore, so something is going wrong within DirectoryWatchers. I think what's happening is there's a changed event on the containing directory and we're not ignoring that.

That's not say this will fix the halt, but it's worth fixing this and then re-testing.

LukePulverenti commented 11 years ago

@ginjaninja1 Please test in next build.

ginjaninja1 commented 11 years ago

sure, but i wont be back to test until thursday this week.

ginjaninja1 commented 11 years ago

I dont think i am having any library scanning halts with movie ingestion into the library now.

If i let mb3 download and save all metadata locally in media folders, then delete the database to get MB3 to index my library from scratch, the library scan completes but the library is not built properly

Scan task finished OK, no log activity for 5 mins, No disk, cpu, mb3 activity, but library is not built image Episodes do not show (even unfetched) image looking at the internet usage monitor and logs its seems that the tv backdrops are redownloaded unneccessarily. image

Ran library scan again at 17:15 but this time it just halted. no disk activity, nothing. lots of running xyzprovider messages but no activity image

log from initial library deletion and restart of mb3 https://dl.dropboxusercontent.com/u/84611964/Server-635049235103693836.log to scan task finnished, no activity and library not finished, then run scan task 2nd time, task halted.

after no activity for 2 and half minutes, with library scan halted, i closed mb3 and restarted loads more activity, scan completed ok...more/all? of the library indexed/viewable in the dashboard.

image. activity all stopped....ran the scan library task at 19:00...more library activity - shouldnt this all have finished after the (multiple) previously sucessfull scan tasks (which completed ok)

log from close and restart of mb3. past 2nd scan at 19:00 to 2nd scan activty finish @ 19:09, through 3rd scan at 19:10, 3rd scan completes at 19:11. mostly previously errors in 3rd scan but still some separate library activtiy? https://dl.dropboxusercontent.com/u/84611964/Server-635049264742219063.log

summary conclusions of potential problems 1) the scan task halting, perhaps these should go into separate issues? 2) the scan task completing but not being effective (the librarys not showing content) 3) the scan task re-evaluating unchanged media which should have finished with ingestion a long time ago.

FWIW heres an example from before of what the logs look like when i delete a blank season folder. The scope of the impact, looking at the log messages, is much wider than expected? - other seasons in the series are checked? or enumerated? see log following 2013-05-23 09:04:51.2364, Info, DirectoryWatchers, Watcher sees change of type Deleted to T:\TV2\Stargate SG-1\Season 11. https://dl.dropboxusercontent.com/u/84611964/Server-635048947654692693.log

LukePulverenti commented 11 years ago

we have some known issues with the log window. please disable it, then delete your library db and try again.

ginjaninja1 commented 11 years ago

with log window disabled it does seem more assured. no halts reproducible (tv ingestion, no blank folders). https://dl.dropboxusercontent.com/u/84611964/Server-635050282612103712.log https://dl.dropboxusercontent.com/u/84611964/Server-635050367999996598.log

ebr11 commented 11 years ago

What is the issue with the log window? I just had a large library scan stop twice at the exact same spot.

micahscopes commented 7 years ago

I'm having this issue with the latest version from git, as well as 3.1.1-1. Running Arch Linux.

I'm scanning a folder filled with nested directories of audio files, cover images and text.

In journalctl I always get something like this: 

Native stacktrace:
 emby-server[12604]:         /usr/bin/mono() [0x4b18ff]
 emby-server[12604]:         /usr/bin/mono() [0x50769e]
 emby-server[12604]:         /usr/bin/mono() [0x4297b3]
 emby-server[12604]:         /usr/lib/libpthread.so.0(+0x11080) [0x7f6d9524b080]
 emby-server[12604]: Debug info from gdb:

It always occurs after an apparently successful ffprobe log entry, not necessarily for the same file.

If I restart emby and attempt to rescan, it freezes at 0%. If I delete my library.db and rescan, it freezes in the exact same spot that it froze in the first time (82.1%). This seems like a recurring issue from what I can tell.

I'm having a hard time finding any other info. Are there other logs somewhere?