ShokoAnime / ShokoServer

Repository for Shoko Server.
https://shokoanime.com/
MIT License
407 stars 74 forks source link

Lot off errors in serverlog #539

Closed kusco123 closed 7 years ago

kusco123 commented 7 years ago

FYI 2017-01-01.zip

Kezxo commented 7 years ago

Same issue here! my server log shows this over and over:

[2017-01-01 15:08:53:679] Warn|ErrorEventHandler.Invoke => BufferingFileSystemWatcher.InvokeHandler => RecoveringFileSystemWatcher.BufferingFileSystemWatcher_Error Felaktig parameter
[2017-01-01 15:08:53:693] Warn|ErrorEventHandler.Invoke => BufferingFileSystemWatcher.InvokeHandler => RecoveringFileSystemWatcher.BufferingFileSystemWatcher_Error Will try to recover automatically!
[2017-01-01 15:08:53:693] Warn|ErrorEventHandler.Invoke => BufferingFileSystemWatcher.InvokeHandler => RecoveringFileSystemWatcher.BufferingFileSystemWatcher_Error Felaktig parameter

and sometimes this:

[2017-01-01 15:08:53:654] Warn|BufferingFileSystemWatcher.InvokeHandler => ErrorEventHandler.Invoke => RecoveringFileSystemWatcher.BufferingFileSystemWatcher_Error Will try to recover automatically!
[2017-01-01 15:08:53:661] Warn|BufferingFileSystemWatcher.InvokeHandler => ErrorEventHandler.Invoke => RecoveringFileSystemWatcher.BufferingFileSystemWatcher_Error Felaktig parameter
[2017-01-01 15:08:53:661] Warn|BufferingFileSystemWatcher.InvokeHandler => ErrorEventHandler.Invoke => RecoveringFileSystemWatcher.BufferingFileSystemWatcher_Error Will try to recover automatically!
RickDB commented 7 years ago

This new file system watched I'm not familiar with but looked over the code for it and can be caused by 2 things:

Now what comes to mind first is that the file system watcher triggers but afterwards file is no longer there or it has a cloud drive it tries to monitor. @maxpiva any ideas on this one?

Kezxo commented 7 years ago

I'm watching some network folders that are running samba on their end. May be related I got them mounted as network drives in windows but they are watched under \\serverhostname\folder\ instead of the mapped drive path. Z:\folder Should i use the paths to the mapped network drives instead? I notice when I try to add a new folder the only option is "local folder", maybe something that changed? I don't remember what I used when I added them initially or why.

I tried running server as admin, the log still does that stuff.

RickDB commented 7 years ago

Would try it with UNC paths as those generally do better with Shoko Server, especially during reconnects or startup mapped drives can have issues and potentially trigger a file system watch event.

Local folder name is normal and can change mapped drive to UNC path there, that was changed because of the new cloud support I believe. Would suggest a database backup beforehand, while during my testing here it did a rename right away with no re-indexing for some users it re-indexed all their files as if it was a new path, also make sure to use the same root path as with the mapped drive.

ElementalCrisis commented 7 years ago

Seems we see this a lot with mapped drives. What about adding in some kind of check or at the very least a message about using the actual location instead of the mapped drive?

Kezxo commented 7 years ago

I I'll leave it like it is then since it Shoko is using UNC already. Could mapped drives that shoko isn't watching cause issues?

I might remove the folders and re-add them again later, I don't rename so that should be pretty painless hopefully.

kusco123 commented 7 years ago

From my side there are no mapped drives - only unc paths. I was trying now for some time (weeks) to import folder per folder (as i have / had issues with files moving also - not moved after recognsion - to see if I can pinpoint ist somehow) - now I'm again at a folder with lots of files in it (Naruto ... 659 file in 19 Folders) ) and it seems to get stuck and only producing the errors. 2017-01-01.zip [Uploading 2017-01-02.zip…]()

se errors ...

maxpiva commented 7 years ago

The new Filesystem watcher has the ability to reconnect after network losses, the normal filesystem watcher, when there is a network loss, or the drive is disconnected for a moment, the watcher will stop sending changes. We may move the logging from warn to debug.

On Mon, Jan 2, 2017 at 3:36 PM, kusco123 notifications@github.com wrote:

Reopened #539 https://github.com/japanesemediamanager/ShokoServer/issues/539.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/japanesemediamanager/ShokoServer/issues/539#event-909191221, or mute the thread https://github.com/notifications/unsubscribe-auth/ABHDbbvnQc6sYAOGiyf8VPrkFwkHuTBsks5rOUPKgaJpZM4LYtkI .

yhoogi commented 7 years ago

Have to report some strange behaviour on server-side. (v3.7.0.1)

Seems like the server is not monitoring the drop folder if it is on a different computer in the network. (either as UNC or via an associated drive letter)

Local folders are monitored and dropped files are immediatly processed

As the function worked with the JMM 3.6.x , looks like a bug introduced with the new version

2nd observation: If you manually delete a file Shoko Desktop seems to be unable to delet it out of the DB Steps reproduce: Delete a file Go to Episodelist of the series in Shoko Desktop Try to delete the file in episode list will result in an error message that the file can not be found .. & it is not deleted from the episode list

Btw: I have a ton of error messages in the log, like: [2017-01-04 13:32:37:730] Warn|ErrorEventHandler.Invoke => BufferingFileSystemWatcher.InvokeHandler => RecoveringFileSystemWatcher.BufferingFileSystemWatcher_Error Falscher Parameter [2017-01-04 13:32:37:730] Warn|ErrorEventHandler.Invoke => BufferingFileSystemWatcher.InvokeHandler => RecoveringFileSystemWatcher.BufferingFileSystemWatcher_Error Will try to recover automatically! [2017-01-04 13:32:38:847] Info|SyncMethodInvoker.Invoke => .SyncInvokeDeleteVideoLocalPlaceAndFile => JMMServiceImplementation.DeleteVideoLocalPlaceAndFile Deleting video local place record and file: \DIOMEDES\Anime\Share\Drifters[HorribleSubs] Drifters - 01 [1080p].mkv [2017-01-04 13:32:38:863] Error|SyncMethodInvoker.Invoke => .SyncInvokeDeleteVideoLocalPlaceAndFile => JMMServiceImplementation.DeleteVideoLocalPlaceAndFile Unable to find file '\DIOMEDES\Anime\Share\Drifters[HorribleSubs] Drifters - 01 [1080p].mkv' [2017-01-04 13:32:39:471] Warn|ErrorEventHandler.Invoke => BufferingFileSystemWatcher.InvokeHandler => RecoveringFileSystemWatcher.BufferingFileSystemWatcher_Error Falscher Parameter [2017-01-04 13:32:39:472] Warn|ErrorEventHandler.Invoke => BufferingFileSystemWatcher.InvokeHandler => RecoveringFileSystemWatcher.BufferingFileSystemWatcher_Error Will try to recover automatically! [2017-01-04 13:32:41:148] Warn|ErrorEventHandler.Invoke => BufferingFileSystemWatcher.InvokeHandler => RecoveringFileSystemWatcher.BufferingFileSystemWatcher_Error Falscher Parameter​

.. these error messages are cloging the log - file, bringing it easiliy in the GB range.

As it started after the update, I attach the associated log-file created after update

2017-01-02.zip

yhoogi commented 7 years ago

Strange thing that in a LAN there are so many disconnections This is hard to believe as I view all my movies (incl. BDs) via network w/o a glitch

RickDB commented 7 years ago

The error in parameter does look like something in code is going wrong, @maxpiva for Nutzcode we added some workarounds for mappings / shares a while back so maybe we need to the same for the file system monitor or think it's something else entirely?

As looking thru logs we can see it hash them properly and the moment it detects a new file on share it throws that error.

kusco123 commented 7 years ago

The drop folder not being scanned automtically I have also since the 3.7.0 Beta. I think I also have reported this? What I also see now is very long responsetime when adding a file manually (30 minutes?!) until the Desktop responds again. I don't see any indication why it takes so long in teh log ... I only see when I add it.

RickDB commented 7 years ago

For local shares it's instant as I tested that some more this week, checking with UNC again on daily build now to see if it same error shows up.

// Update

Exact same error on local servers UNC share (both drop / destination) with latest daily.

RickDB commented 7 years ago

Found solution and pushing in a sec, will be part of next daily build.

kusco123 commented 7 years ago

Can confirm with build 1021 the Drop Folder works again. Also the Log entries are now gone (I think it was set to a different logging level?).

My only issue now is response time - it still takes a long time when matching a file manually until the client responds again - can soemone confirm this?

RickDB commented 7 years ago

Yep log level reduced to debug builds only, so we can supply to user if we need more info :)

Response time was ok here for UNC at least (DB: MSSQL 2017), hashing is still low but that is normal and right after that shows up in client.

kusco123 commented 7 years ago

What I can see now is a very slow import rate ... taking ages with "getting file info"

2017-01-06.zip

da3dsoul commented 7 years ago

Is this closeable?

kusco123 commented 7 years ago

yes