NZBHydra (main process) freezes after a few days.

[torque1]

After using NZBHydra2, as a service on Windows 10, for some while now a recent update has caused it to freeze after running for a few days. The UI is non responsive (timed out), the logs only show cleanup tasks. I can see the process still running in task manager. Restarting the windows service bring the NZBHydra2 back to life.

I have completely removed the Hydra2 directory and started the setup again from scratch but the issue remains. Prior to this I also increased the memory to 1024MB. I set the logging to debug level a few days ago and noted a freeze again this morning. I've attached the logs. I've taken a look into them but can't see any relevant errors.

Current version is 2.3.15. The issue started after updating to this version, but I'm not sure which one I was on previously. Last update probably around 3 months ago.

I've created a schedule task to restart the service daily as a work around, but happy to disable this if you need any further testing,.

nzbhydra-debuginfos-2019-03-07-18-36.zip

[theotherp]

I see warnings that the config file is used by another process. The database has had some issues as well. Make sure Hydra can access its data files and that not two instances are running at the same time.

[torque1]

There are always two hydra processes running, I assume one is a launcher. I do have a backup program running but it uses VSS snapshot so should not restrict access to the files. I also have Windows Defender as an AV. Nothing else will be accessing the files.

I found the version I was on previously was 1.5.2 I’ve reverted back to this and it’s been stable for 3 days. That’s longer than 2.3.15 every ran for.

I’ll leave this for another few days to be sure it’s not a coincidence and then upgrade again to see if I get the same problem.

[theotherp]

Well, 1.5.2 is ancient and too many things have changed since then to pinpoint what could cause this.

[torque1]

I’ll move to 2.3.15 and then keep downgrading a few versions at a time until it’s stable. May take a few weeks but I’ll hopefully pinpoint it.

[Kurisutian]

I'm experiencing a similar problem on my QNAP NAS running version 2.3.15. Don't know if it's related but no matter what I do after a day or maybe two if I'm lucky NZBHydra stops working, but there is a java process for it which keeps running. I cannot send API requests nor open it in a browser so every day I have to log in to the NAS, kill (-9) the java process as nothing else works. With the process running I cannot use to init scripts so I have to do several manual steps to get it back working but only for that little time and it repeats. So right now this is unfortunately very annoying to have to check and fix it almost daily. Is there anything I can do to keep it running stable for way longer? Thanks!

[theotherp]

@Kurisutian Please post your debug infos ZIP.

[torque1]

I've been testing different versions and so far it appears the issue was introduces in between 2.3.10 and 2.3.1.5. I'll continue to try and narrow down further.

[deurk]

Can confirm I have the same behavior on docker.

[theotherp]

@deurk You need to increase the XMX value in your main config to at least 256.

[Kurisutian]

@theotherp how do I get them or where exactly are they stored? Can't access the webfrontend if this happens so I cannot pull them off there... ;)

[theotherp]

Just restart the instance (or kill it and then start again) and then create the debug infos.

[Kurisutian]

Sorry for asking again: How do I create them? What's the flag I have to use? Also when killing it, it comes up just fine but about a day or less later the web frontend cannot be opened and the init.d process is gone. However there is a java process left over which I need to kill otherwise I cannot restart using the init.d files on the QNAP. But doing so does not show anything and I cannot get some information on the cause of the crash that way or find out when it's about to act weird... What other way would there be to trigger the cause and get debug information about it?

[theotherp]

Kill it, start it, go to the system section, bugreport tab, press the button. It creates a ZIP with an anonymized log which hopefully contains some info. If not, let's see.

[Kurisutian]

OK, I did what you say but here's what happens: When I click on 'Log and settings' I can see that the system loads a little bit before a message pops up telling me this:

An error occurred: undefined: undefined

Message: undefined

I did repeat this several times but I do not get any further. However here is what I get in the console when doing this:

2019-04-03 17:22:29.159 INFO --- [0.2-5076-exec-4] o.n.debuginfos.DebugInfosProvider : [IP: 192.168.0.3] Creating debug infos 2019-04-03 17:22:29.160 INFO --- [0.2-5076-exec-4] o.n.debuginfos.DebugInfosProvider : [IP: 192.168.0.3] NZBHydra2 version: 2.3.15 2019-04-03 17:22:29.166 INFO --- [0.2-5076-exec-4] o.n.debuginfos.DebugInfosProvider : [IP: 192.168.0.3] Java command line: /share/CE_CACHEDEV1_DATA/.qpkg/NZBHydra2/lib/core-2.3.15-exec.jar --datafolder /share/CE_CACHEDEV1_DATA/.qpkg/NZBHydra2/data 2019-04-03 17:22:29.166 INFO --- [0.2-5076-exec-4] o.n.debuginfos.DebugInfosProvider : [IP: 192.168.0.3] Java runtime name: Java(TM) SE Runtime Environment 2019-04-03 17:22:29.166 INFO --- [0.2-5076-exec-4] o.n.debuginfos.DebugInfosProvider : [IP: 192.168.0.3] Java runtime version: 1.8.0_201-b09 2019-04-03 17:22:29.167 INFO --- [0.2-5076-exec-4] o.n.debuginfos.DebugInfosProvider : [IP: 192.168.0.3] OS name: Linux 2019-04-03 17:22:29.167 INFO --- [0.2-5076-exec-4] o.n.debuginfos.DebugInfosProvider : [IP: 192.168.0.3] OS architecture: amd64 2019-04-03 17:22:29.167 INFO --- [0.2-5076-exec-4] o.n.debuginfos.DebugInfosProvider : [IP: 192.168.0.3] User country: US 2019-04-03 17:22:29.167 INFO --- [0.2-5076-exec-4] o.n.debuginfos.DebugInfosProvider : [IP: 192.168.0.3] File encoding: UTF-8 2019-04-03 17:22:29.169 INFO --- [0.2-5076-exec-4] o.n.debuginfos.DebugInfosProvider : [IP: 192.168.0.3] Number of rows in table SEARCH: 41669 2019-04-03 17:22:29.170 INFO --- [0.2-5076-exec-4] o.n.debuginfos.DebugInfosProvider : [IP: 192.168.0.3] Number of rows in table SEARCHRESULT: 328734 2019-04-03 17:22:29.172 INFO --- [0.2-5076-exec-4] o.n.debuginfos.DebugInfosProvider : [IP: 192.168.0.3] Number of rows in table INDEXERSEARCH: 235753 2019-04-03 17:22:29.173 INFO --- [0.2-5076-exec-4] o.n.debuginfos.DebugInfosProvider : [IP: 192.168.0.3] Number of rows in table INDEXERAPIACCESS: 238599 2019-04-03 17:22:29.175 INFO --- [0.2-5076-exec-4] o.n.debuginfos.DebugInfosProvider : [IP: 192.168.0.3] Number of rows in table INDEXERAPIACCESS_SHORT: 1534 2019-04-03 17:22:29.176 INFO --- [0.2-5076-exec-4] o.n.debuginfos.DebugInfosProvider : [IP: 192.168.0.3] Number of rows in table INDEXERNZBDOWNLOAD: 305 2019-04-03 17:22:29.177 INFO --- [0.2-5076-exec-4] o.n.debuginfos.DebugInfosProvider : [IP: 192.168.0.3] Size of database folder: 1722MB 2019-04-03 17:22:30.084 ERROR --- [0.2-5076-exec-4] o.a.c.c.C.[.[.[/].[dispatcherServlet] : [IP: 192.168.0.3] Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Handler dispatch failed; nested exception is java.lang.OutOfMemoryError: Java heap space] with root cause

java.lang.OutOfMemoryError: Java heap space

I'm running version 2.3.15 since I cannot upgrade to a newer version right now due to another crash happening when doing so within Nzbhydra.

I hope this helps at least a little for now till I can provide more logs.

[theotherp]

That means that Hydra is unable to write the ZIP because it would be to big. Hmm... Go to the data/database folder, delete all trace files. Go to the data/logs folder and delete all gclog* files. Try to create the infos again.

[Kurisutian]

That worked, here's the file

nzbhydra-debuginfos-2019-04-03-20-24.zip

[theotherp]

@Kurisutian Increase the XMX value in the main config to 256 and restart. See what happens.

By the way, the default is 256. Did you reduce that yourself? It could also be a very old installation from when 128 was the default.

[Kurisutian]

Awesome, this seemed to have fixed it for now. At least I'm running the second day and no crash so far! :+1:

[SgtBatten]

My hdyra is crashing also regularly, sometimes more than one per day. logs indicate heap space errors and then that the db is alreayd in use which is what prevents it from restarting itself i imagine. Im going to try the xmx tweak now.

previously have had zero issues, started within the last month or so

[theotherp]

@SgtBatten Increasing the XMX value should fix this.

[SgtBatten]

Mine was set to 256 already, have gone to 512 now and just restarted it so i will let you know

[theotherp]

@SgtBatten Please post your debug infos.

[torque1]

I can confirm that the issue was introduced in version 2.3.15. Version 2.3.14 does not hang. I’m now testing 2.6.1 as I’ve been running old versions for a while so do not know if the bug is fixed in later versions.

[soupy14]

Having same concerns as well, I thought it was my web browsers but only started recently my memory is set high on hydra. Logs appear to be fine but i'll go to the tab and it shows hydra something went wrong reload the page.

I'll monitor and try to grab some logs.

[theotherp]

@soupy14 That's a different issue. I've opened #379 for that.

[torque1]

I've finished testing now and can conclude that the bug was introduced in version 2.3.15 and is still present in version 2.6.2. For now I've created a schedule task that connects to the http server every 15 minutes and restarts the windows service if it doesn't respond.

Powershell.exe try{Invoke-WebRequest -Uri http://localhost:5076 -TimeoutSec 15}catch{$restart=$True};if($Restart){Restart-Service NzbHydra2}

Please let me know if you want anything further testing.

[samkatakouzinos]

nzbhydra-debuginfos-2019-05-30-17-05.zip

Same issue, crashed page.

[SgtBatten]

Still having issues after increasing xmx values unfortunately. I am also restarting regularly to bypass this

[theotherp]

@samkatakouzinos This issue is not about the browser page crashing.

[samkatakouzinos]

Hmmm #379 then?

[theotherp]

Yes.

[samkatakouzinos]

@theotherp thanks. Posted in #379 .

[tehniemer]

I'm having this out of memory error issue as well and my XMX is set to 512. Debug logs attached nzbhydra-debuginfos-2019-07-11-08-58.zip

[tehniemer]

@theotherp what can I do about this? I've increased XMX to 1024 and am still getting out of memory errors.

[theotherp]

Anybody still have this problem?

[torque1]

I’ve just checked my logs and can confirm it’s no longer an issue for me. Last time the web server was unavailable was early March. This may have been during an upgrade or PC restart.