motioneye-project / motioneye

A web frontend for the motion daemon.
GNU General Public License v3.0
3.93k stars 653 forks source link

MotionEye GUI overheats CPUs #2163

Open lacojim opened 3 years ago

lacojim commented 3 years ago

While motioneye itself doesn't seem to use much CPU, if I open the GUI with Firefox v88 on the same computer which motioneye is running, I immediately overheat and shutdown within about 60 seconds. This is an 8 core AMD CPU w/32GB RAM running Fedora 32. I have 6 cameras configured, all but one in full HD. This happens even though no cameras are recording.

Although I have had bad experiences with Ffirefox hardware acceleration, I have tried it with this with no apparent affect.

starbasessd commented 3 years ago

Firefox is up to version 90. Before trying anything else, make sure you are up to date both with FF and the base OS If you run top (htop) what is your CPU usage? What version of motion & motionEye? What install instructions were used?

lacojim commented 3 years ago

htop shows cpu's (all 8 cores) at 100% motionEye Version | 0.42.1 Motion Version | 4.3.2 OS Version | Fedora 32 I followed the Motioneye Wiki to do the install. I did have to jump thru several hoops however, as Fedora 32 no longer includes pip2 (python 2 pip), I had to build in order to install motioneye. I am under the impression that motioneye for python3 is not nearly ready for primetime.

I did switch all the ffmpeg codecs to use nvec so that helped quite a bit when recording, but makes no difference with viewing the webpage.

starbasessd commented 3 years ago

Are you running FF on the same machine as motionEye? I run an Intel i3 LenovoM73 and i5 (i5 is my production motionEye server on Ubuntu server 20.04 LTS/LenovoM73/16GB/1TB/10TBNAS) and neither have issues. I do other OSs on a Ryzen 7/1700/VirtualBox6.1 Host/32GB/10TB where I have no issues with any tested OS. I can run it on RPi4-4GB or 8GB, RPi3B+, RPi3, all with passive or active cooling, and don't have issues. My PiZeroWs can have thermal issues, but most do not. I'm not sure where your issue lies, maybe it's hardware, sustained load and lack of / improper cooling? If you connect to the GUI from another computer, without firing up FF on your AMD, do you get the same issue? If you try Chrome or another browser, do you get the same issue? I will build a VM and see what I can find. Were you able to upgrade to FF 90?

starbasessd commented 3 years ago

I forgot: Which version of Fedora 32? Desktop, Workstation, Server?

starbasessd commented 3 years ago

Also: Fedora is up to 34, are you current and up to date?

lacojim commented 3 years ago

It is the Mate Desktop spin of Fedora 32 Workstation with several add on services. F32 is EOL now, but I always stay one version behind the current release (plus or minus a few weeks). I plan on upgrading to F33 this weekend. Just been too busy to do it and couldn't afford any downtime.

lacojim commented 3 years ago

Are you running FF on the same machine as motionEye?

Yes

I run an Intel i3 LenovoM73 and i5 (i5 is my production motionEye server on Ubuntu server 20.04 LTS/LenovoM73/16GB/1TB/10TBNAS) and neither have issues. I do other OSs on a Ryzen 7/1700/VirtualBox6.1 Host/32GB/10TB where I have no issues with any tested OS. I can run it on RPi4-4GB or 8GB, RPi3B+, RPi3, all with passive or active cooling, and don't have issues. My PiZeroWs can have thermal issues, but most do not. I'm not sure where your issue lies, maybe it's hardware, sustained load and lack of / improper cooling? If you connect to the GUI from another computer, without firing up FF on your AMD, do you get the same issue?

I do not. It's only on this particular machine.

If you try Chrome or another browser, do you get the same issue?

Yes

I will build a VM and see what I can find. Were you able to upgrade to FF 90?

I can't until upgrading to F33. Coming soon.

lacojim commented 3 years ago

This computer also has a Gforce Titan Black GFX card in it, so I pretty sure that is not the problem.

starbasessd commented 3 years ago

This sounds like an issue with the particular installation, maybe something not being handled properly either in FF, or Fedora 32.

starbasessd commented 3 years ago

Please let me know after you've brought it up to more current versions of Fedora and FF, and I'll continue trying to help.

lacojim commented 3 years ago

This sounds like an issue with the particular installation, maybe something not being handled properly either in FF, or Fedora 32.

Yeah, that's kinda the way I am leaning too.

Please let me know after you've brought it up to more current versions of Fedora and FF, and I'll continue trying to help.

Will do. Perhaps later this afternoon.

lacojim commented 3 years ago

I would love to do a fresh install of F33, but I have so many self-compiled apps on here that it would take me a few months to recover from that. Just getting pip2 built was a royal PITA.

starbasessd commented 3 years ago

Understood. How do you updates to those apps? Are you running pre-compiled binaries for FF and python2?

lacojim commented 3 years ago

Understood. How do you updates to those apps? Are you running pre-compiled binaries for FF and python2?

Yes, both Firefox and python2 are distribution RPMs. For some reason Fedora decided to exclude pip from P2 and only includes the pip for P3.

Regarding updating self compiled apps, I usually use a script that does a git pull and then a make;make install, depending on what the build requires, of course. Some use cmake and one even uses waf.

starbasessd commented 3 years ago

Holy heck. 740 updates to Fedora34 even before getting dirty... I'll grab Fedora33 in a bit to do the same thing.

lacojim commented 3 years ago

Holy heck. 740 updates to Fedora34 even before getting dirty... I'll grab Fedora33 in a bit to do the same thing

Started the upgrade. 4528 packages to upgrade. Yikes! This is going to take a while, even with a 100Mb connection.

starbasessd commented 3 years ago

I have a 400Mbsec and I'm not looking forward to it..

lacojim commented 3 years ago

Okay, all upgraded to Fedora 33 w/Firefox 90.0.2. Made no difference at all. Just take a look at this screensnap of htop. This is crazy. I had to do all this very quickly too. The CPU temp was rising about 8 degrees F per second, according to GKrelliM.

Selection_181

I am going to pull the box from the rack and check the cooling components and the heat sink compound as well.

starbasessd commented 3 years ago

And if you kill firefox? What other browser is installed if any? And you said if you connect from another PCs web browser, it doesn't overload, correct?

lacojim commented 3 years ago

And if you kill firefox?

The load completely disappears.

What other browser is installed if any?

Chrome and Chromium.

And you said if you connect from another PCs web browser, it doesn't overload, correct?

Correct.

Here is the load with Chrome

Selection_182

The load average with no browser running, but with the MotionEye service running is ~1.5.

starbasessd commented 3 years ago

Looks like it's a firefox issue. It is killing your CPU. If you go to youtube or something else, is your CPU still killed?

lacojim commented 3 years ago

Looks like it's a firefox issue. It is killing your CPU. If you go to youtube or something else, is your CPU still killed?

It has some load, but nothing like what viewing MotionEye does to it. This is the results watching a full HD 1080P video from YouTube.

Selection_183

starbasessd commented 3 years ago

So, what happens if you turn off Hardware acceleration?

lacojim commented 3 years ago

I just tried that already. Seems to make things a tad bit worse. I think it's time to check the cooling components in this box. I am going to hit it with a compressed air hose while I have it opened up.

starbasessd commented 3 years ago

Well, I think motionEye by itself isn't the problem, when you connect from another PC it doesn't overheat and crash. It may be the combination on the local machine's FF, but I don't have a clue where to begin on that, how you're set up that is triggering that. I'll leave it open in case someone else has thoughts on where to look...

lacojim commented 3 years ago

Okay, de-racked it, and popped off the lid. Wow was it dusty in there! Hit it with the air hose until it was shiny and new looking again. Took off the CPU heat sink and removed the old heat compound and replaced with new silver stuff. The fan on the rear of the case was rotating about half speed so I took it apart, cleaned and re-oiled it. It's now rebooted and running very quietly. Running FF with MotionEye does still top out the CPU, but it no longer overheats and shuts down. It gets up to about 150F and holds there. So one problem fixed. Now just need to find the root of problem number 2.

lacojim commented 3 years ago

Is it possible to limit FF to how many CPU cores is uses? Definitely not a true fix, but could be a reasonable work-around.

starbasessd commented 3 years ago

Check Wesley Branton answer for https://support.mozilla.org/en-US/questions/1247528 There seems to be some other answers, too all over google.

lacojim commented 3 years ago

Check Wesley Branton answer for https://support.mozilla.org/en-US/questions/1247528 There seems to be some other answers, too all over google.

Good find. I made this change. It still appears to use all 8 cores, but not pounding them nearly as much. I will test for several hours and report my findings. Thanks! Edit: I changed it from 8 to 4.

starbasessd commented 3 years ago

BTW, when I have my production server (i5/16GB/1TB/10TBNAS) in my forefront on my Windows machine (3 tabs open) in Firefox my CPU usage jumps to 35-40%, but if any other tab is front and center, its 5-10%. Google Chrome on Win10 is 50-60% or thereabouts FF not running Edge Chrome about the same. IE11 60-70% Hmmmm Something about the video processing... I have 6 feeds in production. But FF has the lowest CPU usage at least in Win10....

lacojim commented 3 years ago

I have been having another weird issue for a few months now, and I am wondering if they are related. Whenever I am running Kodi on one monitor and firefox on a different monitor, it causes an Xorg freeze. I mean it totally locks up. However, the OS is not crashed. I can still ssh into this computer from my phone via Turmux and #reboot. When it does reboot, the monitor that was running Kodi gives me a quad-split image of the last video that Kodi displayed. I have so far not experienced this since upgrading to F33, but I suspect it will happen again at some point. It could be a framebuffer error in my GFX card or even something in Xorg itself. Nothing ever gets logged.

I have been running the MotionEye GUI now for several hours in FF, CPU temp is at 137F, and watching live TV in Kodi. So far, so good. Fingers crossed.