Buster raspimjpeg causes the pi to run at 80 deg C - not stretch

[ksdehoff]

Hello:

I've used this code successfully for at least 4 years. I have 4 cameras all on stretch builds. I upgraded one to buster (clean build) and had failures of the pi due to high heat - over 80 degrees C. The other units I have run 65-70. I fell back to the stretch build and now it is running at 66 degrees C instead of 80. There's got to be something complicated going on so I thought I'd let you know. For now I wont be using Buster!

[PCAssistSoftware]

Must perhaps be something specific to your setup as I have it running on Buster and it is on 24/7 in a hedgehog box outside through summer and winter and never had any issues with pi failing due to temperatures, or any overheating pi even in last few weeks when temperatures have been really high (well high for UK anyway)

[ksdehoff]

Can you do a vcgencmd measure_temp and let me know what it says?

From: wingers999 Sent: Sunday, August 23, 2020 12:52 PM To: silvanmelchior/RPi_Cam_Web_Interface Cc: ksdehoff; Author Subject: Re: [silvanmelchior/RPi_Cam_Web_Interface] Buster raspimjpeg causesthe pi to run at 80 deg C - not stretch (#578)

Must perhaps be something specific to your setup as I have it running on Buster and it is on 24/7 in a hedgehog box outside through summer and winter and never had any issues with pi failing due to temperatures, or any overheating pi even in last few weeks when temperatures have been really high (well high for UK anyway) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[PCAssistSoftware]

[roberttidey]

My test on a Buster Pi 3 in a case showed temp of 52C in an environment of 28C

Try running a top cmd to see what is using cpu. Mine shows raspimjpeg (main process) running at 1%. Even on a Pi Zero it is typically less than 5%. Much of the processing is happening in the GPU.

A Pi should not fail if it gets hot; it dials down the cpu frequency.

[BETLOG]

Mine periodically overheats too, for no apparent reason. Sometimes I'll look at it (mine display temperature on the cam overlay) and it'll be 73C+.

edit: right now. The sun has just set and ambient is noticeably cooler, and yet the cam keeps getting hotter. It's now 70.2C so i closed the browser window. Nothing else should be causing it to use wifi now.

for enclosure/mounting/heatsinking reference:

PizeroW: LEDs and HDMI disabled, using the integrated ribbon and camera module pizerocam, not the typical fullsize ribbon pi one.

I don't typically leave the web interface open, but I have never seen it lock up when I have. Rather it does it when I'm not looking.

After it seemed to die while it was relatively cool (<20C) last night, I decided to try stretch... but that also has issues now it seems.

Edit: i wonder if its additional load being generated by sunrise events that cause this. Being the west side of my property the camera does not see the sun, but no doubt there is a lot of light intensity and whitebalance shift going on at the times it seems to happen.....not to mention sudden ambient temperature changes, so maybe looking at temps is misleading. The big gaps in graph at the start of the weekly graph are just me powering down..iirc. And unfortunately have not been graphing this camera for as long as iv'e been using it (also iirc).

[BETLOG]

Should the wifi be broadcasting continuously even when nothing is connected to it's web server/stream port?

[roberttidey]

Not sure what is doing that. I don't see that behaviour myself. It seems to start pretty quickly after boot maybe before the camera system has got going. One could try disabling the autostart of the camera software to be sure.

[BETLOG]

One could try disabling the autostart of the camera software to be sure.

Please specify how you would do that.

The labels are approximate relationships, I didnt time how long it took after pluging it in till the start of the wifi traffic, but it probably corresponds with the camera/motion kicking in rather than just the OS bringing the network up.

Temperature is currently 74.5C only a short time after starting it today. Disabling motion detection and camera, then checking temp via ssh a few minutes later shows 55C

Today I'll try disabling the few non-buster-stock things I have installed:

munin-node

HDMI disabled: /usr/bin/tvservice -o

zero LED disabled: dtparam=act_led_trigger=none dtparam=act_led_activelow=on

camera LED disabled: disable_camera_led=1

Please excuse the image spam, given the spikes in thermals it seems this issue is still the appropriate one to be discussing this.

On uptime you can see the most recent boot, about 11pm last night, and the subsequent activity between then and my checking it at ~10am

Something seems to be happening just as it starts getting light each day, when ambient temperature is ~20C.

[BETLOG]

• It seems apparent that the constant wifi broadcast is the thermal source

• And you are almost certainly correct to say the OS version is moot.

ie:

new tfcard, fresh stretch install (previously using buster): at left: ./install completing and before selecting start at right: after starting camera, motiondetect off

So: again with the constant wifi wailing as soon as RPi_Cam_Web_Interface is active..

motion detection is off

wifi volume persists after: browser - 'stop camera' (No browsers are pointed at the stream until or after doing this) and ssh pizerocam1 "~/RPi_Cam_Web_Interface/stop.sh"

~20 minutes after starting RPiCam: temperature is just under 70C

ssh pizerocam "sudo service apache2 stop" as expected: immediately terminates the ~20Mbit/s wifi wailing, and temperature quickly drops to 58C

NOTE: by 'reinstalled' i mean:

lsblk
sudo ddrescue -D --force /home/user/downloads/ISOs/pizero/2019-04-08-raspbian-stretch-lite.img /dev/sdf
#     COPY wpa_supplicant TO SDCARD 
touch /media/user/boot/ssh
ssh-keygen -f "/home/user/.ssh/known_hosts" -R "192.168.1.61" # pizerocam1
#     REMOVE SDCARD AND BOOT PI
# wait 3 minutes for initial setup
# actually takes ~1min to appear in routers wifi 'associated stations'
# ...and a minute after that to accept ssh

ssh pizerocam1
passwd
sudo raspi-config
#         locale
#         timezone
#         wifi country - AU
#         camera
#         update raspi-config
exit # ssh pizerocam1

scp /home/user/.ssh/id_rsa.pub pizerocam1:/tmp
ssh pizerocam1 "ssh-keygen -C $(whoami)@$(hostname)-$(date +%Y-%m-%d--%H-%M-%S); cat /tmp/id_rsa.pub >>/home/pi/.ssh/authorized_keys"

ssh pizerocam1
sudo apt update & sudo apt-get dist-upgrade -y && sudo reboot

ssh pizerocam1
sudo apt install -y git
git clone https://github.com/silvanmelchior/RPi_Cam_Web_Interface.git
cd ~/RPi_Cam_Web_Interface
# RUNNING STRETCH OR EARLIER!
cd bin && mv raspimjpeg raspimjpeg-buster && mv raspimjpeg-stretch raspimjpeg && cd ..
./install.sh
# set the compatible php version in the initial install.sh dialog (7.0

[BETLOG]

back to the previous buster tfcard with motiondetect off it drops ~5C, and with it on eventually this happens: At which point it locks up, or i shut it down

I don't get it; i had it running for several days and now it just dies almost immediately. ..unless this thermal issue is progressively killing this zero. I guess I need to try the same tfcard on another (probably undamaged) zero with the more standard pi camera on a pcb attached.

[BETLOG]

Interesting: A different pizeroW with the same buster tfcard and the 'normal' pcb camera works fine. <55C The same pizeroW and the afore-mentioned pcb cam also works fine. <55C ie: this conversion-ribbon and this pcb cam

All of my grief seems to happen when I use this cam [edit]: nope, it happens with any cam

I don't have a simple way to mount the PBC cam in the same place i had the other one mounted, so i'm not sure how to test that right now. But I get the feeling the location (and its marginally different airflow/ambient temps) wont make any difference. [edit]: yep, do, and did, and it still locks up somewhere after 70C I bet the integrated cam/cable has some badly calculated resistance somewhere. [edit}: nope, probably not

@ksdehoff : please confirm what cam you are using.

[roberttidey]

Interesting. I have never used one of those cameras with electronics in the cable.

If it is the cpu core temperature you are monitoring then it is unlikely to be anything as simple as a resistor in the cable. More likely that it is something in the interface that is causing excess cpu or gpu usage.

If you run top from a command line then you can see the real time usage of the cpu. The raspimjpeg process should be < 10% even on a pi zero as most of the processing is in the GPU.

To disable autostart of the raspimjpeg support on boot then either select no for the autostart option when installing or edit /etc/rc.local and remove the lines between #START RASPIMJPEG SECTION and #END RASPIMJPEG SECTION

[BETLOG]

Thanks. Specifying how to do things makes it much easier to give meaningful comment on the results.

I'm kind of fatigued right now, but I'll continue prodding this issue as I play with things, and report as i observe. I currently have all of my zeros set up with cameras, pizerocam1 is the one with the half-case and short camera/cable. The other 3 are the usual revision v1.3 pcb cams. I'll see if any choke overnight. Munin should be able to shed some graph-light on how they run.

PS: motioneye makes generating a multi stream screen really simple.... even if it otherwise sucks for framerate, motioncapture, and all of the other things RPIcam does so beautifully.

[BETLOG]

The timing might be coincidence... I don't recall if other crashed have been this precise, and it's hard to say by looking at the weekly graph, given that many of these gradients are manual start/stops.

router system log

Sat Jan  2 07:06:24 2021 daemon.notice hostapd: wlan0: AP-STA-DISCONNECTED b8:27:eb:aa:d3:10
Sat Jan  2 07:06:24 2021 daemon.info hostapd: wlan0: STA b8:27:eb:aa:d3:10 IEEE 802.11: disassociated due to inactivity
Sat Jan  2 07:06:24 2021 daemon.info logread[1550]: failed to send log data to 192.168.1.60:514 via udp
Sat Jan  2 07:06:25 2021 daemon.info logread[1550]: Logread connected to 192.168.1.60:514
Sat Jan  2 07:06:25 2021 daemon.info hostapd: wlan0: STA b8:27:eb:aa:d3:10 IEEE 802.11: deauthenticated due to inactivity (timer DEAUTH/REMOVE)

[BETLOG]

Yeah, it's thermal. While figuring out python enough to include a UPS power meter in the cam text overlay it has become obvious that if ambient is ~30C, and motion is off; the temps are up around 65C. If motion detection is turned on the temps immediately climb to 70+ and the pi dies. I have 4 of them, and can reproduce it with them all. It's even more tedious because the one spot I need to put a camera in seems to be the worst. Presumably the wifi signal is a little lower there, or the airflow a little worse than the table i was running them all on above (where they still died anyway).

[roberttidey]

Raspberry Pi 3b Ambient temp 23C Buster Standard pi camera

Running RPIWebCam from a remote browser showing live preview

Internal Motion detection off core temp 49.4C raspimjpeg cpu 1.5% apache 1.7% Internal Motion detection on core temp 50.5C raspimjpeg cpu 1.8% apache 1.7%

I never use external motion detection; that will use more cpu but I didn't think it was enough to cause a significant extra rise in temps.

[BETLOG]

Should the cam-pi be constantly broadcasting wifi when nothing is connected to it's apache server? I mentioned that before but never got an answer. I see why it might be, but it would account for a lot of additional power absorption too. So I'm wondering if it's meant to be doing that.

I can document the setup I have sitting on a stone tile on a table quite easily, it is stable and basically just looking at the ceiling. So I will do that below. The cam i want to put in the window as an actual working security camera, and it has trees almost always moving in the masked out region, and frequently shifting lighting conditions is when it fails a few minutes after boot, so documenting that is a little more tricky/complicated. But I'll try doing that in another post soon. And no, it doesn't matter which of the 4 cams i put in the working position, they all fail. Strangely enough they now fail a lot more readily than they did when i had one up there a week or two ago; when it would typically run until beginning of dawn light the next day.

Typical 'normal' and functional RPCWI on a pizeroW stats:

Raspberry pi Zero W ambient: 30C buster standard rev 1.3 cam pcb

Running RPIWebCam from a remote browser showing live preview

Motion detect mode: internal (have never used anything else) 1GHz 64C raspijpeg ~8% apache2 ~10%

[BETLOG]

Unsure how useful this is: If I let it run while motion is active and that kworker (wifi related?) is thrashing, it will lock up the machine a short time later.

https://user-images.githubusercontent.com/12265451/103493061-e7640300-4e7a-11eb-91b6-1eef8a5dfbc9.mp4

[BETLOG]

if i have a terminal open when it dies, it usually looks like this

root@pizerocam2:/home/pi# 
Message from syslogd@pizerocam2 at Jan  4 14:51:41 ...
 kernel:[ 4106.456055] Internal error: Oops: 805 [#1] ARM

Message from syslogd@pizerocam2 at Jan  4 14:51:41 ...
 kernel:[ 4106.611866] Process processes (pid: 9716, stack limit = 0x73e60b59)

Message from syslogd@pizerocam2 at Jan  4 14:51:41 ...
 kernel:[ 4106.620579] Stack: (0xcb86be30 to 0xcb86c000)

Message from syslogd@pizerocam2 at Jan  4 14:51:41 ...
 kernel:[ 4106.627953] be20:                                     00000000 00000001 00000406 00000402

Message from syslogd@pizerocam2 at Jan  4 14:51:41 ...
 kernel:[ 4106.640509] be40: cb86beac cb86be50 00000000 a39c8401 c0194b50 cb86bebc c0a97028 cb86d5a0

Message from syslogd@pizerocam2 at Jan  4 14:51:41 ...
 kernel:[ 4106.652976] be60: 00000055 00000040 ffffe000 ca1fc73c cb86beac cb86be80 c018a140 c0187750

Message from syslogd@pizerocam2 at Jan  4 14:51:41 ...
 kernel:[ 4106.666339] be80: c0194b50 c001bf18 00000002 a39c8401 cb86d5a0 00d1e000 c0a97028 00000055

Message from syslogd@pizerocam2 at Jan  4 14:51:41 ...
 kernel:[ 4106.679790] bea0: cb86bf24 cb86beb0 c018b630 c018a0a8 ca1fc700 c0019f8c cb86befc cb86d5a0

Message from syslogd@pizerocam2 at Jan  4 14:51:41 ...
 kernel:[ 4106.692661] bec0: 00000055 00000cc0 00000d1e 00d1e000 cbb38030 cbb38030 089c43cf 00000000

Message from syslogd@pizerocam2 at Jan  4 14:51:41 ...
 kernel:[ 4106.705616] bee0: 00000000 d7da7f90 cba4a478 ca1fc73c 00000000 a39c8401 cb86bf24 cb86bfb0

Message from syslogd@pizerocam2 at Jan  4 14:51:41 ...
 kernel:[ 4106.718697] bf00: cba00e40 ca1fc700 00d1ef14 0000081f cb86d5a0 ca1fc73c cb86bf74 cb86bf28

Message from syslogd@pizerocam2 at Jan  4 14:51:41 ...
 kernel:[ 4106.732919] bf20: c07a832c c018b0d8 cb86bf54 cb86bf38 c0014714 c00134d4 00000000 c0a97028

Message from syslogd@pizerocam2 at Jan  4 14:51:41 ...
 kernel:[ 4106.747325] bf40: 00000800 00000055 cb86bf8c c0a9c228 0000081f c07a81e8 00d1ef14 cb86bfb0

Message from syslogd@pizerocam2 at Jan  4 14:51:41 ...
 kernel:[ 4106.760882] bf60: 00000001 000361c4 cb86bfac cb86bf78 c0019f8c c07a81f4 c0009cfc 00000001

Message from syslogd@pizerocam2 at Jan  4 14:51:41 ...
 kernel:[ 4106.775348] bf80: cb86bfac cb86bf90 c00eac74 b6ec5248 80000010 ffffffff 00c5387d 00c5387d

Message from syslogd@pizerocam2 at Jan  4 14:51:41 ...
 kernel:[ 4106.789235] bfa0: 00000000 cb86bfb0 c0009d04 c0019f50 b6fe0f50 00000000 00000000 00d15008

Message from syslogd@pizerocam2 at Jan  4 14:51:41 ...
 kernel:[ 4106.803302] bfc0: 000001f7 b6fe0f78 b6f9c000 000001fc 00d15100 00000001 000361c4 00000006

Message from syslogd@pizerocam2 at Jan  4 14:51:41 ...
 kernel:[ 4106.817137] bfe0: 00d1ef10 be9a4be8 b6ec5128 b6ec5248 80000010 ffffffff 00000000 00000000

Message from syslogd@pizerocam2 at Jan  4 14:51:41 ...
 kernel:[ 4107.020215] Code: eb006236 e2506000 0a000097 e3550000 (e594c000) 

[roberttidey]

raspicam does not directly cause any network traffic, neither does apache if nothing is connected.

This is my wifi traffic when I open a browser connected to camera preview page for a while and then close it.

raspimjpeg cpu usage on a zero looks OK. Does top shown any other high usage processes.

Crash is a kernel crash which is unlikely to be directly raspimjpeg related.

For the power / temperature issue can you measure the current being drawn from the power supply. On a zero I see about 320mA with camera active (camera draws about 180mA of this).

I would also do a clean install of buster with no extra software just the RPIWebCam as a baseline to check out power and stability from there.

[BETLOG]

sigh

two things:

1) a fan will help keep temperatures low enough to prevent thermal lockup, so if you do need to be streaming continuously on a pi zeroW; like if you are using motion externally on another machine, a fan will probably be required. Heatsinks by themselves do not cut it.

2) motioneye - if you have ever set it up on another machine in order to view multiple streams, or do external motion detection - DISABLE/REMOVE IT. I had it set up to view all my streams. And had everything in motioneye, including motion detect disabled. ....but apparently that doesn't stop it from making persistent network sockets to all of the machines you told it about, even across reboots and even when it's interface isnt running.

netstat -46pan FTW

So yeah, apparently that solves the issues I was seeing with thermal lockups every 5 minutes, and constant wifi broadcasting. facepalm

Incidentally: power usage for a pi zeroW with (only) motioneye pulling one stream ~0.7A with (only) a browser pulling one stream ~0.5A with nothing pulling any streams ~0.4A Note: mains powered by a 5V 3A 'charger' across a ~1.5m USB cable, UPS Lite r1.2 attached and fully charged, and a 5V 0.26A fan attached to the GPIO pins. ie: if you really do need to thrash a pizero by maintaining a constantly open wifi port, this is your power/cooling baseline.