75% CPU usage on RPi 3

[philhough]

Hi,

Great library, and a genuine relief to be using something that spell colour like that. I'm using with an RPi 3, and along with someone else we've both found that simply using the python library pops the Pi's CPU to 75%, regardless of what the program is doing (in my case, sleeping for 10 minutes, so really, nothing).

I was using the Sense-Hat to act as a simple barometer, but as it started getting quite warm, and then spotted the CPU usage, I felt it a bad idea.

Threads here, with code examples: https://www.raspberrypi.org/forums/viewtopic.php?f=32&t=171012 https://www.raspberrypi.org/forums/viewtopic.php?f=32&t=172365

Is there something strange, fixable, odd?

Thanks

[gfichtengithub]

I a having the same issues. I am testing for temperature and if it is reaching 40, calling time.sleep(10), however CPU always stays at or above 75%.

Must be bug in Sense Hat. Any idea? Update?

[XECDesign]

What's the minimum amount required to reproduce the issue? I can only imagine this happening if you're constantly updating the screen, since the python implementation of writing to the screen is not very efficient.

Edit: Using the example posted here (https://www.raspberrypi.org/forums/viewtopic.php?f=32&t=172365#p1103355), I am not seeing the issue.

[gfichtengithub]

Well, I am trying to wait after re-draw and and executing time.sleep(20), which should bring CPU down, but it doesn't!

On Tue, Feb 28, 2017 at 6:57 AM, XECDesign notifications@github.com wrote:

What's the minimum amount required to reproduce the issue. I can only imagine this happening if you're constantly updating the screen, since the python implementation of writing to the screen is not very efficient.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/RPi-Distro/python-sense-hat/issues/64#issuecomment-283061252, or mute the thread https://github.com/notifications/unsubscribe-auth/AY366DtJ-fkbRHFViyJ1xXEuLQ5om4caks5rhDXMgaJpZM4Ls0yP .

[XECDesign]

I am not seeing it go up in the first place. Where is the 75% figure from?

[gfichtengithub]

CPU utilization. Should I paste the code I am using?

On Tue, Feb 28, 2017 at 1:50 PM, XECDesign notifications@github.com wrote:

I am not seeing it go up in the first place. Where is the 75% figure from?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/RPi-Distro/python-sense-hat/issues/64#issuecomment-283171542, or mute the thread https://github.com/notifications/unsubscribe-auth/AY366KSLqlxQD9AdzRMK7TqS5JdL_pToks5rhJa5gaJpZM4Ls0yP .

[XECDesign]

A minimal example to reproduce the behaviour would be appreciated, since the linked ones seem to behave as expected.

CPU utilisation as reported by what?

[philhough]

Hi, code below. CPU as reported by the Pi desktop CPU meter:

#!/usr/bin/python
from sense_hat import SenseHat

import time

red = (255,0,0)
green = (0,255,0)
blue = (0,0,255)
sense = SenseHat()
sense.clear()

sense.set_rotation(180)

displaypressure = int(round(sense.get_pressure()))
pressure = int(round(sense.get_pressure()*10))
sense.show_message(str(displaypressure),text_colour=blue)

while True:
        time.sleep(600)
        newpressure = int(round(sense.get_pressure()*10))
        sense.set_rotation(180)
        displaypressure = int(round(sense.get_pressure()))

        if newpressure<pressure:
                pressure=newpressure
                sense.show_message(str(displaypressure),text_colour=red)
                sense.show_letter("V",text_colour=red)
        elif newpressure>pressure:
                pressure=newpressure
                sense.show_message(str(displaypressure),text_colour=green)
                sense.set_rotation(0)
                sense.show_letter("V",text_colour=green)
        else:
                sense.show_message(str(displaypressure),text_colour=blue)
                sense.show_letter("=",text_colour=blue)

[philhough]

It should do exactly the same if you boil it down to:

#!/usr/bin/python
from sense_hat import SenseHat

import time

while True:
        time.sleep(600)

The mere import of the SenseHat code seems to keep the CPU busy,.... I originally thought that the cause was a "busy" sleep, but curring out the sensehat code, doesn't give CPU usage when doing a while sleep.

[philhough]

Maybe that's the crux of it... the cpu usage. I'm talking about the one in the top right of this image: https://goo.gl/images/AW6QnF

It's the one showing 3% on that image. When running the code above, that same display sits at 75% constantly. In addition the PI gets noticeably warm. I'd ignore that if it weren't for the CPU showing it's being mostly used, despite being in a sleep(600) state.

TIA

Phil

[XECDesign]

I was checking cpu usage as reported by top and htop from Raspbian Lite. Will check if it behaves any differently using the full image tomorrow.

Are you also using a pi 3?

[gfichtengithub]

Here is the code:

This is simply modified example for Sense Hat. I am running Pi 3 - just got it 1 week ago.

The logic is very simple:

1. Draw a bar chart consisting of current temperature, pressure and humidity. Re-draw every 0.1 seconds (heating CPU up);
2. Every 100 seconds flip the bar over (for fun)
3. If temperature reached maximum (here 30C), display "HOT!" in the SenseHat display and start "sleeping" for 20 seconds
4. Sleep unit temperature drops below 30

BUT, since CPU always stays at 75% or above, the device never cools down.

Try it for yourself.

I am running the latest version of everything - just did the "sudo apt-get update" and "sudo apt-get upgrade"

What do you think?

-------- CODE EXAMPLE -----------------

This file has been written to your home directory for convenience. It is

saved as "/home/pi/bar_graph-2017-02-28-06-29-21.py"

import numpy as np from time import sleep

from sense_emu import SenseHat

from sense_hat import SenseHat temperature_max=30

def clamp(value, min_value, max_value): """ Returns value clamped to the range min_value to max_value inclusive. """ return min(max_value, max(min_value, value))

def scale(value, from_min, from_max, to_min=0, to_max=8): """ Returns value, which is expected to be in the range from_min to from_max inclusive, scaled to the range to_min to to_max inclusive. If value is not within the expected range, the result is not guaranteed to be in the scaled range. """ from_range = from_max - from_min to_range = to_max - to_min return (((value - from_min) / from_range) * to_range) + to_min

def render_bar(screen, origin, width, height, color): """ Fills a rectangle within screen based at origin (an (x, y) tuple), width pixels wide and height pixels high. The rectangle will be filled in color. """

Calculate the coordinates of the boundaries

x1, y1 = origin
x2 = x1 + width
y2 = y1 + height
# Invert the Y-coords so we're drawing bottom up
max_y, max_x = screen.shape[:2]
y1, y2 = max_y - y2, max_y - y1
# Draw the bar
screen[y1:y2, x1:x2, :] = color

def display_readings(hat): """ Display the temperature, pressure, and humidity readings of the HAT as red, green, and blue bars on the screen respectively. """

Calculate the environment values in screen coordinates

temperature_range = (0, 40)
pressure_range = (950, 1050)
humidity_range = (0, 100)
temperature = scale(clamp(hat.temperature, *temperature_range),

temperature_range) pressure = scale(clamp(hat.pressure, pressure_range), pressure_range) humidity = scale(clamp(hat.humidity, humidity_range), *humidity_range)

Render the bars

screen = np.zeros((8, 8, 3), dtype=np.uint8)
render_bar(screen, (0, 0), 2, round(temperature), color=(255, 0, 0))
render_bar(screen, (3, 0), 2, round(pressure), color=(0, 255, 0))
render_bar(screen, (6, 0), 2, round(humidity), color=(0, 0, 255))
hat.set_pixels([pixel for row in screen for pixel in row])

hat = SenseHat() loops = 0 while True: display_readings(hat) sleep(0.1) loops+=1 if loops%100==0: hat.flip_h() sleep(1) print hat.temperature if hat.temperature>=temperature_max: hat.show_message("Hot!") sleep(5) while hat.temperature>=temperature_max: sleep(20) #Give CPU a rest. Hopefully CPU goes down while we sleep print hat.temperature print "Temperatire reached" + str(temperature_max) + ". Quitting..."

On Tue, Feb 28, 2017 at 2:41 PM, XECDesign notifications@github.com wrote:

I was checking cpu usage as reported by top and htop from Raspbian Lite. Will check if it behaves any differently using the full image tomorrow.

Are you also using a pi 3?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/RPi-Distro/python-sense-hat/issues/64#issuecomment-283184872, or mute the thread https://github.com/notifications/unsubscribe-auth/AY366A2lThIJAMlm5Rf-Wljoj3Gy4NzNks5rhKKZgaJpZM4Ls0yP .

[XECDesign]

@philhough Looks good from here: @gfichtengithub Your example hasn't been wrapped as code, so indentation has been lost.

[XECDesign]

I am not a python person, so I don't know the specifics, but you should be able to profile it and see what's actually happening in your case.

https://docs.python.org/2/library/profile.html

[philhough]

Hi, thanks for testing.

Doing the same as you yields 75% on the desktop display, and top lists python at the top taking 291% CPU.

On that basis, there's something different between our setups. It isn't the code, I'm doing the same. So back to the hardware, it's Pi3 B. Python version is listed as 2.7.9 The sense-hat python library is over a year old, so unless you've got a really old version, we're running the same.

I appreciate you're unable to reproduce the problem, but I guarantee you, it's there, and I can reproduce it over and over.

[XECDesign]

My setup: pi 3, latest raspbian image (full) upgraded using apt-get update && apt-get dist-upgrade.

@davidhoness @bennuttall Any idea what could be causing this? What runs when the library is imported? I am guessing there should be some contructor which @philhough could add hooks into to investigate what's eating up the CPU.

[philhough]

Thanks for confirming. I'm on a very recent image as new Pi, and have recently (as in past week) updated, though will try your commands above just in case!

I'm happy to try running anything to help isolate the issue, but am far from a Python expert, so it'd need to be spelled out for me. Crikey, couldn't even crop the screengrab as don't have the tools installed.

Cheers

[philhough]

Hi, OS updated as per above. Problem persists.

Thanks for assistance so far.

[dennishvo]

I can reproduce this problem every time. I started with NOOBS_lite_v2_2 image and I'm running node-red. When node-red executes my flows, python eats up all CPU. I went through the same install procedure 3 times in a row on 3 different brand new RPi 3 Model B units with new Sense HATs, using multiple versions of node-red and nodejs, and get the same result. I'm seeing python at 299% CPU in TOP and 75-76% on CPU usage monitor on desktop. As soon as I kill node-red, CPU drops to 2%.

[bennuttall]

Node red is horribly inefficient. It constantly sends messages to python, and gets python to talk to the sense hat (same for gpio).

[XECDesign]

@bennuttall, @philhough is reporting that the problem occurs as soon as the module is imported though, without nodered playing a part.

[bennuttall]

I don't know what's causing @philhough's issue but don't think @dennishvo's comment is relevant. That's just node-red.

[philhough]

Hi @bennuttall, thanks for looking into this.

I can confirm I'm not using node-red, don't even know what it is. And as @XECDesign says (and you can see in the example code we both ran), just the mere import shoots the CPU up, you don't even have to call the lib or use the sense hat.

If it helps I'd be happy to loan (and post) my pi as it is now to someone. Or as mentioned follow and run anything requested.

I'd love to make this useable and so will help in ways I can, just let me know.

Cheers

Phil

[XECDesign]

Are you UK based? Does this happen if you use a different pi with a clean image?

[philhough]

Hi, Yes, I'm UK based.

I've not tried with another Pi, so here's my plan:

Wipe current SD card and install from scratch, see if there's still an issue. If there is, wheel out Pi Mk2, clean install, and try that. Report back.

I'll not being doing this today, so if anyone though having my Pi complete with it's problem, or wanted something checking first, was a good idea.... let me know now before I wipe it, and perhaps the chance it'll no longer be a problem. (Yes, I realise that's a good thing, for me, but as above, trying to help others who may have an issue and either not noticed or not reported).

Thanks

Phil

[XECDesign]

I would recommend keeping the current sd card and trying a clean install on another. If one works and the other doesn't, it will be a matter of comparing the two to find the difference. You could even upload a .img of the card without having to mail anything.

Might be best if you email me (serge at the obvious address dot org) then we can sort out shipping if necessary.

[philhough]

Cheers @XECDesign I'll go that route and lets see what occurs. I've no problems with an upload, so good thinking there.

I'll report back shortly, thanks again to everyone!

[dennishvo]

I have an image from November 2016 in which RPi with Sense HAT running node-red uses very little CPU while running my flows (I'm collecting time-series data every 10 seconds and shipping to the cloud). I have never seen it over 25%.

[dennishvo]

I've isolated the problem in my env. From a clean install of Raspbian, I import my node-red flows and deploy. The CPU usage immediately jumps to 75% or more. One flow contains a Sense HAT input node. When I delete that node and redeploy, the CPU drops to 1-3%.

[XECDesign]

This post might be relevant. https://www.raspberrypi.org/forums/viewtopic.php?f=32&t=171012&view=unread#p1128948

[dennishvo]

Problem solved! Thank you!

[philhough]

Thanks, that worked... so a few notes for anyone else who reads this. Following the above I choose (as you're asked to) the top "auto" option in both instances. I didn't really understand the question, but seemed to work.

It removed a number of bits from the system:

The following packages will be REMOVED:

  coinor-libipopt1{u} libboost-filesystem1.55.0{u} 
  libboost-program-options1.55.0{u} libboost-regex1.55.0{u} libcwiid1{u} 
  libffi5{u} libgmime-2.6-0{u} libjs-prettify{u} libllvm3.7{u} 
  libmumps-seq-4.10.0{u} liboauth0{u} libopenblas-base{p} 
  libqt4-xmlpatterns{u} libqtwebkit4{u} libraw10{u} libruby1.9.1{u} 
  libruby1.9.1-dbg{u} libtcl8.5{u} libtcltk-ruby1.9.1{u} libtk8.5{u} 
  ri1.9.1{u} ruby1.9.1{u} ruby1.9.1-dev{u} ruby1.9.1-examples{u} 
  ruby1.9.1-full{u} ruby1.9.3{u} supercollider{u} supercollider-common{u} 
  supercollider-ide{u} supercollider-language{u} supercollider-supernova{u} 
  tcl8.5{u} tk8.5{u} wolfram-engine{a} wolframscript{u} 

Removing wolfram-engine (11.0.1+2017022002) ...
Removing coinor-libipopt1 (3.11.9-2) ...
Removing wolframscript (1.0.1-19) ...
Removing supercollider-supernova (1:3.6.6~repack-2-1) ...
Removing libboost-program-options1.55.0:armhf (1.55.0+dfsg-3) ...
Removing supercollider (1:3.6.6~repack-2-1) ...
Removing supercollider-ide (1:3.6.6~repack-2-1) ...
Removing supercollider-language (1:3.6.6~repack-2-1) ...
Removing libboost-regex1.55.0:armhf (1.55.0+dfsg-3) ...
Removing libcwiid1 (0.6.00+svn201-3.1) ...
Removing libffi5:armhf (3.0.10-3+rpi1) ...
Removing libgmime-2.6-0:armhf (2.6.20-1) ...
Removing supercollider-common (1:3.6.6~repack-2-1) ...
Removing libjs-prettify (2013.03.04+dfsg-4) ...
Removing libllvm3.7:armhf (1:3.7-4~bpo8+1) ...
Removing libmumps-seq-4.10.0 (4.10.0.dfsg-3+b2) ...
Removing liboauth0:armhf (1.0.1-1) ...
Removing libboost-filesystem1.55.0:armhf (1.55.0+dfsg-3) ...
Removing libqtwebkit4:armhf (2.3.4.dfsg-3) ...
Removing libqt4-xmlpatterns:armhf (4:4.8.6+git64-g5dc8b2b+dfsg-3+deb8u1+rpi1) ...
Removing libraw10:armhf (0.16.0-9+deb8u2) ...
Removing ruby1.9.1-full (1.9.3.484-2) ...
Removing ruby1.9.1-dev (1.9.3.484-2) ...
Removing ruby1.9.3 (1.9.3.484-2) ...
Removing libruby1.9.1-dbg (1.9.3.484-2) ...
Removing libtcltk-ruby1.9.1 (1.9.3.484-2) ...
Removing tk8.5 (8.5.17-1) ...
Removing tcl8.5 (8.5.17-1) ...
Removing libtk8.5:armhf (8.5.17-1) ...
Removing ri1.9.1 (1.9.3.484-2) ...
Removing ruby1.9.1-examples (1.9.3.484-2) ...
Removing libtcl8.5:armhf (8.5.17-1) ...
Removing ruby1.9.1 (1.9.3.484-2) ...
Removing libruby1.9.1 (1.9.3.484-2) ...

Worth bearing in mind before you go ballistic and remove half the system you used :)

Cheers to @XECDesign you're help has proved immense!

Thanks

[XECDesign]

All credit goes to raspJam on the forum and rsteca. Will see if we can fix this properly in the repo. We may have to do something with openblas. I believe Julia devs have reported the same issue and got around it by using a newer version.

[XECDesign]

Looks like the problem occurs when multi-threaded applications try to use openblas. The default image comes with libblas instead of libopenblas, which doesn't have this problem. I am curious how you ended up with openblas installed without explicitly making the decision that you want to use it instead of libblas.

If openblas must be used, it works if you set the OPENBLAS_NUM_THREADS environment variable to 1. Another option is to recompile openblas with single thread support only, but I suspect that might break other applications which are single threaded, but take advantage of the multithreading provided by openblas. If I understand the problem correctly, then they would just run slower.

[philhough]

No idea here, the OS was downloaded and installed, no options, and nothing installed bar this sensehat lib.

[XECDesign]

Ah, looks like the latest wolfram-engine has a dependency on libopenblas-base, which does indeed get included in the latest image and causes this problem for the python sense hat library, numpy and others.

[XECDesign]

Wolfram have removed the dependency on openblas from Mathematica, so it should be possible to remove it without taking out any other packages.

I've patched openblas to disable multithreading (which doesn't appear to work properly in the majority of cases) and also reduced the update-alternatives priority from 40 to 5. That means that any other installed blas provider package will take priority over it (I recommend libatlas3-base) and even if openblas is the only one installed or was manually selected by the user, it won't cause the 75% CPU usage.