RIOT is saw-toothing in energy consumption (even when idling)

[miri64]

I after I encountered the same problems mentioned by @PeterKietzmann in this thread when conducting the experiments for my thesis I did some experiments independent from that. I post this as a RIOT issue since according to Peter, the IoT-LAB tutorials do not produce this kind of output so this might be a RIOT problem.

I used the energy consumption monitoring feature of the IoT-LAB to measure with a period of 140 μs and an average of 4.

I used to setups:

First, I setup an application that went into main and left it immediately for idling:

int main(void) {
    return 0;
}

I did not add any other modules to the binary and disabled auto_init (so no xtimer or anything is running in the background).

The energy consumption for this looks as follows:

Also with an disabled auto_init I had another application running which remained in main()

int main(void) {
    while(1);
    return 0;
}

The spurious peaks are a known IoT-LAB issue: https://github.com/iot-lab/iot-lab/issues/148, the saw-toothing however is not. The frequency of the tooths isn't constant and differs from experiment to experiment. They can be very long or very short (as seen above with my two experiments), but up until now they always were more likely in a resolution of seconds, not milli- or microseconds.

[jnohlgard]

I have seen this same sawtooth in RIOT on Mulle with local measurements as well so it's also not limited to the iotlab-m3 boards. I get a more or less horizontal line idling in Contiki.

[emmanuelsearch]

@gebart a stupid question: you did use the same measurement tools/techniques for both RIOT and Contiki on mulle, right?

[jnohlgard]

yes, even the same board and connections, I only reflashed a new image on the Mulle between the tests. But this was probably 6 months ago, and I don't know where I put the data sets from the measurement.

[miri64]

For reference: here is the application I used to generate above plots. The idling state can be achieved by setting the environment variable IDLE to a non-zero value.

[miri64]

And here is the link ^^: https://github.com/authmillenon/RIOT_playground/tree/master/empty

[LudwigKnuepfer]

I have seen this same sawtooth in RIOT on Mulle with local measurements as well so it's also not limited to the iotlab-m3 boards. I get a more or less horizontal line idling in Contiki.

@gebart May I ask which value this line resides at?

[jnohlgard]

@LudwigKnuepfer depends on low power mode settings in the Contiki port, but anything from 40 uA to a few mA (radio listening and cpu awake idle).

[jnohlgard]

@LudwigKnuepfer I just had an idea, could it be transceiver related? Is there any analog filter on the measured signal in the IoT-lab?

[miri64]

Not to say this should not be investigated, but are you sure? In my experiments I did not initialize anything, so why should the transceiver pull power?

[miri64]

(though this might be the easily testable with a transceiver-less board as an arduino-due e.g.)

[malosek]

Hello,

Have made quick test using MSP430F2618 loaded in ts430pm64 and checked via energy trace feature of msp-fet in ccs.

1HMz internal DCO

8MHz internal DCO

can not see any sawtooth wave there, have just while (1); in the main wo autoinit. why do you think is riot related? since while(1); is just something like 0x1c48: 3FFF JMP (0x1c48) so wo xtimer there is no scheduler involved and CPU runs always at full speed/power.

interesting would be to know if the sawtooth freq is dependent from MPU freq of if changing int/ext oscillator makes some difference.

PS: for me it smells like some charge is build somewhere (on some floating cap) and when reaching critical value - is discharged. Or maybe some un-initialized peripheral? So I would rather look around the MCU then inside.

wbr malo

[miri64]

why do you think is riot related?

Because Contiki on the same hardware (Cortex-M in @gebart and my case) does not show this kind of behavior. Even if it is some dangling peripherals that is a problem that should be fixed IMHO.

[malosek]

Hello,

so you have exactly the same port settings "while" looping in the "while" compared to contiki and with riot you are getting sawtooth in power consumption?

wbr malo

[miri64]

I'm just the messenger. @PeterKietzmann and @gebart had ;-)

[jnohlgard]

The measurements that I did before were made with the RIOT scheduler alive, but only idling, which is more useful for my purposes than measuring an empty while(1); loop. The same was done for Contiki, the application is idling waiting for an event (an etimer with a 60 second timeout was queued). Foremost I am interested in getting a longer device life time on battery, so I want the application to be idle but still responsive.

[jnohlgard]

It could be some pin floating and building up a charge which makes it leech less and less until floats low enough to trigger some piece inside a device which would let it discharge again, like @malosek suggested, but this is pure speculation. I also believe we should look outside the MCU for starters, turn off peripherals and make sure nothing is floating. The common denominator between mulle and iotlab-m3 is that they both have an at86rf2xx radio so I'd begin checking that. I don't have any time right now to do these measurements though.

[malosek]

Hello,

yes is pure speculation, since I did not even see the schematics, but the period of tooth saw is quite long which makes me quite curious what it can be. are you 100% sure with your measurement setup? Anyway do not have the measurement setup with enough bw and resolution to measure MCU in some low power state and waking up just on timer overflow for few cycles and go to low power again. But have made measurement with toggling led each 500ms - no sign of tooth saw (8MHz int DCO)

Will try to get some reasonable setup to measure wo the led. wbr malo

[miri64]

@MichelRottleuthner did you observer this behavior?

[MichelRottleuthner]

@miri64 not yet, but I also haven't done any measurements specific to this issue yet ;)

[miri64]

Do you have access to an IoT-LAB Opennode M3 or Mulle (the platforms the issue is known on)?

[roberthartung]

Using my PotatoScope with the most recent pm changes, cpu goes to idle and the radio being off I see the following for the inga_blue board:

I also see some periodic events (~66ms), but no saw toothing. So I guess it's board specific not RIOT specific?

[miri64]

I also see some periodic events (~66ms), but no saw toothing. So I guess it's board specific not RIOT specific?

But then why doesn't Contiki do the same?

[jnohlgard]

I could do some measurements on mulle on Saturday if I can borrow some equipment from a colleague before then. I used to see this pattern before but I haven't checked in a while.

[miri64]

:+1: maybe I find some time to redo the tests on IoT-LAB.

[kYc0o]

AFAIR in my last tests I concluded that it was produced by the noisy environment on the IoT-Lab sites (lots of neighbor packet's dropped/refused). I'm skeptical about Contiki but I'll try to test soon similar setups with both OS.

[MichelRottleuthner]

@miri64 I have access to iotlab-m3 boards, but for measuring in a sensible way it would be nice to have one of those "iot-lab connectors" to interface with the board without falling back to measure over USB or soldering probes directly to the board. Anyone here has one of those lying arround?

@roberthartung: just to be sure, you also verified that there is no sawtooth on a wider timescale? I'm asking because the scale of your screenshot is way smaller compared to those initially provided by @miri64

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want me to ignore this issue, please mark it with the "State: don't stale" label. Thank you for your contributions.

[miri64]

Was there any work done on this the past few years @MichelRottleuthner @roberthartung?

[roberthartung]

I can grab a STM nucleo board (hopefully tomorrow) and verify the latest power consumption behavior.

[roberthartung]

@miri64 Got the nucleo-l476rg here with me. I am measuring with a Keithley DMM7510. I just downloaded the CSV and plotted with gnuplot for now; I can't see saw-toothing here. My code is the same as what you did (main that does to idle).

Let me double check on a larger timescale right now.

[roberthartung]

Over a timeframe of 100seconds, I don't see any saw toothing. 4.63mA.

[PeterKietzmann]

Do you have a nucleo-f103rb available? Otherwise, the issue is quite old and someone should check on iotlab if we still face that problem.

[roberthartung]

@PeterKietzmann Unfortunately not. Let's wait until someone verifies that this problem actually still exists.

[miri64]

I'll give it a look on IoT-LAB.

[PeterKietzmann]

I'm already at it. Seems to be still there. Will send plots in a moment

[miri64]

Thanks @PeterKietzmann! Might also be a good idea to check the results with a proper Osci, if you have the hardware for it.

[PeterKietzmann]

With examples/hello-world on ad28752e4e09e73bdd86d9fde5de087e29c6ec99, FIT IoT-LAB grenoble node m3-101, power profiler settings with period of 140us and averae 4 samples.

[PeterKietzmann]

Theoretically I should have the hardware available. Don't know if I can make it today, though.

One other thing: There used to be s powering option for M3 nodes either from a battery or USB. I cannot find that option anymore, did they get rid of that?

[roberthartung]

This looks like a strange effect from some other hardware, not RIOT itself.

[miri64]

One other thing: There used to be s powering option for M3 nodes either from a battery or USB. I cannot find that option anymore, did they get rid of that?

I don't remember that being a thing. Maybe @aabadie can help to answer that?

[miri64]

This looks like a strange effect from some other hardware, not RIOT itself.

There is reason to believe it might be a RIOT issue:

I have seen this same sawtooth in RIOT on Mulle with local measurements as well so it's also not limited to the iotlab-m3 boards. I get a more or less horizontal line idling in Contiki.

[PeterKietzmann]

I agree that it looks like it, however, @gebart faced similar issues locally with a Mulle board (see comment) with RIOT but not contiki and I could not reproduce the issue with a different RIOT image (see mailing list). @kYc0o pointed out, that it might relate to a noisy environment (see comment). Although the period is too deterministic for noise, it might relate to some transceiver current drain due to inappropriate "uninitialization"? Needs further investigation that I cannot provide right now. But anyone with iot-lab access should be able to.

[roberthartung]

@miri64 yes, true. I've read the comment as well. My problem here is that components of the MCU and sensors will probably have a very fixed set of modes and therefore saw-toothing is quite hard. Saw-Toothing often comes from some analogue hardware or other stuff that happends over time. I just can't imagine of some piece of software that is responsible for this kind of smooth saw-tooth pattern.

[miri64]

I just can't imagine of some piece of software that is responsible for this kind of smooth saw-tooth pattern.

No, that wasn't what I was saying either. Rather I agree with @PeterKietzmann that it might be some "uninitialization" issue. Some analogue pin or antenna that is hanging open in RIOT but not in Contiki, for some reason.

[roberthartung]

I just can't imagine of some piece of software that is responsible for this kind of smooth saw-tooth pattern.

No, that wasn't what I was saying either. Rather I agree with @PeterKietzmann that it might be some "uninitialization" issue. Some analogue pin or antenna that is hanging open in RIOT but not in Contiki, for some reason.

Yes, absolutely true. Totally looks like this kind of issue.

[roberthartung]

I just can't imagine of some piece of software that is responsible for this kind of smooth saw-tooth pattern.

No, that wasn't what I was saying either. Rather I agree with @PeterKietzmann that it might be some "uninitialization" issue. Some analogue pin or antenna that is hanging open in RIOT but not in Contiki, for some reason.

Yes, absolutely true. Totally looks like this kind of issue.

There is a battery connector on the board. Maybe the battery is responsible for some side effects.

[PeterKietzmann]

Looking at this schematic (sect. 3.2) and this schematic (sect. 3.11) it is not clear to me how to set up the same power supply conditions as on the testbed, and where to put my probes to measure the same point as the INA226 of the control node. @aabadie can you help out?

[aabadie]

I cannot find that option anymore, did they get rid of that?

Battery options were removed 5 years ago because the batteries had physical issues (and could explode).

It would help is someone could flash an hello_world application from Contiki or OpenLAB (FreeRTOS) and see if there's a difference. This way we could have a better idea where the problem comes from (RIOT or IoT-LAB).

[PeterKietzmann]

@aabadie thanks for the information, I didn't know about the exploding batteries. As reported in this comment there are a couple of indications pointing to RIOT. However, I'd like to reproduce the measurement locally with a multimeter. Can you tell how to set up the same power supply conditions as on the testbed and where to attach my probes to measure the same point as the INA226 of the control node?

[aabadie]

Can you tell how to set up the same power supply conditions as on the testbed and where to attach my probes to measure the same point as the INA226 of the control node?

All pins are router to the gateway via the specific connector which beside the usb port. I think it's quite difficult to directly access the hardware this way (unless you have a compatible connector). I think the FTDI chip is not taken into account when measuring power with the INA226.