Awtrix 0.96 & 0.94 on two Ulanzi TC001 reboot up to two times a day

[UCaGk]

Bug report

I use two different Awtrix versions on two Ulanzi TC0001 clocks. The clocks are under control of Home Assistant MQTT, each clock only shows time and on different events ICON, Sound and Text, but mostly time in format HH:MM:SS or HH:MM in color white, brithness 1-20, depending on some events. Every clock reboots at a purely random time, without an event, just like that. Sometimes a colored dot appears, sometimes part of the matrix turns bright white, then the firmware restarts and clock run again. Looks like the display "freeze" (sometimes a few leds flicker) and a watchdog reboots... Both clocks has accumulator removed, some changes to pcb and a 3.3V stepdown regulator. I'm not sure if the clocks reboot more than 2 times, because I'm not sitting in front of them the whole day :-)

Devices involved:

• Model: 2x Ulanzi (TC001)
• awtrix3 version: v0.94 & v0.96

Steps to reproduce the behavior:

• not possible, because this happens randomly

A clear and concise description of what you expected to happen.

• It would be nice to know if there ist problem in software, if so, a new version would be great :-)

[Blueforcer]

Check the RAM usage. If your using the Homeassistant discovery entities very often the RAM usage increases because of a memory leak in the used library. Switch to raw MQTT API commands and you're fine.

[UCaGk]

Oh, that was fast :-)

Sorry for my misunderstanding, but what do you mean with "Switch to raw MQTT API commands"

This is a code snippet: Is this raw MQTT ? If not, how looks it like ?

  - alias: Zeitformat HH:MM Hell
    service: mqtt.publish
    metadata: {}
    data:
      qos: 0
      retain: false
      topic: UhrWZ/settings
      payload: >-
        {    "TIME_COL": [255,255,255],  "WD": false,  
        "TFORMAT": "%H:%M",   "TMODE": 0, "ABRI": false,  "BRI":
        20 }
    enabled: true

[Blueforcer]

Thats correct. i mean using the awtrix entities like

[UCaGk]

I checked the free ram usage of both clocks, each has about 135000 Bytes of free RAM. HA reboot each clock in the middle of the night, so hopefully the memory should be free after that. Not sure how fast the free memory space shrinks. But if my code is correct and memory looks like much enough, what could I do to prevent random rebooting ?

[Jeandhom]

if the clocks reboot more than 2 times, because I'm not sitting in front of them the whole day

You can monitor the device's uptime.

[Blueforcer]

Not sure how fast the free memory space shrinks.

Just use Homeassistants history, you can see the ram over time in a graph 📉

[UCaGk]

Thanks for the hints, i made some entities, here are the results.

The other clock looks similar, only the battery history is a line at 100%. It`s a strange battery plot, but the uptime only restarts at ~ 4:30 h this morning. (I reboot the clock at 3 am every day) If there is a power problem the uptime counter should restart more than once in this history, right ? I think it's not a memory problem.

[Blueforcer]

RAM usage looks good, and there was no other cause of reboots in the past. I have read your issue again carefully I assume that something is wrong with the DIY power supply. When the display freezes but some leds flickers, its not a software issue.

[UCaGk]

Thanks for your help, I was also worried that it was the power supply, I was already thinking about whether I would only use the RGB LEDs with housing from the Ulanzi and replace the hardware with a simple ESP, will see.

[Trychlik]

i have same behaviour - once a day it freezes to half display working and after few seconds/minutes it reboots and works to next day ok. it crashes about at 40MB free ram. connected to 5V 2A power. battery is always 100%. maybe memory is a problem.

[Blueforcer]

40kb is too low. Check wich automation cause the memory drop

[UCaGk]

Sorry for writing again, maybe it helps, if there is a bug. After a better look onto the clock I saw that the pixel do not flicker. Often there is a dot on the uper left, the clock is frozen and about 10 s the colored AWTRIX - IP - MQTT startup comes up, after that the clock is running like before, if display was HH:MM than ist shows HH:MM, was the display HH:MM:SS before, HH:MM:SS is shown and everything look like before. Situation is like before, memory always >110kb

[Blueforcer]

You should read the whole issue. Don't control the HA entities often, instead use raw mqtt commands

[Blueforcer]

Did you find the error?

[UCaGk]

Hello Stephan, thanks for asking. The last few weeks have been a bit stressful, I haven't had time, but I'm currently checking the voltages and wanted to solder a capacitor to the 3.3V voltage converter in case of short-term current spikes. In the meantime, I also had the situation where the clock restarted and switched to AP mode, and once it didn't start automatically at all.

[UCaGk]

Update:

After adding the capacitor to both clocks, things getting more and more strange... Before adding the capacitor, the "uptime counter" only increases to about 16000 (see diagram) After adding the capacitor, the counter continues to increase without resetting at about 16000 (see diagram after 12.6./9:00) ???? What is the bit size of the uptime counter?

[UCaGk]

Update after one day running with the capacitor the clock didn't reset, the uptime value increase. But, sometimes the display switch off (fully dark) for about 1 sec, after that the display switch on again as if there was nothing, without restarting (uptime still increasing). Think this is the reason why the voltage of DC-DC stepdown regulator without cap has a voltage spike the restarts the esp inside. Now it would be nice to know why the display switch off for a second ?

[Blueforcer]

There is no software automation to switch off the display. Maybe one of your automation send the power request. But for hardware related things, this repo isnt the right place. unfortunately i can't help if someone modifies the hardware. i only do software :) I recommend you join the discrod, a few have done it there. And there's also the problem that the Ulanzi will eventually switch itself off after the mod.

[vogtmh]

I already mentioned my "RAM problem" in another issue, but it might apply here as well:

I had a similar issue, could successfully reproduce and fix it. Long story short: I didn't know about the blink or fade functionality included in Awtrix, so I used a loop in a Home Assistant automation to turn the lights on and off every second, using a custom component which defaulted to QoS of 1.

The solution was to remove the custom component and use the built-in mqtt.publish instead, defining a QoS of 0 for all commands. I also replaced the loops with single mqtt.publish commands, where I set the indicators to blink or fade.

All problems are gone since then. Thanks for this AWesome custom firmware! :)

[eku]

I observed the following yesterday.

During the display of a custom app, its value was changed as a result of an update via MQTT. The value of the display still changed, but it was no longer switched to the next app. The software was frozen and was forced to restart by the watchdog.

[UCaGk]

FYI After a few days of using the capacitor, the operating time no longer resets. What I see is that the clock display sometimes "turns off" for about 1 second. One time, the top left LED was green while the rest of the display was black. After the second, the display turns on, the green dot is gone, and the time continues as if nothing happened. The memory remains almost constant except for the usual fluctuations. Because this short "turn off" is very rare and the clock does not restart, I am fine with it.