Black screen after a day and a half

[billgod]

I have looked around and found many black screen issues but non of them seem to match what I have. My screenly would run for about a day and half sometimes 2 days then black screen. I just downloaded the latest 2 days ago, new sd card, different pi 3 and a different power supply. After about a day and a half screen went black again. The screen is on and back lit. just black. Has anyone else suffered this? I find it odd that 2 version, 2 pies same problem?

I am only displaying an image and 2 urls to google calendars.

ALSO.. PLEASE add a reboot option to the web interface!

When filing a bug, please include the following headings if possible. Any example text in this template can be deleted.

Overview of the Issue

A paragraph or two about the issue you're experiencing.

Reproduction Steps

Steps to reproduce this issue, eg:

• Press Add Asset
• Enter the Asset URL ...
• Press "On"
• Wait for the asset to show up on the screen

Environment

• Raspberry Pi Hardware Version:
• Screenly OSE Version:

[Gifford47]

i am at the development branch. all is working fine after a restart of screenly-viewer.service but without i get a black screen... here is the log: memory.txt screenly_log_25.11.19.txt pmap.txt

[rusko124]

@Gifford47 Interestingly, most of the memory is occupied by docker and uzbl, which looks contradictory. Since one must exclude the other. I assume we are dealing with 2 viewers at once. Cause of this is switching between the master and the experimental branch.

Also I see that the Screenly ansible-playbook doesn't have a cleanup of Docker when choose the development branch after experimental.

Not sure we need to create PR to fix that so the Screenly will be migrate to experimental browser as default. Could you one more install the last Screenly image to SD and upgrade to the development branch without touching the experimental branch, please?

[Gifford47]

Sure! I'll try and post the results this afternoon.

[Gifford47]

thanks a lot for the answer, now it works for me too! but screenly reserves about 230mb in total. is that normal? from time to time it also drops 150mb (e.g. when restarting screenly). is that normal and can I still optimize it?

memory.txt

[ealmonte32]

thanks a lot for the answer, now it works for me too! but screenly reserves about 230mb in total. is that normal? from time to time it also drops 150mb (e.g. when restarting screenly). is that normal and can I still optimize it?

Remember the viewer is always running and calls upon the uzbl-core to display the assets and omxplayer for videos etc, so the memory will vary a lot depending on your assets and the page load.. i think the memory issue is worse when it cant be swapped and oom-killer kills the browser but for some reason the viewer is not restarting the dead browser automatically which makes the screen either stuck or black, so the viewer needs to also restart itself instead of just uzbl via systemctl restart screenly-viewer.service..

@rusko124 the screenshot below was taken on latest screenly image production with manual celery upgrade.. results are bad, look at locked processes, never seen that before... i will upgrade to development and test that branch to see if better memory management is handled..

[ealmonte32]

update: zram modified too high (1.5x built-in memory) is not working well on the RPi 3... it slows the pi to a crawl after some time.. and shows all these locked processes..

Testing on development branch + zram at normal 1x built-in memory and changed swappiness to 10.. no good.. still hangs when memory gets filled..

[ealmonte32]

so testing development branch with load heavy sites like news.google.com and videos and google slides.. and without swap memory the browser just stops, this time instead of black screen it's white, if the viewer somehow knew it was in a "dead" or "failed" state itself, restarting the viewer fixes the issue and no need to reboot the Pi.. when swap is used, and all memory is allocated, the Pi functions less and every command in terminal crawls to execute..

hmm... i thought the viewer was capable of finding the browser dead and restarting it, seems like it is not doing it..

---

screenshots for documentation:

[vpetersson]

and without swap memory the browser just stops, this time instead of black screen it's white, if the viewer somehow knew it was in a "dead" or "failed" state itself,

A white screens means that UZBL started, but for whatever reason, viewer.py was unable to instruct UZBL to switch to the default black screen. This is likely because there is no more memory available.

So taking a step back, it's likely that because we've now added more components (celery, rabbit/redis), we are now running out of memory. My gut feeling tells me that this is either redis or rabbit that is eating up the memory (given that they are databases). With the move to Docker, we can simply confine the memory using Docker metadata.

[ealmonte32]

@vpetersson After much testing and trying to deal with the issue being the memory, to prove it, I tested by creating a script that runs chrome-browser like this:

#!/bin/bash

while [ ! -f /home/pi/screenly/STOP ]
do

DISPLAY=:0 chromium-browser --start-fullscreen -app=https://news.google.com &
sleep 20;
pkill -f chromium-browser;
sleep 0.5;

DISPLAY=:0 chromium-browser --start-fullscreen -app=https://news.ycombinator.com &
sleep 20;
pkill -f chromium-browser;
sleep 0.5;

DISPLAY=:0 chromium-browser --start-fullscreen -app=https://www.nytimes.com & 
sleep 60;
pkill -f chromium-browser;
sleep 0.5;

DISPLAY=:0 chromium-browser --start-fullscreen -app=https://www.cnn.com &
sleep 60;
pkill -f chromium-browser;
sleep 0.5;

DISPLAY=:0 chromium-browser --start-fullscreen -app=https://weather.srly.io &
sleep 30;
pkill -f chromium-browser;
sleep 0.5;

DISPLAY=:0 chromium-browser --start-fullscreen -app=https://clock.srly.io &
sleep 30;
pkill -f chromium-browser;

done
#end

Yes, bad, archaic, but simple and good for testing this rotation of assets in order to generate memory consumption... chrome-browser as we know is a huge memory hog on the Pi, and some of these assets are very memory consuming, when I tried this same asset list on uzbl, it would crash the browser and make the Pi crawl to unusable state within an hour..

So..the results are.. that if killing the process before the next asset kept this working without black screen or freezing browser page, etc, means all we have to do is put in viewer.py a pkill -f uzbl-core after the end of every Asset loop (not after every asset, just once after the loop), that way every iteration has more memory to work with as oppose to how it's handled now.. besides that, as of right now, I cant think of anything else..

[vpetersson]

I think we need to be a bit more intelligent than this. The most obvious way that I can think of is to use cgroups for this. What do you think?

That said, good tests!

[ealmonte32]

Yes, cgroups looks like the perfect tool for this. Will do some tests and see how it goes.

[ealmonte32]

Waiting for rusko to review, but this PR (https://github.com/Screenly/screenly-ose/pull/1304 , along with the celery one) fixes this. I would love for these users to try this and see how it works for them.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Moderatopeab]

I have this same problem. I have google calendar embedded webpage, weather webpage and few jpg/png files. After about 6 hours the screen goes black but screenly admin panel works fine. Balena allso doesn't show any errors. Only solution is to unplug the power supply. I have this power timer to shut off rasperry pi 3B+ every night but it hasn't helped. Unfortunately my linux skills are so poor that i cant troubleshoot exact problem. How can i update Screenly OSE when possible fix is released?

[ealmonte32]

@Moderatopeab If your linux skills are very poor, they would still be capable of doing something super simple in the crontab instead of literally having a timer on the power supply.. can't believe you're doing that!

Ok, so basically you could literally set up a reboot of the Pi every number of X hours, or you could simply restart the screenly viewer service which is the better approach to this, and it will all be working without the need to restart anything physically or the Pi system itself.

I am very tired right now so I really dont want to start linking things for you, but if you could, please read how to access the raspberry pi via SSH or physically, and log on to the terminal / console via the pi user, and follow the crontab instructions I mention here towards the end: https://forums.screenly.io/t/display-often-hangs/123/31

and the part about 40 - you will need to figure out how often you want to restart the service, and then set this accordingly, you could find simple online documents that tell you how to set timing in crontab and the format (https://www.adminschoice.com/crontab-quick-reference) ..

hope this helps..

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.