enlarge swap partition on every boot after genup

[zd59]

Hi!

Yesterday I upgraded system with genup after you helped me with key server problem - thanks again. After genup finished after 6 hours (at night) the power manager shut off the display, but the next day the screen was black with only mouse cursor, which can be moved in a small square in a middle of a screen - GUI logon I guessed, so I entered a password. Then after some time the mouse was movable all over the black screen. From its shape change I discovered a place with teminal window, click and write reboot. The system rebooted and started with swap partition resize.. I must notice, that I checked from remote access terminal that genup finished - log files were a few hours old with no errors. After that check I rebooted blindly with above procedure. After reboot and logon with user I created at first install, porthole reported two config files need to be updated (do not explicitly name them), so I fired dispatch-conf as you suggested. There were many options I selected Replace for both config files. Now after every boot system enlarges swap to 1GB.

There are two problems:

1. black screen after a period of inactive time, that can not be changed - only movable mouse cursor is on it
2. swap partition enlarge on every boot.

[sakaki-]

Did the last line output from genup state All done - your system is now up-to-date!? What capacity your microSD card? It is strange that the swap (file, not partition) resizing is happening every time you reboot. What do you get if you run:

pi64 ~ # df
pi64 ~ # ls -l /var/cache/swap/swap1

Do you have any other processes running in the background on your RPi3? The 'black screen' issue is normally indicative of the system running out of memory. As it happens, I have an RPi3B+ here that was genup-d from a clean instance of the image, then rebooted, dating from when we were discussing issue #61, and it responded immediately to mouse movement just now.

Can you post the recent boot output from /var/log/rc.log please?

[zd59]

Hi sakaki!

Did the last line output from genup state All done - your system is now up-to-date!?

Do not know, black screen issue, but looking to log files from remote ssh console, no errors reported. The system is pure as installed, 16GB microSD:

zd@pi64 ~ $ df -H
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        15G  9.6G  4.9G  67% /
devtmpfs         11M     0   11M   0% /dev
tmpfs           102M  820k  101M   1% /run
cgroup_root      11M     0   11M   0% /sys/fs/cgroup
shm             508M     0  508M   0% /dev/shm
/dev/mmcblk0p1   67M   32M   35M  48% /boot
none            508M  4.1k  508M   1% /run/user/1001
zd@pi64 ~ $ ls -l /var/cache/swap/swap1
-rw------- 1 root root 977272832 Nov  9 16:18 /var/cache/swap/swap1
zd@pi64 ~ $ 

The above looks OK. /var/log/rc.log listing from last boot:

rc boot logging started at Thu Jan 1 01:00:14 1970

• WARNING: clock skew detected!
• Setting the local clock based on last shutdown time ... [ ok ]
• Checking local filesystems ... root: clean, 440370/3833856 files, 2591740/3808224 blocks fsck.fat 4.1 (2017-01-24) /dev/mmcblk0p1: 175 files, 3815/8057 clusters [ ok ]
• Remounting root filesystem read/write ... [ ok ]
• Remounting filesystems ... [ ok ]
• Updating /etc/mtab ...
• Creating mtab symbolic link [ ok ]
• Activating swap devices ... [ ok ]
• Mounting local filesystems ... [ ok ]
• Configuring kernel parameters ... [ ok ]
• Creating user login records ... [ ok ]
• Wiping /tmp directory ... [ ok ]
• Restoring Mixer Levels ... [ ok ]
• Mounting misc binary format filesystem ... [ ok ]
• Loading custom binary format handlers ... [ ok ]
• Setting hostname to pi64 ... [ ok ]
• Setting terminal encoding [UTF-8] ... [ ok ]
• Setting keyboard mode [UTF-8] ... [ ok ]
• Loading key mappings [uk] ... [ ok ]
• Bringing up network interface lo ... [ ok ]
• Setting up tmpfiles.d entries ... [ ok ]
• Initializing random number generator ... [ ok ]
• Starting rngd ... Note, reference of entropy sources by index is deprecated, use entropy source short name instead

Disabling 1: TPM RNG Device (tpm)

Initalizing available sources

[ ok ]

rc boot logging stopped at Fri Nov 9 16:14:51 2018

rc default logging started at Fri Nov 9 16:14:51 2018

• WARNING: clock skew detected!
• Starting dbus ... [ ok ]
• Checking your configfile (/etc/syslog-ng/syslog-ng.conf) ... [ ok ]
• Starting syslog-ng ... [ ok ]
• Starting consolekit ... [ ok ]
• Starting NetworkManager ... [ ok ] Connecting......... 1sConnecting.......... 1sConnecting........... 1sConnecting............ 1sConnecting............. 1sConnecting.............. 1sConnecting............... 1sConnecting............... 0s [offline]
• Marking NetworkManager as inactive. It will automatically be marked
• as started after a network connection has been established.
• WARNING: NetworkManager has started, but is inactive
• Starting bluetooth ... [ ok ]
• Starting chronyd ... [ ok ]
• Starting cronie ... [ ok ]
• Starting cupsd ... [ ok ]
• WARNING: netmount will start when NetworkManager has started
• Starting rpi3-expand-swap ...
• Setting swapfile size to 932 MiB, please wait 95420416 bytes (95 MB, 91 MiB) copied, 2 s, 39.4 MB/s96468992 bytes 932+0 records in and so on to 1GB 932+0 records out 977272832 bytes (977 MB, 932 MiB) copied, 143.348 s, 6.8 MB/s mode of '/var/cache/swap/swap1' changed from 0644 (rw-r--r--) to 0600 (rw-------) Setting up swapspace version 1, size = 932 MiB (977268736 bytes) LABEL=swap1, UUID=46fef95b-ce6d-4e00-8529-741c5051b30a [ ok ]
• Starting rpi3-safecompositor ... [ ok ]
• Starting rpi3-safecursor ... [ ok ]
• Setting up lightdm ... [ ok ]
• Starting rpi3-bluetooth ... [ ok ]
• Starting rpi3-ethfix ...
• RPi3 model B+ detected
• Turning off eth0 offloading, for stability [ ok ]
• Starting rpi3-wifi-regdom ...
• Setting WiFi regulatory domain to 'GB' [ ok ]
• Starting rpi3-zswap ...
• Enabling transparent ZSWAP compressed cache [ ok ]
• Starting sshd ... [ ok ]
• Starting local ... [ ok ]

rc default logging stopped at Fri Nov 9 16:18:13 2018

[sakaki-]

OK, edit (as root) /etc/conf.d/rpi3-expand-swap, uncomment the SWAPFILE_PCT_AVAIL line, and edit it so it reads:

SWAPFILE_PCT_AVAIL=50

This allows the swapfile to take up 50% of the free space on the drive, at most. (It will still dynamically resize once space becomes tight - arguably this is what you want though.) Leave the rest of the file as is. Save, and reboot. You will get one more resize, but the swapfile should be a full 1GiB now. Reboot again - it should not resize this time?

Given that 16GB is a common microSD card capacity, I'll probably change the default parameters for the swap resizer a little, so this issue doesn't crop up.

[zd59]

Thank you, solved. Confirmed, no more swap resize after that change you suggested. But GUI freeze a lot. Then I access RPI remote SSH - CPU load near zero, RAM usage 500-700MB. There is other reason for freeze than A RAM shortage.

[sakaki-]

Odd about the freezing, haven't seen it here. Could you post the output of:

pi64 ~ # eix-installed -a

so I can check the versions of your installed packages, now that you have run genup, match what I am using here. Thanks!

[zd59]

The system is now very slow, unresponsive. Firefox occupy the most of a RAM. And suspecting constantly swapping to created swap file. Waiting for FF open and show a web page in minutes. Is there any lighter web browser to install instead of FF. Might be Chromium. The listing of installed components is in appended file.

I observed freezing only of a GUI (xfce), remote SSH aceess works and never freeze. I can work remote ssh terminal even when local GUI is frozen.

zd59_installed_RPI3_sakaki_20181110.txt

[sakaki-]

OK thanks, other than tigervnc-1.9.0-r1 on your system, these look identical. Are you running a tigervnc server by any chance?

[zd59]

Thanks for your help.

No, TigerVNC is my main remote access to GUI, but it is not configured to autostart. I explicitly start vncserver from remote terminal when need it, else is not running. At current state GUI greeze every time not in use some time. Right now it is frozen and from remote terminal htop: CPU load 0 mem 215M/969M Swap 18.5M/1024M

The above data show no load and low resource usage, but still GUI frozen.

Have nice Sunday.

[sakaki-]

Do you use the xscreensaver by any chance? What does rc-update show display?

[sakaki-]

Also, as you can ssh in as root even when the GUI is frozen, what happens if you do (over ssh, in such a situation):

pi64 ~ # rc-service xdm stop
pi64 ~ # rc-service xdm start

does the GUI boot up again?

[zd59]

The first thing, I done with, when I saw a frozen/ blank GUI was disabled screensaver and also power manager. Still not helping with the issue.

pi64 ~ # rc-service xdm stop
pi64 ~ # rc-service xdm start

The result is black screeen with mouse cursor movable.

[sakaki-]

OK, just to double-check could you please post the output of rc-update show?

Also, what is your typical workload when the freeze happens? You state above you use tigervnc as the main access to the GUI - so do you have an actual screen and keyboard attached to the RPi3 and then serve that remotely via e.g. x0vncserver, and/or do you start a separate virtual display with vncserver for remote access? If the latter, is the Pi running without any physical monitor attached?

[zd59]

You misunderstood me: I'm testing RPI & Gentoo as a desktop - monitor attached to HDMI, keyboard & mouse wireless USB dongle (Genius one dongle for mouse & keyboard). Attached LAN, WiFi deliberately off (after tested and working). I was playing with settings/config of Tigervnc server until successfully working (vncserver :1) - remote connected to it. Then vncesever :1 -kill and never run it again. I know it can be resources hungry. I did not set it to activate at boot process.

The below was at local work - monitor & mouse & keyboard & LAN attached. Only once frozen at work, opened Firefox (maybe 2 or 3 tabs), xterminal, htop, RAM mostly used. The majority hangs was when GUI was idle (no program besides GUI opened) for some time - machine left alone. After I'm returning, the screen was black, with only mouse cursor. In a case, when logged off and logon prompt window was on screen when left it for some time, the screen was frozen in that state, not become black. Only mouse was movable, none of a mouse buttons or keyboard could wake it.

rc-update_show.txt

[sakaki-]

OK, I understand your configuration now, thanks for clarifying.

So, could you please try commenting out (temporarily) the following line in /boot/config.txt, so it reads:

#dtoverlay=vc4-fkms-v3d,cma-256

Leave the rest of the file as-is, save, reboot, and then check if the black screen issue is still present?

Obviously, this is going to make video performance etc. much worse, as the vc4 kernel module etc. won't be used, but it should help to narrow down the source of the problem.

Thanks for the time you are taking on this issue btw, and apologies it isn't working fully for you ><

[zd59]

Thought about that at the beginning :-) Done. Will have to wait to return home in > 4 hours to check the influence.

What about the last line in config.txt --> gpu_mem=16 - isn't it a way to low?

I'm interested for your OS, so I'm also willing to help you make it better. Computers (HW & SW) are my hobby and work since ZX Spectrum, when I finished my study at university. So time spent there is countless. :-)

[sakaki-]

The gpu_mem stanza only affects reserving buffer space for VC4 accelerated ops, which aren't available under 64-bit atm, so that (minimal) allocation should still be fine.

[zd59]

Uh, what a difference! Commenting dtoverlay in config.txt is like in a car put in 6 gear from the first. Everything is working, no more freezing or blank screen. Whole system is responsive, Firefox is much faster, but every page scroll or new tab opening results in almost 100% CPU load on all cores.

Conclusion: vc4-fkms do not work on my RPI 3B+.

Regards!

[sakaki-]

OK, well it sounds as if something is going wrong with vc4-fkms when memory gets tight, because it is certainly faster than the default framebuffer otherwise (you can check this with movie playback etc.)

I'll have a look this end to see if anything has changed with recent versions of the userland mesa stuff, and with prior versions of the kernel. Thanks, sakaki.

[sakaki-]

On other thing - if you get a chance, could you try booting with:

dtoverlay=vc4-kms-v3d,cma-256

in /boot/config.txt (i.e., try the kms, not fkms, overlay) and see if that is any better?

[zd59]

Hello Sakaki!

vc4-kms-v3d - there is no practical speed difference with, or without. Waited over night - GUI logon screen reappear after mouse move, but keyboard is dead. So for my case dtoverlay should be commented in config.txt and system booted without vc4-kms-v3d or vc4-fkms-v3d.

p.s.: I installed mc with porthole and discovered, that compilation is done with only one CPU core (1/4 available speed). This should be corrected.

Regards!

[sakaki-]

Compilation uses only one core deliberately, since with more assigned the machine can get totally unresponsive and/or start swapping, and many people complained about that (I originally shipped it with settings that would use all four cores). Those who wish to do so can override the make.defaults settings in /etc/portage/make.conf; they are currently set at:

# conservative build parallelism, to avoid hitting swap
# override locally if desired
MAKEOPTS="-j2 -l1"
EMERGE_DEFAULT_OPTS="--jobs=2 --load-average=1"

If you wanted to be fully aggressive, you could put the following at the end of /etc/portage/make.conf:

MAKEOPTS="-j5 -l4"
EMERGE_DEFAULT_OPTS="--jobs=5 --load-average=4"

or whatever values in-between you wish, of course.

It is possible the vc4 issues are due to a mismatch between the kernel driver and mesa userspace driver. I'm trying out a more recent kernel release at the moment to see if this helps. Thanks again for taking the time to test this!

[zd59]

Thanks for an explanation regarding compiling. Interesting. When compiling on a normal machine (intel i5 or i7) I always set up "make -jX" - X = numbers of cores or threads, and machine works pretty normal/responsive.

I like to help developers if I can.

[sakaki-]

Closing now, please re-open if the issue is still present given the comments above, and/or with the new 1.4.0 release / kernel 4.19.y. Best, sakaki