Folder2ram can't sync, might due to target is busy

[wolverinevn]

Armbian Stretch 5.3.9-sunxi64, / on SSD Issue: folder2ram can not sync /var/log even when "folder2ram -syncall" gives successfully message.

Details in step:

1. I mount /var/log with folder2ram using conf file
2. Everytime I reboot, folder2ram failed to mount /var/log, log filled with erros likes "cannot create regular file etc."
3. Then I manually mount /var/log with "folder2ram -mountall"
4. I remove all "regular files" then "folder2ram -syncall"
5. Next time, folder2ram failed again with same error, same files
6. If "folder2ram -umountall" then remove all "regular files", next reboot is still failed
7. "folder2ram -umountall" then "mount | grep /var/log" shows that it is still being mounted
8. "umount -l /var/log" and remove all "regular files" fix the issue with reboot. "unmount /var/log" is not successful with error "target is busy" (no doubt)

[bobafetthotmail]

I assume that you are running folder2ram as root already.

I can't reproduce this on Debian and I don't have a device supported by Armbian, maybe they are doing something Debian does not do.

Can you post the result of this command? (add sudo if you are not logged in as root) lsattr /var/log

Also can you post the actual errors you are getting in the terminal/logs?

[wolverinevn]

@bobafetthotmail lsattr: Inappropriate ioctl for device While reading flags on /var/log/debug lsattr: Inappropriate ioctl for device While reading flags on /var/log/wtmp lsattr: Inappropriate ioctl for device While reading flags on /var/log/syslog lsattr: Inappropriate ioctl for device While reading flags on /var/log/messages lsattr: Inappropriate ioctl for device While reading flags on /var/log/user.log lsattr: Inappropriate ioctl for device While reading flags on /var/log/auth.log lsattr: Inappropriate ioctl for device While reading flags on /var/log/kern.log lsattr: Inappropriate ioctl for device While reading flags on /var/log/daemon.log lsattr: Inappropriate ioctl for device While reading flags on /var/log/mail.log lsattr: Inappropriate ioctl for device While reading flags on /var/log/mail.warn lsattr: Inappropriate ioctl for device While reading flags on /var/log/mail.info lsattr: Inappropriate ioctl for device While reading flags on /var/log/php7.0-fpm.log lsattr: Inappropriate ioctl for device While reading flags on /var/log/armbian-hardware-monitor.log lsattr: Inappropriate ioctl for device While reading flags on /var/log/mail.err lsattr: Inappropriate ioctl for device While reading flags on /var/log/lost+found lsattr: Inappropriate ioctl for device While reading flags on /var/log/plexmediaserver lsattr: Inappropriate ioctl for device While reading flags on /var/log/mosquitto lsattr: Inappropriate ioctl for device While reading flags on /var/log/letsencrypt lsattr: Inappropriate ioctl for device While reading flags on /var/log/chrony lsattr: Inappropriate ioctl for device While reading flags on /var/log/openmediavault lsattr: Inappropriate ioctl for device While reading flags on /var/log/proftpd lsattr: Inappropriate ioctl for device While reading flags on /var/log/nginx lsattr: Inappropriate ioctl for device While reading flags on /var/log/cron-apt lsattr: Inappropriate ioctl for device While reading flags on /var/log/samba lsattr: Inappropriate ioctl for device While reading flags on /var/log/ntpstats lsattr: Inappropriate ioctl for device While reading flags on /var/log/watchdog lsattr: Inappropriate ioctl for device While reading flags on /var/log/unattended-upgrades lsattr: Inappropriate ioctl for device While reading flags on /var/log/sysstat lsattr: Inappropriate ioctl for device While reading flags on /var/log/apt lsattr: Inappropriate ioctl for device While reading flags on /var/log/btmp lsattr: Inappropriate ioctl for device While reading flags on /var/log/tallylog lsattr: Inappropriate ioctl for device While reading flags on /var/log/rpimonitor_top.txt lsattr: Inappropriate ioctl for device While reading flags on /var/log/fontconfig.log lsattr: Inappropriate ioctl for device While reading flags on /var/log/rpimonitor lsattr: Inappropriate ioctl for device While reading flags on /var/log/monit.log lsattr: Inappropriate ioctl for device While reading flags on /var/log/lastlog lsattr: Inappropriate ioctl for device While reading flags on /var/log/dpkg.log lsattr: Inappropriate ioctl for device While reading flags on /var/log/faillog lsattr: Inappropriate ioctl for device While reading flags on /var/log/bootstrap.log lsattr: Inappropriate ioctl for device While reading flags on /var/log/alternatives.log

I think the issue began when I "redirect" some programs to use syslog instead of their own log files but not so sure. I can overcome this error with some work around but my mainly concern is folder2ram did not throw any error with umount or syncall command. Another concern is an error when shutdown might send my device into un-controllable state until I physically reset it.

[wolverinevn]

I afraid that I don't keep the last log of error since I fixed it with unmount -l /var/log and remove all "regular files". logs looks like:

folder2ram[622]: start /var/log folder2ram[622]: can not create regular file /var/log/nginx/openmediavault-webgui_access.log, no such file or directory ... folder2ram[622]: failed to mount /var/log folder2ram[622]: start /var/tmp folder2ram[622]: start /var/spool

[bobafetthotmail]

Can you run the same lsattr /var/log command with /var/log unmounted? I forgot to say that before, it does not work if the folder is mounted in RAM by folder2ram.

While you wait for me to look at this issue you can disable folder2ram for /var/log by editing its config file at /etc/folder2ram/folder2ram.conf and commenting out the line of /var/log

I think the issue began when I "redirect" some programs to use syslog instead of their own log files but not so sure.

What did you use to redirect program logs to syslog? You changed their configuration or used links or symlinks?

[wolverinevn]

sudo folder2ram -umountall
sudo lsattr /var/log

--------------e---- /var/log/rpimonitor
--------------e---- /var/log/wtmp
--------------e---- /var/log/debug
--------------e---- /var/log/mosquitto
--------------e---- /var/log/ntpstats
--------------e---- /var/log/nginx.log
--------------e---- /var/log/proftpd
--------------e---- /var/log/php7.0-fpm.log
--------------e---- /var/log/unattended-upgrades
--------------e---- /var/log/openmediavault
--------------e---- /var/log/apt
--------------e---- /var/log/armbian-hardware-monitor.log
--------------e---- /var/log/lastlog
--------------e---- /var/log/cron-apt
--------------e---- /var/log/watchdog
--------------e---- /var/log/sysstat
--------------e---- /var/log/btmp
--------------e---- /var/log/messages
--------------e---- /var/log/mail.log
--------------e---- /var/log/tallylog
--------------e---- /var/log/mail.info
--------------e---- /var/log/syslog
--------------e---- /var/log/letsencrypt
--------------e---- /var/log/alternatives.log
--------------e---- /var/log/rpimonitor_top.txt
--------------e---- /var/log/mail.err
--------------e---- /var/log/kern.log
--------------e---- /var/log/bootstrap.log
--------------e---- /var/log/mail.warn
--------------e---- /var/log/chrony
--------------e---- /var/log/lost+found
--------------e---- /var/log/monit.log
--------------e---- /var/log/fontconfig.log
--------------e---- /var/log/daemon.log
--------------e---- /var/log/user.log
--------------e---- /var/log/nginx
--------------e---- /var/log/faillog
--------------e---- /var/log/samba
--------------e---- /var/log/plexmediaserver
--------------e---- /var/log/dpkg.log
--------------e---- /var/log/auth.log

This time it successfully unmount /var/log (check with mount | grep /var/log after). So I guess the issue with mouting /var/log at boot was gone. With redirecting program to syslog, I use 3 method, including their configuration, mount a directory in /var/log to docker and symlink. But none of them is involved with /var/log/nginx.

[wolverinevn]

Today, issue happen again. Here the log:

Jan 28 13:39:29 moOde folder2ram[630]: will now start all mountpoints
Jan 28 13:39:29 moOde folder2ram[630]: start /var/log
Jan 28 13:39:29 moOde folder2ram[630]: cp: cannot create regular file '/var/log/samba/log.10.8.0.2': No such file or directory
Jan 28 13:39:29 moOde folder2ram[630]: cp: cannot create regular file '/var/log/samba/log.': No such file or directory
Jan 28 13:39:29 moOde folder2ram[630]: cp: cannot create regular file '/var/log/samba/log.nmbd': No such file or directory
Jan 28 13:39:29 moOde folder2ram[630]: cp: cannot create regular file '/var/log/samba/log.smbd': No such file or directory
Jan 28 13:39:29 moOde folder2ram[630]: cp: cannot create regular file '/var/log/samba/log.192.168.30.1': No such file or directory
Jan 28 13:39:29 moOde folder2ram[630]: cp: cannot create directory '/var/log/samba/cores': No such file or directory
Jan 28 13:39:29 moOde folder2ram[630]: cp: preserving times for '/var/log/samba': No such file or directory
Jan 28 13:39:29 moOde folder2ram[630]: cp: cannot create directory '/var/log/plexmediaserver/PMS Plugin Logs': No such file or directory
Jan 28 13:39:29 moOde folder2ram[630]: cp: preserving times for '/var/log/plexmediaserver': No such file or directory
Jan 28 13:39:29 moOde folder2ram[630]: copy files to /var/log failure, rolling back the mount

and sudo lsattr /var/log:

lsattr: Inappropriate ioctl for device While reading flags on /var/log/plexmediaserver
lsattr: Inappropriate ioctl for device While reading flags on /var/log/mosquitto
lsattr: Inappropriate ioctl for device While reading flags on /var/log/rpimonitor
lsattr: Inappropriate ioctl for device While reading flags on /var/log/nginx
lsattr: Inappropriate ioctl for device While reading flags on /var/log/proftpd
lsattr: Inappropriate ioctl for device While reading flags on /var/log/cron-apt

[wolverinevn]

dmesg

[    2.744116]  sda: sda1
[    3.146789] EXT4-fs (sda1): mounted filesystem with writeback data mode. Opts: (null)
[    4.389585] EXT4-fs (sda1): re-mounted. Opts: errors=remount-ro,commit=300

Does it look related?