search

MichaIng / DietPi

Lightweight justice for your single-board computer!
https://dietpi.com/
GNU General Public License v2.0
4.85k stars 495 forks source link

DietPi-Drive_Manager | Rewrite/improve #1858

Closed butterbrot21 closed 6 years ago

butterbrot21 commented 6 years ago

Required Information:

Additional Information:

Steps to reproduce:

  1. Fresh install
  2. drivemanager-> no hard drive visible
  3. manuell formating + mount -> still not visible

Expected behaviour:

Actual behaviour:

Extra details:

Unfortunately, I can't use the hard disk on my odroid hc2 (wdRed 2TB). I did a fresh install and i want to use the harddrive(2tb) for my userdata (nextcloud, ampache...)

During isntall i cant choose the drive for my useredata and i cant mount it with the drive'_manager. After the first install i tried to format the drive and mount it manually.

Unfortunatelly i still cant use the drive for my userdata. The Strange thing is, that the drive_manager says that the user data is on sda1 and on mmcblk1p1.

which irritates me too is that sda1 via drive_manager only is 15Gb big? lsbkid shows me the 1.8tb drive as sda1??

drive_manager shows this:

─(/dev/sda)──────────────────────────────────────── │ / : /dev/sda1 | ext4 | Size: 15G | Used: 1.9G (14%) │ ─(/dev/mmcblk1)──────────────────────────────────────── │ /boot : /dev/mmcblk1p1 | vfat | Size: 71M | Used: 32M (45%) │ / : /dev/mmcblk1p2 | ext4 | Size: 15G | Used: 1.9G (14%) │ ─────────────────────────────────────────────────────── CODE: SELECT ALL

lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 1.8T 0 disk └─sda1 8:1 0 1.8T 0 part /datastore mmcblk1 179:0 0 14.9G 0 disk ├─mmcblk1p1 179:1 0 71M 0 part /boot └─mmcblk1p2 179:2 0 14.8G 0 part / CODE: SELECT ALL

nano /etc/fstab

Samba Client------------------------------------------------------

/mnt/samba . Please use dietpi-config and the Networking Options: NAS menu to setup this mount

FTP Client Mount--------------------------------------------------

/mnt/ftp_client . Please use dietpi-config and the Networking Options: NAS menu to setup this mount

NFS Client Mount--------------------------------------------------

/mnt/nfs_client . Please use dietpi-config and the Networking Options: NAS menu to setup this mount

TMPFS / MISC ------------------------------------------------------

proc /proc proc defaults 0 0 tmpfs /tmp tmpfs defaults,size=1023M,noatime,nodev,nosuid,mode=1777 0 0 tmpfs /var/log tmpfs defaults,size=20m,noatime,nodev,nosuid,mode=1777 0 0 tmpfs /DietPi tmpfs defaults,size=10m,noatime,nodev,nosuid,mode=1777 0 0

Internal Drives---------------------------------------------------

UUID=af314fbe-6db6-4970-84b4-19fd22bd6bf7 / auto defaults,noatime 0 0 UUID=7451-4AA3 /boot auto defaults,noatime 0 0 UUID=cce05d0d-6515-497e-a473-b5660d25ab86 / auto defaults,noatime 0 0 /var/swap none swap sw 0 0 UUID="cce05d0d-6515-497e-a473-b5660d25ab86" /datastore ext4 defaults 0 0

Unfortunately I am not a Linux expert and would be very happy about simply commands

I really hope you can help me. I love dietpi!!!

LexiconCode commented 6 years ago

I can replicate this but I can manually mount a drive.

butterbrot21 commented 6 years ago

Sounds good can you help me with wth this? After manually mount, can you see the drive in the drive_manager? If so is it possible to use it as userdata?

this is what i did: mkdir /datastore mount /dev/sda1 /datastore blkid /dev/sda1 ->UUID="cce05d0d-6515-497e-a473-b5660d25ab86" nano /etc/fstab -->UUID=cce05d0d-6515-497e-a473-b5660d25ab86 /datastore ext4 defaults 0 0

now i see the drive but i cant use it for my userdata:

lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 1.8T 0 disk └─sda1 8:1 0 1.8T 0 part /datastore mmcblk1 179:0 0 14.9G 0 disk ├─mmcblk1p1 179:1 0 71M 0 part /boot └─mmcblk1p2 179:2 0 14.8G 0 part / CODE: SELECT ALL

drive_manager shows this: ─(/dev/sda)──────────────────────────────────────── │ / : /dev/sda1 | ext4 | Size: 15G | Used: 1.9G (14%) │ ─(/dev/mmcblk1)──────────────────────────────────────── │ /boot : /dev/mmcblk1p1 | vfat | Size: 71M | Used: 32M (45%) │ / : /dev/mmcblk1p2 | ext4 | Size: 15G | Used: 1.9G (14%) │ ─────────────────────────────────────────────────────── --> here sda1 is only 15G same as mmcblk1p2 -both mmcblk1p2 and sda1 shows me the x for userdata, is this normal?

Is there a other way to mount the drive an use it for userdata? Thanks for the response and help!

Fourdee commented 6 years ago

@butterbrot21

Thanks for the report 👍

Appears /dev/sda1 is being incorrectly identified as rootfs

│ / : /dev/sda1 | ext4 | Size: 15G | Used: 1.9G (14%)
│ / : /dev/mmcblk1p2 | ext4 | Size: 15G | Used: 1.9G (14%)

#ref
UUID=cce05d0d-6515-497e-a473-b5660d25ab86 / auto defaults,noatime 0 0
/var/swap none swap sw 0 0
UUID="cce05d0d-6515-497e-a473-b5660d25ab86" /datastore ext4 defaults 0 0

I'll run some local tests, see if we can replicate.

@butterbrot21 @LexiconCode Was this on a fresh DietPi image install, or after running the dietpi-update ?

butterbrot21 commented 6 years ago

Hey,

Yes it was on a fresh install.

It looked as if it had automatically run an update after first login via putty. But i dont run dietpi-update. After install i did the normal setup with selected software.

One time i tied to format the disk and mount it manually before the first setup/inmstaller. Unfortunately without success.

i hope i can help I'm really looking forward to use dietpi on my odroid hc2 :)

Keep up the good work 👍

Fourdee commented 6 years ago

🈯️ https://github.com/Fourdee/DietPi/commit/8f053fdb26c00c34ddc988a649cdcaef409dad85 >> Another bug with incorrect cat listing: image

Fourdee commented 6 years ago

Notes:

df faster than lsblk

root@DietPi:~# time lsblk /dev/sda1 | awk '{print $NF}'
MOUNTPOINT
/datastore

real    0m0.024s
user    0m0.008s
sys     0m0.015s
root@DietPi:~# time df -P /dev/sda1 | awk '{print $NF}'
on
/datastore

real    0m0.012s
user    0m0.003s
sys     0m0.013s
butterbrot21 commented 6 years ago

in another try/reinstal, I tried to format sda after the first setup/software-install and mount it afterwords, becaouse it looked like that the userdata was on sda and on sd:

─(/dev/sda)──────────────────────────────────────── │ / : /dev/sda1 | ext4 | Size: 15G | Used: 1.9G (14%) │ ─(/dev/mmcblk1)──────────────────────────────────────── │ /boot : /dev/mmcblk1p1 | vfat | Size: 71M | Used: 32M (45%) │ / : /dev/mmcblk1p2 | ext4 | Size: 15G | Used: 1.9G (14%)

Unfortunately without success. After a reboot I could not access via putty anymore^^ I think the error message was " ssh root@xxx.xxx .... declined" Maybee this info can help a little bit. I'm not an expert with linux and programming, but I try to learn quickly :)

I am very happy and excited that you look at the issue for the hc2 👍

Fourdee commented 6 years ago

Notes to self, detection rewrite:

Fourdee commented 6 years ago

Working on a rewrite, should be done in a few days. Now support network mounts aswell.

butterbrot21 commented 6 years ago

Sounds great! Thanks for the fast response and fix! Keep up the good work 👍

Fourdee commented 6 years ago

ToDo:

Fourdee commented 6 years ago

image

MichaIng commented 6 years ago

@Fourdee As you already touch this. I have an issue with drive manager currently not recognizing my two drives, that are attached via HDD docking station:

Device Boot Start End Sectors Size Id Type /dev/mmcblk0p1 8192 96453 88262 43.1M c W95 FAT32 (LBA) /dev/mmcblk0p2 98304 125435903 125337600 59.8G 83 Linux

Disk /dev/sdb: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/sda: 2.7 TiB, 3000592982016 bytes, 5860533168 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes

- `mount`

`/dev/sda on /mnt/sda type ext4 (rw,noatime,data=ordered,x-systemd.automount)`
- `df`

`/dev/sda       2884154032 226415000 2511209320   9% /mnt/sda`
- `lsblk`

NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 2.7T 0 disk /mnt/sda sdb 8:16 0 1.8T 0 disk mmcblk0 179:0 0 59.8G 0 disk ├─mmcblk0p1 179:1 0 43.1M 0 part /boot └─mmcblk0p2 179:2 0 59.8G 0 part /


`/dev/sda` is my data drive, `/dev/sdb` my backup drive, thus only mounted once at night for backup.
Don't ask my why their dev name does not contain any partition number. Of course they just have one partition, ones formated via Raspbian.
Docking station: `Inateck FD2003B` with two docks
- The two docks already needed some Linux/Debian feature to be recognized correctly, that came first with Jessie, if I remember right. Before that it was not able to connect two separate drives via single USB port or something like that. Can't remember any more exactly, but I guess there might be a connection with the strange dev naming.

That time I wrote down the following steps to create the HDD file systems, thus there was no partition number since the first time I attached them:
fdisk -l => sda
mkfs.ext4 /dev/sda
mkdir /mnt/sda
mount /dev/sda /mnt/sda


The problem I guess is, how the check if the `/dev/sda` is already the partition/fs, or if it's just the raw device, where partitions are needed, especially if it's not mounted (otherwise `mount` would work well as check). `fdisk -l` does not have any `drive` or partition info about them at all.

I remember some DietPi versions ago, they WERE recognized! I know this because I always needed to adjust the mount points from /dev/UUID to /mnt/sda|sdb again, as these locations are used within some scripts, Nextcloud etc 😆.
Fourdee commented 6 years ago

@MichaIng

no partition number detection appears to be working for me with latest testing version: image

MichaIng commented 6 years ago

@Fourdee Ah great, nice rework btw., drive_manager detection speed is impressively improved!

Do you know about the condition for no partition number? When I use USB stick on RPi or VM, even with just 1 partition, the drive to mount always has a partition number 1 and the /dev/sdX without partition number exist besides, but itself is not mountable.

Fourdee commented 6 years ago

Note to self: We can use to detect unformatted connected drives: Check block devices against blkid

root@DietPi:~# ls -lha /sys/block/
loop0/   loop4/   mmcblk0/ ram11/   ram15/   ram5/    ram9/
loop1/   loop5/   ram0/    ram12/   ram2/    ram6/    sda/
loop2/   loop6/   ram1/    ram13/   ram3/    ram7/
loop3/   loop7/   ram10/   ram14/   ram4/    ram8/

root@DietPi:~# blkid
/dev/mmcblk0p1: LABEL="boot" UUID="5DB0-971B" TYPE="vfat" PARTUUID="27504eef-01"
/dev/mmcblk0p2: LABEL="rootfs" UUID="060b57a8-62bd-4d48-a471-0d28466d1fbb" TYPE="ext4" PARTUUID="27504eef-02"
/dev/mmcblk0: PTUUID="27504eef" PTTYPE="dos"

Do you know about the condition for no partition number? When I use USB stick on RPi or VM, even with just 1 partition, the drive to mount always has a partition number 1 and the /dev/sdX without partition number exist besides, but itself is not mountable.

Unsure, I believe it depends if a partition table is created or not.

Fourdee commented 6 years ago

Hmmm:

root@DietPi:~# blkid
/dev/mmcblk0p1: LABEL="boot" UUID="5DB0-971B" TYPE="vfat" PARTUUID="27504eef-01"
/dev/mmcblk0p2: LABEL="rootfs" UUID="060b57a8-62bd-4d48-a471-0d28466d1fbb" TYPE="ext4" PARTUUID="27504eef-02"
/dev/mmcblk0: PTUUID="27504eef" PTTYPE="dos"
root@DietPi:~# lsblk
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda           8:0    1 14.9G  0 disk
mmcblk0     179:0    0 14.9G  0 disk
├─mmcblk0p1 179:1    0 41.8M  0 part /boot
└─mmcblk0p2 179:2    0 14.8G  0 part /
root@DietPi:~# lsblk^C
root@DietPi:~# parted /dev/sda
Error: Could not stat device /dev/sda - No such file or directory.
Retry/Cancel? c
root@DietPi:~# parted /dev/sda1
Error: Could not stat device /dev/sda1 - No such file or directory.
Retry/Cancel?
dd if=/dev/zero of=/dev/sda bs=4K count=1000
parted
mklabel msdos
#MUST REBOOT HERE system not updating partitions....... even partprobe does nothing...
mkpart primary ext4 0% 100%

Warning: The resulting partition is not properly aligned for best performance.
Ignore/Cancel? i
(parted) p
Model:  (file)
Disk /dev/sda: 4096kB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start  End     Size    Type     File system  Flags
 1      512B   4096kB  4095kB  primary  ext4         lba

---
root@DietPi:~# mkfs.ext4 -F -m 0 /dev/sda1
mke2fs 1.43.4 (31-Jan-2017)
The file /dev/sda1 does not exist and no size was specified.
root@DietPi:~# mkfs.ext4 -F -m 0 /dev/sda
mke2fs 1.43.4 (31-Jan-2017)
Found a dos partition table in /dev/sda
Discarding device blocks: done
Creating filesystem with 4000 1k blocks and 1000 inodes

Allocating group tables: done
Writing inode tables: done
Creating journal (1024 blocks): done
Writing superblocks and filesystem accounting information: done

root@DietPi:~# resize2fs /dev/sda
resize2fs 1.43.4 (31-Jan-2017)
The filesystem is already 4000 (1k) blocks long.  Nothing to do!

root@DietPi:~# blkid
/dev/mmcblk0p1: LABEL="boot" UUID="5DB0-971B" TYPE="vfat" PARTUUID="27504eef-01"
/dev/mmcblk0p2: LABEL="rootfs" UUID="060b57a8-62bd-4d48-a471-0d28466d1fbb" TYPE="ext4" PARTUUID="27504eef-02"
/dev/mmcblk0: PTUUID="27504eef" PTTYPE="dos"
root@DietPi:~# lsblk
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda           8:0    1 14.9G  0 disk
mmcblk0     179:0    0 14.9G  0 disk
├─mmcblk0p1 179:1    0 41.8M  0 part /boot
└─mmcblk0p2 179:2    0 14.8G  0 part /
root@DietPi:~# parted /dev/sda
GNU Parted 3.2
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p
Model:  (file)
Disk /dev/sda: 4096kB
Sector size (logical/physical): 512B/512B
Partition Table: loop
Disk Flags:

Number  Start  End     Size    File system  Flags
 1      0.00B  4096kB  4096kB  ext4
MichaIng commented 6 years ago

@Fourdee

Fourdee commented 6 years ago

@MichaIng

Allow fs check on reboot for root and boot partition. However, we did not yet patch the needed fstab flag: 0eb0db0

Let me send commits 1st, then its open for implementation.

Expand FS manually: #1821

Done, just doing last testing.

Hmm..

root@DietPi:~# dd if=/dev/zero of=/dev/sda bs=4K
dd: error writing '/dev/sda': No space left on device
184628+0 records in
184627+0 records out
756232192 bytes (756 MB, 721 MiB) copied, 1.56076 s, 485 MB/s
root@DietPi:~# partprobe /dev/sda
root@DietPi:~# dd if=/dev/zero of=/dev/sda bs=4K
dd: error writing '/dev/sda': No space left on device
184628+0 records in
184627+0 records out
756232192 bytes (756 MB, 721 MiB) copied, 1.55418 s, 487 MB/s
root@DietPi:~# lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda           8:0    1  14.3G  0 disk
mmcblk0     179:0    0 116.5G  0 disk
├─mmcblk0p1 179:1    0   128M  0 part /boot
└─mmcblk0p2 179:2    0 116.4G  0 part /
Fourdee commented 6 years ago

🈯️ Handled in script with: https://github.com/Fourdee/DietPi/commit/c06fe531818d5ebb6b2c2f63af45dea9d3641229

How to test no partition table formatting:

PREP: Clear any tables, then reboot:

dd if=/dev/zero of=/dev/sda bs=4K count=1337
reboot
Fourdee commented 6 years ago

Ok, outstanding:

@MichaIng If you want a crack at either of the top, go for it.

Fourdee commented 6 years ago

Note to self: 🈯️ Reserved blocks : needs to be below mount/unmount options in whip list.

Fourdee commented 6 years ago

RPi RootFS transfer:

MichaIng commented 6 years ago

@Fourdee

Fourdee commented 6 years ago

@MichaIng

x-systemd.automount is removed, if not supported by kernel, but it is never added currently. I just checked, what the benefits of systemd mount management are and if I understand it right, it just allows in combination with noauto to not mount the drive at boot, but on first access. l

Remove the USB drive, reboot, plug in the USB drive, automatically mounts the drive when plugged in. Or at least it should 😃

Yes your right, appears to disable for all devices. I'll take a look.

EDIT: CONFIG_AUTOFS4_FS=y, built into kernel, module will fail? EDIT2: As far as i'am aware Rock64 was the only SBC with this issue: https://github.com/Fourdee/DietPi/issues/1607#issuecomment-372030565

I found, even that I usually disable RAMlog, /var/log is mounted to RAM again, overmounting existing logs, as dietpi-ramlog.service is not active. Reason is that currently every drive_manager start resets /etc/fstab, re-adding block devices well, but resets all network drives, ram drives and swap file. We need to check for installed/configured network drives, dietpi-ramlog and swapfile and of course manually configured RAM drives. Actually the scraping and re-adding of all existing /etc/fstab entries is missing.

Yep, this needs fixing.

We no longer scrape fstab, instead we generate a fresh /etc/fstab each time via detected devices. The reason for this, basically, scraping fstab is messy and unpredictable. With the system scrapes, we have a valid, standardized entry we can work with.

What the hell are those 7 loop drives??

Inactive default loop devices available to system, i believe. same as ls -lha /dev/tty*

Another feature wish I have, is allowing to set a manual mount target. I just don't like those long UUIDs, even that I understand the benefit of allowing no chance that another drive is accidentally over-mounting an used mount target.

Yep, agree nice idea 👍

Fourdee commented 6 years ago

@MichaIng

An alternative is to use tune2fs -c 1 to make let the system check this drive on every reboot and change this back to initial value again (which is then needed to be safed). But this definitely needs the fstab flag. €: Interesting, it does not on VM!

Tried this on NanoPC T4, doesn't seem to have any effect (include fstab 1 change), no info of repair in dmesg or logs.

Hmm, had this idea of copying /sbin/e2fsck to a ramdisk, we can use during boot before rootfs mount, then repair via service, but, not had a chance to try it yet.

MichaIng commented 6 years ago

@Fourdee Hmm that's sad. Boot cmd line is a bid messy, since it is found very differently on the SBCs:

I seems that indeed all systems handle fsck on reboot quite different and methods that work on the one, wont on the other: https://superuser.com/questions/401217/how-to-check-root-partition-with-fsck

There is this:

2018-06-29 10:56:34 root@micha:/var/log# cat /lib/systemd/system/systemd-fsck-root.service
[Unit]
Description=File System Check on Root Device
Documentation=man:systemd-fsck-root.service(8)
DefaultDependencies=no
Wants=systemd-fsckd.socket
Before=local-fs.target shutdown.target
After=systemd-fsckd.socket
ConditionPathIsReadWrite=!/
ConditionPathExists=!/run/initramfs/fsck-root

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/lib/systemd/systemd-fsck
TimeoutSec=0

https://manpages.debian.org/experimental/systemd/systemd-fsck-root.service.8.en.html

/run/initramfs/fsck-root would be created by initramfs running fsck: https://manpages.debian.org/jessie/initramfs-tools/initramfs-tools.8.en.html#FILES

On RPi, the initramfs seems to be not active by default or at least does not run fsck. Don't know much about how to use initramfs btw or where it is possible etc. This is the reason why RPi boot fsck strictly depends on fstab flag, why VM does not.

So what actually should work on every system, is the systemd-fsck-root, if not initramfs already checked and created /run/initramfs/fsck-root. The latter also seems to be not used any more on newest versions, but hmm on my Buster VM it is still created: https://github.com/systemd/systemd/commit/956eaf2b8d6c9999024705ddadc7393bc707de02

So to further investigate on other systems:

ls -al /run/initramfs/
systemctl status systemd-fsck-root
tune2fs -l /dev/sda1 | grep 'Last checked'

Maybe we can add the feature for x86_64 and RPi only first and add other SBCs as fast as we can find a way.

I would currently go this way, as long as there is no cleaner way via custom initramfs?

MichaIng commented 6 years ago

Generally about formatting:

Fourdee commented 6 years ago

tune2fs -C 2 -c 1

Odroid C2:

NanoPC T4:

Appears boot code forces a check on this device. Unsure if its repairing? I'll need to test another system.

root@DietPi:~# tune2fs -C 2 -c 1 /dev/sda
tune2fs 1.44.2 (14-May-2018)
Setting maximal mount count to 1
Setting current mount count to 2

#rootfs
root@DietPi:~# tune2fs -C 2 -c 1 /dev/mmcblk1p6
tune2fs 1.44.2 (14-May-2018)
Setting maximal mount count to 1
Setting current mount count to 2

#fstab
#Internal Drives---------------------------------------------------
UUID=ff313567-e9f1-5a5d-9895-3ba130b4a864 / auto rw,noatime 0 1
UUID=c976771f-1ec9-445b-949c-d6b2883e7860 /mnt/c976771f-1ec9-445b-949c-d6b2883e7860 auto rw,x-systemd.automount,nofail,noatime 0 2

#reboot

root@DietPi:~# tune2fs -l /dev/sda | grep 'Last checked'
Last checked:             Tue Jun 26 15:14:46 2018

root@DietPi:~# tune2fs -l /dev/mmcblk1p6 | grep 'Last checked'
Last checked:             Fri Jun 29 15:03:38 2018

#set fstab entries back to 0 0
#reboot

root@DietPi:~# tune2fs -l /dev/mmcblk1p6 | grep 'Last checked'
Last checked:             Fri Jun 29 15:04:59 2018

Interesting Lifetime writes looks useful, but only works since last formatted (as far as I can tell, not total lifetime) My Odroid C2:

root@DietPi:~#  tune2fs -l /dev/mmcblk0p2 | grep writes
Lifetime writes:          123 GB

Recently formatted RPi SD card (1 year old card, heavly used)

Lifetime writes:          8190 MB
MichaIng commented 6 years ago

@Fourdee

Interesting Lifetime writes looks useful, but only works since last formatted (as far as I can tell, not total lifetime)

  • Jep, seems to be related/reset on format/fs creation. Nearly same writes here 😄: 
    Filesystem created:       Wed Apr 18 03:07:02 2018
Lifetime writes:          8104 MB

    Did you check for /run/initramfs/fsck-root existence after reboot? As said this initramfs check doesn't show up on any log.

    ls -al /run/initramfs/
systemctl status systemd-fsck-root

    If this file doesn't exist, systemd-fsck-root should definitely kick in. It's a normal static unit that runs Before=local-fs.target and exist on every Raspbian + Debian systemd.... Ah found the reason: This service and systemd-fsck binary does NOT exist on Debian Jessie systemd version, just Stretch+: https://packages.debian.org/de/jessie/amd64/systemd/filelist Hmm as > /forcefsck on the other hand does not work any more on newer kernel, does in instead work on Odroids? As at least the file is removed as well on Stretch+, we could just create it anyway as fail safe.

    Okay, so I think on Stretch+ we can then rely on systemd-fsck. If I understand the man pages right, then the service itself understands fsck.mode=force and fsck.repair=yes from cmd line.

  • The first can be used as well of course to force fsck on next reboot, but as said I don't like that cmd line is located differently on every SBC. tune2fs seems to me the cleaner method then.
  • Default fsck.repair=preen repairs only what can be safely repaired. Looks actually as a good default to me. On RPi we already have fsck.repair=yes, I don't know if we should add this as default to all devices? Hmm if we anyway need to edit cmd lines, then we can also use it to force fsck on next reboot 🤔. Alternative would be to leave it as it is and show log on reboot to user (temporary bashrc.d script that also removes itself?): 
    2018-06-29 22:15:32 root@micha:~# echo "$(journalctl -t systemd-fsck)"
-- Logs begin at Thu 2018-06-28 01:05:01 CEST, end at Fri 2018-06-29 22:15:04 CEST. --
Jun 29 20:58:15 micha.gnoedi.org systemd-fsck[85]: e2fsck 1.44.2 (14-May-2018)
Jun 29 20:58:15 micha.gnoedi.org systemd-fsck[85]: rootfs: clean, 42911/3786016 files, 518504/15667200 blocks (check after next mount)
  • No idea if errors are logged here as well? There seems to be no other log location at least. Respectively, if fsck was performed by initramfs: 
    
root@VM-Stretch:~# cat /run/initramfs/fsck.log
Log of fsck -C -a -V -t ext4 /dev/sda1
Fri Jun 29 20:22:02 2018

fsck from util-linux 2.29.2 [/sbin/fsck.ext4 (1) -- /dev/sda1] fsck.ext4 -a -C0 /dev/sda1 /dev/sda1: clean, 68439/524288 files, 635991/2096640 blocks

Fri Jun 29 20:22:02 2018

**€: `fsck.mode` applies to all drives within fstab:**

journalctl -t systemd-fsck Jun 30 00:14:17 micha.gnoedi.org systemd-fsck[85]: e2fsck 1.44.2 (14-May-2018) Jun 30 00:14:17 micha.gnoedi.org systemd-fsck[85]: Pass 1: Checking inodes, blocks, and sizes Jun 30 00:14:56 micha.gnoedi.org systemd-fsck[85]: Pass 2: Checking directory structure Jun 30 00:14:57 micha.gnoedi.org systemd-fsck[85]: Pass 3: Checking directory connectivity Jun 30 00:14:57 micha.gnoedi.org systemd-fsck[85]: Pass 4: Checking reference counts Jun 30 00:14:59 micha.gnoedi.org systemd-fsck[85]: Pass 5: Checking group summary information Jun 30 00:15:01 micha.gnoedi.org systemd-fsck[85]: rootfs: 42590/3786016 files (0.2% non-contiguous), 518417/15667200 blocks Jun 30 00:15:05 micha.gnoedi.org systemd-fsck[214]: fsck.fat 4.1 (2017-01-24) Jun 30 00:15:05 micha.gnoedi.org systemd-fsck[214]: /dev/mmcblk0p1: 258 files, 46827/86872 clusters Jun 30 00:15:05 micha.gnoedi.org systemd-fsck[235]: e2fsck 1.44.2 (14-May-2018) Jun 30 00:15:19 micha.gnoedi.org systemd-fsck[235]: Pass 1: Checking inodes, blocks, and sizes Jun 30 00:16:22 micha.gnoedi.org systemd-fsck[235]: Pass 2: Checking directory structure Jun 30 00:16:46 micha.gnoedi.org systemd-fsck[235]: Pass 3: Checking directory connectivity Jun 30 00:17:00 micha.gnoedi.org systemd-fsck[235]: Pass 4: Checking reference counts Jun 30 00:17:59 micha.gnoedi.org systemd-fsck[235]: Pass 5: Checking group summary information Jun 30 00:18:06 micha.gnoedi.org systemd-fsck[235]: /dev/sda: 979279/183148544 files (0.4% non-contiguous), 68303272/732566646 blocks

Just with `tune2fs -c 1 /dev/mmcblk0p2`:

journalctl -t systemd-fsck Jun 30 00:57:29 micha.gnoedi.org systemd-fsck[87]: e2fsck 1.44.2 (14-May-2018) Jun 30 00:57:29 micha.gnoedi.org systemd-fsck[87]: rootfs has been mounted 1 times without being checked, check forced. Jun 30 00:57:29 micha.gnoedi.org systemd-fsck[87]: Pass 1: Checking inodes, blocks, and sizes Jun 30 00:58:07 micha.gnoedi.org systemd-fsck[87]: Pass 2: Checking directory structure Jun 30 00:58:09 micha.gnoedi.org systemd-fsck[87]: Pass 3: Checking directory connectivity Jun 30 00:58:09 micha.gnoedi.org systemd-fsck[87]: Pass 4: Checking reference counts Jun 30 00:58:10 micha.gnoedi.org systemd-fsck[87]: Pass 5: Checking group summary information Jun 30 00:58:13 micha.gnoedi.org systemd-fsck[87]: rootfs: 42596/3786016 files (0.2% non-contiguous), 518425/15667200 blocks Jun 30 00:58:15 micha.gnoedi.org systemd-fsck[200]: fsck.fat 4.1 (2017-01-24) Jun 30 00:58:15 micha.gnoedi.org systemd-fsck[200]: /dev/mmcblk0p1: 258 files, 46827/86872 clusters Jun 30 00:58:15 micha.gnoedi.org systemd-fsck[235]: e2fsck 1.44.2 (14-May-2018) Jun 30 00:58:28 micha.gnoedi.org systemd-fsck[235]: /dev/sda: clean, 979279/183148544 files, 68303272/732566646 blocks

- It seems to "touch" every disk, but just checks the ones, where mount count was reached. But as can be seen, this takes quite a lot of time, even on nearly empty 64 GB SDcard. Thus I agree this should not be done on every reboot. Default here is as well, that fsck is done, if flagged by kernel I/O error or unordered shutdown (powerloss).
________________________
Btw I found the reason why on RPi initramfs does no fsck:

2018-06-29 21:30:59 root@micha:~# cat /etc/default/raspberrypi-kernel

Defaults for raspberrypi-kernel

Uncomment the following line to enable generation of

/boot/initrd.img-KVER files (requires initramfs-tools)

INITRD=Yes

Uncomment the following line to enable generation of

/boot/initrd(7).img files (requires rpi-initramfs-tools)

RPI_INITRD=Yes


- Not sure if it is used somewhere else, but found this now: https://raspberrypi.stackexchange.com/a/26871
- Looks like indeed RPi does not need `initramfs-tools` unless you want to do something special before mounting root. As well the config file indicates that `rpi-initramfs-tools` might offer an alternative. But I can't find this package on any repo, nor via web search anything about it.
**€: Tested and works fine. `kmod` and `udev` need to be marked as manual then, all other packages can be safely removed, especially:**
  - `plymouth` for custom splash screen
  - `mountall` is deprecated anyway, not available on Debian/Raspbian repo since Stretch, but preinstalled on Raspbian Stretch... no dependency of anything, but marked as "important", thus not autoremoved. Looks like Raspbian images are just ported from one version to another instead of build up fresh each time. If not taken care about obsolete packages, once marked as important, they will survive from one image to the next. This can be seen as well for several versions of `gcc-x.y-base` installed on Raspbian. All but the most current can be securely purged.
  - `linux-base` seems to be used by linux-image-* package install scripts, but not by raspberrypi-firmware packages.
- Might it be the case for other SBCs with non mainline kernel as well?
pruperting commented 6 years ago

Just adding a note to say I experienced the same originally reported issue yesterday with an ODROID HC2 with the most recent download available on the website. I reported this in the forum via a post already about this (https://dietpi.com/phpbb/viewtopic.php?f=11&t=3908) but only found this bug report today.

I'm happy to help test fixes to sort this, but will need to read through this full post first.

Fourdee commented 6 years ago

@MichaIng

https://github.com/Fourdee/DietPi/commit/f91bf4b44856d37ecb4dd6f6ddcbcaea835d8e0e A few more fstab scrapes than i'd like, but this was the fastest and most efficient way to resolve RAMLOG ignored and /tmp issues.

MichaIng commented 6 years ago

Move RootFS on x86_64: https://unix.stackexchange.com/questions/5297/debian-grub2-moving-root-partition-to-new-drive

Fourdee commented 6 years ago

Note to self: I still need to do work on this:

etc.

Menu_Drive_Manager_Edit_Drive getting messy, near impossible to control cat/shown options, rewrite?

if (( has_fs && mounted && read_write )); then

show_benchmark=1

fi

Then

if (( $show_benchmark )); then

G_WHIP_MENU+=('Benchmark' 'xxx')

fi


Then calculate?
Fourdee commented 6 years ago

Ok as it stands, this is completed.

Minor issue with: https://github.com/Fourdee/DietPi/issues/1858#issuecomment-401970424, however, does not affect functionality, only code layout. If it bugs me enough, and I get time, i'll tweak it post v6.10 release.

Marking as closed.

butterbrot21 commented 6 years ago

sounds great! You guys are doing a great job!!! i'm really looking forward to the update/image :))