search

MichaIng / DietPi

Lightweight justice for your single-board computer!
https://dietpi.com/
GNU General Public License v2.0
4.88k stars 497 forks source link

Linux 5.15 | NFS with QNAP NAS server broken #5358

Open JaScoMa opened 2 years ago

JaScoMa commented 2 years ago

Creating a bug report/issue

Required Information

Additional Information (if applicable)

Steps to reproduce

  1. Fresh installation of DietPI
  2. 'apt upgrade' and install both kernel updates
  3. Reboot
  4. 'ls -l /mnt/nfs_NAS/Folder' displays nothing and just sits there.

Expected behaviour

Actual behaviour

Extra details

MichaIng commented 2 years ago

Many thanks for your report.

Did a fresh installation of DietPI. Then perform the kernel update.

Can you clarify this? When you boot a fresh DietPi image, it performs a full package upgrade automatically, hence a manual apt upgrade afterwards won't upgrade anything. Furthermore, at which point did you create the NFS mount, if it was a fresh DietPi image?

This shouldn't be related to the kernel upgrade. Does it work to mount it manually?

mount /mnt/nfs_NAS/Folder

And can you show automount and mount unit logs:

journalctl -u mnt-nfs_NAS-Folder.automount -u mnt-nfs_NAS-Folder.mount
JaScoMa commented 2 years ago

Hello.. Earlier today when I did a fresh installation, I can confirm that the the kernel number was different than what was listed above; it was a .63 number. Once I did the apt update/apt upgrade, it installed the two kernel updates and listing the mount quit working afterwards. The mount was working prior to the reboot as I had copied a backup to the NAS device and rebooted.

I just did a fresh dietpi installation; grabbed the eMMC, flashed the image of bullseye to it and started up the SBC. During the installation process, I installed Emby server; same as last time. Once it was done with this portion of the configuration, it auto-rebooted.

I then logged in, changed the local to en_US and the time zone to Eastern. And I changed the devices IP to the normal static IP; 192.168.1.5; afterwards I rebooted once again. Opened a new terminal window with the static IP and rebooted.

After this, logged in using the static IP and went into DietPI config and then configured the NFS mount for my NAS. Everything appeared to work properly. FSTAB entry shows:

192.168.1.4:/ /mnt/nfs_NAS nfs nofail,noauto,x-systemd.automount

When I do a 'mount' command, it shows:

192.168.1.4:/ on /mnt/nfs_NAS type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.1.5,local_lock=none,addr=192.168.1.4)

I can umount this mount without any issues. When I do a ls -l on the /mnt/nfs_NAS folder (after umount), it shows 0..

I can then run 'mount /mnt/nfs_NAS' and there aren't any errors. But if I do a ls -l /mnt/nfs_NAS it just sits there and shows nothing until I cancel.

Hope this helps.

uname -a shows:

Linux DietPi 5.15.25-rockchip64 #22.02.1 SMP PREEMPT Sun Feb 27 09:05:47 UTC 2022 aarch64 GNU/Linux
root@DietPi:~# journalctl -u mnt-nfs_NAS.automount -u mnt-nfs_NAS.mount
-- Journal begins at Tue 2022-03-15 23:35:17 EDT, ends at Tue 2022-03-15 23:36:07 EDT. --
Mar 15 23:35:18 DietPi systemd[1]: Set up automount mnt-nfs_NAS.automount.
Mar 15 23:36:06 DietPi systemd[1]: mnt-nfs_NAS.automount: Got automount request for /mnt/nfs_NAS, triggered by 1181 (ls)
Mar 15 23:36:06 DietPi systemd[1]: Mounting /mnt/nfs_NAS...
Mar 15 23:36:07 DietPi systemd[1]: Mounted /mnt/nfs_NAS.

Have found that after rebooting the machine, the mount is not being mounted; even through its listed within the /etc/fstab. I can manually run the mount command without any issues. But the LS still just sits there.

JaScoMa commented 2 years ago

Just a quick update. I created new shares via SAMBA and didn't have any issues creating the shares. Afterwards, was able to access the shares without any issues. To ensure everything is working properly, rebooted the device. After logging in was able to access the mount shares without any issues.

Appears as though NFS isn't working properly.

Joulinar commented 2 years ago

can you share following

df -h
showmount -e <IP_of_NAS_server>
cat /etc/fstab
MichaIng commented 2 years ago

Have found that after rebooting the machine, the mount is not being mounted; even through its listed within the /etc/fstab. I can manually run the mount command without any issues. But the LS still just sits there.

It is managed by systemd automount, i.e. it is not mounted automatically on boot but as fast as anything is accessing the mount point, like your ls command. If the mount fails, then systemd automount hangs the input for up to 90 seconds timeout, a bit annoying, but it means the drive/share is not available/cannot be reached. Strange is that you can manually mount the drive, because this should fail just the same way systemd fails to mount it. So indeed it would be interesting to know whether the drive has really been mounted, checking it via df as suggested by Joulinar.

The other theory would be that the mount worked well but it is accessing the individual directory that fails for some reason, like a permissions- or other server config issue.

JaScoMa commented 2 years ago

can you share following

df -h
showmount -e <IP_of_NAS_server>
cat /etc/fstab

Here's for the NFS configuration:

root@DietPi:/mnt/nfs_NAS# df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            1.9G     0  1.9G   0% /dev
tmpfs           387M   11M  376M   3% /run
/dev/mmcblk2p1  115G  821M  113G   1% /
tmpfs           1.9G     0  1.9G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           1.9G  4.0K  1.9G   1% /tmp
tmpfs            50M  8.0K   50M   1% /var/log
192.168.1.4:/    16M     0   16M   0% /mnt/nfs_NAS
root@DietPi:/mnt/nfs_NAS# showmount -e 192.168.1.4
Export list for 192.168.1.4:
/MediaServer 192.168.1.5
/EmbyUpdates 192.168.1.5
root@DietPi:/mnt/nfs_NAS# cat /etc/fstab
# You can use "dietpi-drive_manager" to setup mounts.
# NB: It overwrites and re-creates physical drive mount entries on use.
#----------------------------------------------------------------
# NETWORK
#----------------------------------------------------------------
192.168.1.4:/ /mnt/nfs_NAS nfs nofail,noauto,x-systemd.automount

#----------------------------------------------------------------
# TMPFS
#----------------------------------------------------------------
tmpfs /tmp tmpfs size=1931M,noatime,lazytime,nodev,nosuid,mode=1777
tmpfs /var/log tmpfs size=50M,noatime,lazytime,nodev,nosuid,mode=1777

#----------------------------------------------------------------
# MISC: ecryptfs, vboxsf, glusterfs, bind, Btrfs subvolume mounts
#----------------------------------------------------------------

#----------------------------------------------------------------
# SWAP SPACE
#----------------------------------------------------------------

#----------------------------------------------------------------
# PHYSICAL DRIVES
#----------------------------------------------------------------
UUID=16fd56b1-1af3-4832-b584-e15c71d16b10 / ext4 noatime,lazytime,rw 0 1
JaScoMa commented 2 years ago

can you share following

df -h
showmount -e <IP_of_NAS_server>
cat /etc/fstab

And for reference, SAMBA:

root@DietPi:~# df -h
Filesystem                 Size  Used Avail Use% Mounted on
udev                       1.9G     0  1.9G   0% /dev
tmpfs                      387M  5.7M  381M   2% /run
/dev/mmcblk2p1             115G  822M  113G   1% /
tmpfs                      1.9G     0  1.9G   0% /dev/shm
tmpfs                      5.0M     0  5.0M   0% /run/lock
tmpfs                      1.9G  4.0K  1.9G   1% /tmp
tmpfs                       50M  8.0K   50M   1% /var/log
//192.168.1.4/MediaServer   11T  6.3T  4.4T  60% /mnt/smb_QNAP/MediaServer
//192.168.1.4/EmbyUpdates   11T  6.3T  4.4T  60% /mnt/smb_QNAP/EmbyUpdates
root@DietPi:~# showmount -e 192.168.1.4
Export list for 192.168.1.4:
/MediaServer 192.168.1.5
/EmbyUpdates 192.168.1.5
root@DietPi:~# cat /etc/fstab
# You can use "dietpi-drive_manager" to setup mounts.
# NB: It overwrites and re-creates physical drive mount entries on use.
#----------------------------------------------------------------
# NETWORK
#----------------------------------------------------------------
//192.168.1.4/MediaServer /mnt/smb_QNAP/MediaServer cifs cred=/var/lib/dietpi/dietpi-drive_manager/mnt-smb_QNAP-MediaServer.cred,iocharset=utf8,uid=dietpi,gid=dietpi,file_mode=0770,dir_mode=0770,vers=3.1.1,nofail,noauto,x-systemd.automount
//192.168.1.4/EmbyUpdates /mnt/smb_QNAP/EmbyUpdates cifs cred=/var/lib/dietpi/dietpi-drive_manager/mnt-smb_QNAP-EmbyUpdates.cred,iocharset=utf8,uid=dietpi,gid=dietpi,file_mode=0770,dir_mode=0770,vers=3.1.1,nofail,noauto,x-systemd.automount

#----------------------------------------------------------------
# TMPFS
#----------------------------------------------------------------
tmpfs /tmp tmpfs size=1931M,noatime,lazytime,nodev,nosuid,mode=1777
tmpfs /var/log tmpfs size=50M,noatime,lazytime,nodev,nosuid,mode=1777

#----------------------------------------------------------------
# MISC: ecryptfs, vboxsf, glusterfs, bind, Btrfs subvolume mounts
#----------------------------------------------------------------

#----------------------------------------------------------------
# SWAP SPACE
#----------------------------------------------------------------

#----------------------------------------------------------------
# PHYSICAL DRIVES
#----------------------------------------------------------------
UUID=16fd56b1-1af3-4832-b584-e15c71d16b10 / ext4 noatime,lazytime,rw 0 1
Joulinar commented 2 years ago

ok the mount definition inside /etc/fstab doesn't seems to be correct. Unfortunately a downside of dietpi-drive_manager atm

Currently you try to mount from root folder

192.168.1.4:/    16M     0   16M   0% /mnt/nfs_NAS

and

192.168.1.4:/ /mnt/nfs_NAS

But your NFS server is exporting 2 sub folder

Export list for 192.168.1.4:
/MediaServer 192.168.1.5
/EmbyUpdates 192.168.1.5

Therefore you would need to update your /etc/fstab manually to be similar to your SMB mounts

192.168.1.4:/MediaServer /mnt/nfs_NAS/MediaServer nfs nofail,noauto,x-systemd.automount
192.168.1.4:/EmbyUpdates /mnt/nfs_NAS/EmbyUpdates nfs nofail,noauto,x-systemd.automount
MichaIng commented 2 years ago

But strange that it worked before then 🤔. I'll implement the NFS export scan now, long overdue.

MichaIng commented 2 years ago

PR up to allow selection from detected exports: #5363

Let's see whether this really solves your issue. What I found on this topic is that only NFSv3 strictly requires a path. NFSv4 server exports can be mounted without path, but as long as there is no export defined as root via "fsid=0", you'll have the full rootfs directory structure at the mount point where you need to (annoyingly) navigate to the actual share directory to see the intended content, but otherwise it works.

JaScoMa commented 2 years ago

PR up to allow selection from detected exports: #5363

Let's see whether this really solves your issue. What I found on this topic is that only NFSv3 strictly requires a path. NFSv4 server exports can be mounted without path, but as long as there is no export defined as root via "fsid=0", you'll have the full rootfs directory structure at the mount point where you need to (annoyingly) navigate to the actual share directory to see the intended content, but otherwise it works.

Hello.. Can confirm that editing the fstab with the two nfs entries did not work. When I went to the /mnt/nfs_NAS folder and did a ls, it just ended up sitting there doing nothing.

And that appears true with nfsv4; which is running on my NAS. During the setup process, Dietpi only asks for the IP of the NAS and whatever directories I have configured for NFS on the devices, show up within the mount point. I like NFS better as it seemed to have less overhead than smb, seemed faster in transfer speeds, and a lot easier to configured permissions wise.

Thank you..

Joulinar commented 2 years ago

can you share your adjusted /etc/fstab

JaScoMa commented 2 years ago

can you share your adjusted /etc/fstab

Here you go. Thank you.

root@RockPRO64:~# cat /etc/fstab
# You can use "dietpi-drive_manager" to setup mounts.
# NB: It overwrites and re-creates physical drive mount entries on use.
#----------------------------------------------------------------
# NETWORK
#----------------------------------------------------------------
192.168.1.4:/MediaServer /mnt/nfs_NAS/MediaServer nfs nofail,noauto,x-systemd.automount
192.168.1.4:/EmbyUpdates /mnt/nfs_NAS/EmbyUpdates nfs nofail,noauto,x-systemd.automount

#----------------------------------------------------------------
# TMPFS
#----------------------------------------------------------------
tmpfs /tmp tmpfs size=1931M,noatime,lazytime,nodev,nosuid,mode=1777
tmpfs /var/log tmpfs size=50M,noatime,lazytime,nodev,nosuid,mode=1777

#----------------------------------------------------------------
# MISC: ecryptfs, vboxsf, glusterfs, bind, Btrfs subvolume mounts
#----------------------------------------------------------------

#----------------------------------------------------------------
# SWAP SPACE
#----------------------------------------------------------------

#----------------------------------------------------------------
# PHYSICAL DRIVES
#----------------------------------------------------------------
UUID=16fd56b1-1af3-4832-b584-e15c71d16b10 / ext4 noatime,lazytime,rw 0 1
Joulinar commented 2 years ago

the 2 directories /mnt/nfs_NAS/EmbyUpdates as well as /mnt/nfs_NAS/MediaServer already exist? If not, create them first.

JaScoMa commented 2 years ago

Created the nfs_NAS directory and the sub-directories and same thing. Trying to do a LS on the /mnt/nfs_NAS directory results in things just sitting there until I cancel.

Joulinar commented 2 years ago

can you try umount /mnt/nfs_NAS/. Probably still something mounted to the directory. A reboot should do it as well.

JaScoMa commented 2 years ago

can you try umount /mnt/nfs_NAS/. Probably still something mounted to the directory. A reboot should do it as well.

Hello.. I did that as well as remove all entries from the fstab and rebooted after each time. I also rebooted every time I made a change. No change.

Thank you..

Joulinar commented 2 years ago

can you share the output of mount command

MichaIng commented 2 years ago

When all related fstab entries are removed and you rebooted (so that there is no systemd automount unit etc), try to do a manual mount and paste the output in case:

mkdir -p /mnt/nfs_NAS/MediaServer
mount -t nfs '192.168.1.4:/MediaServer' /mnt/nfs_NAS/MediaServer
ls -l /mnt/nfs_NAS/MediaServer
dmesg | tail -10
JaScoMa commented 2 years ago

can you share the output of mount command

root@RockPRO64:~# mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=1905104k,nr_inodes=476276,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,noexec,relatime,size=395668k,mode=755)
/dev/mmcblk2p1 on / type ext4 (rw,noatime,lazytime)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime)
tracefs on /sys/kernel/tracing type tracefs (rw,nosuid,nodev,noexec,relatime)
sunrpc on /run/rpc_pipefs type rpc_pipefs (rw,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
systemd-1 on /mnt/nfs_NAS/EmbyUpdates type autofs (rw,relatime,fd=52,pgrp=1,timeout=0,minproto=5,maxproto=5,direct)
systemd-1 on /mnt/nfs_NAS/MediaServer type autofs (rw,relatime,fd=53,pgrp=1,timeout=0,minproto=5,maxproto=5,direct)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noatime,lazytime,size=1977344k)
tmpfs on /var/log type tmpfs (rw,nosuid,nodev,noatime,lazytime,size=51200k)
JaScoMa commented 2 years ago

When all related fstab entries are removed and you rebooted (so that there is no systemd automount unit etc), try to do a manual mount and paste the output in case:

mkdir -p /mnt/nfs_NAS/MediaServer
mount -t nfs '192.168.1.4:/MediaServer' /mnt/nfs_NAS/MediaServer
ls -l /mnt/nfs_NAS/MediaServer
dmesg | tail -10

mkdir works when I try the manual mount, it just sits there and does nothing until I cancel. ls displays an empty folder. dmesg entry listed below:

root@RockPRO64:/mnt/nfs_NAS/MediaServer# dmesg | tail -10
[    7.101356] rk_gmac-dwmac fe300000.ethernet eth0: PTP not supported by HW
[    7.102612] rk_gmac-dwmac fe300000.ethernet eth0: configuring for phy/rgmii link mode
[    8.393244] random: crng init done
[    8.393254] random: 7 urandom warning(s) missed due to ratelimiting
[   11.234937] rk_gmac-dwmac fe300000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off
[  227.146896] FS-Cache: Loaded
[  227.232070] FS-Cache: Netfs 'nfs' registered for caching
[  227.585413] NFS: Registering the id_resolver key type
[  227.585446] Key type id_resolver registered
[  227.585448] Key type id_legacy registered
MichaIng commented 2 years ago

when I try the manual mount, it just sits there and does nothing until I cancel.

How long did you wait before cancelling it?

So then the mount itself fails, like if the server was not reachable or the connection blocked somewhere. Did you have a look into the NFS server logs?

JaScoMa commented 2 years ago

Hello.. The manual mount has been running for the last 20 minute without anything.

It's a QNAP NAS and I didn't have any issues with NFS prior to the latest .25 kernel update; from .63. Everything was working, then I read about the Dirty Pipe issue and performed the APT update, etc. After rebooting the system, NFS quit working.

So I did some searching on QNAPs website about turning on logging for NFS and found in a forum post concerning logging of NFS: For NFS, only mounts are made visible - no file access, no dismount - as NFS is a stateless protocol, served mostly on the Kernel level.

So if its true that NFS is served mostly on the Kernel level, then it makes sense that something was broke concerning the update to the latest .25 kernel. As the famous saying goes, It use to work.

The next step I'm going to try is install the NFS client on Windows 10 and see if the mount works properly and if I can see the shares, etc. EDIT: I only have Windows 10 home, and NFS is only available for Professional; so no dice.

MichaIng commented 2 years ago

You can also downgrade the kernel:

apt install --reinstall --allow-downgrades 'linux-image-current-rockchip64=21.08.2' 'linux-dtb-current-rockchip64=21.08.2'

If this indeed helps, set it on hold:

apt-mark hold linux-image-current-rockchip64 linux-dtb-current-rockchip64

We can then report it to Armbian as bug in the current kernel. I just checked their forum, but couldn't find a related report yet: https://forum.armbian.com/forum/33-rockchip-3399/

JaScoMa commented 2 years ago

You can also downgrade the kernel:

apt install --reinstall --allow-downgrades 'linux-image-current-rockchip64=21.08.2' 'linux-dtb-current-rockchip64=21.08.2'

If this indeed helps, set it on hold:

apt-mark hold linux-image-current-rockchip64 linux-dtb-current-rockchip64

We can then report it to Armbian as bug in the current kernel. I just checked their forum, but couldn't find a related report yet: https://forum.armbian.com/forum/33-rockchip-3399/

Hello.. Rolling back to the previous kernel worked. I'm able to reference the drive without any issues. For reference:

root@RockPRO64:~# uname -a
Linux RockPRO64 5.10.63-rockchip64 #21.08.2 SMP PREEMPT Wed Sep 8 10:57:23 UTC 2021 aarch64 GNU/Linux
root@RockPRO64:/mnt/nfs_NAS/MediaServer# df -h
Filesystem                Size  Used Avail Use% Mounted on
udev                      1.9G     0  1.9G   0% /dev
tmpfs                     387M  5.7M  381M   2% /run
/dev/mmcblk2p1            115G  2.1G  112G   2% /
tmpfs                     1.9G     0  1.9G   0% /dev/shm
tmpfs                     5.0M     0  5.0M   0% /run/lock
tmpfs                     1.9G  4.0K  1.9G   1% /tmp
tmpfs                      50M  8.0K   50M   1% /var/log
192.168.1.4:/              16M     0   16M   0% /mnt/nfs_NAS
192.168.1.4:/MediaServer   11T  6.2T  4.4T  59% /mnt/nfs_NAS/MediaServer
192.168.1.4:/EmbyUpdates   11T  6.2T  4.4T  59% /mnt/nfs_NAS/EmbyUpdates
root@RockPRO64:/mnt/nfs_NAS/MediaServer# showmount -e 192.168.1.4
Export list for 192.168.1.4:
/MediaServer 192.168.1.5
/EmbyUpdates 192.168.1.5
root@RockPRO64:/mnt/nfs_NAS/MediaServer# cat /etc/fstab
# You can use "dietpi-drive_manager" to setup mounts.
# NB: It overwrites and re-creates physical drive mount entries on use.
#----------------------------------------------------------------
# NETWORK
#----------------------------------------------------------------
192.168.1.4:/ /mnt/nfs_NAS nfs nofail,noauto,x-systemd.automount

#----------------------------------------------------------------
# TMPFS
#----------------------------------------------------------------
tmpfs /tmp tmpfs size=1931M,noatime,lazytime,nodev,nosuid,mode=1777
tmpfs /var/log tmpfs size=50M,noatime,lazytime,nodev,nosuid,mode=1777

#----------------------------------------------------------------
# MISC: ecryptfs, vboxsf, glusterfs, bind, Btrfs subvolume mounts
#----------------------------------------------------------------

#----------------------------------------------------------------
# SWAP SPACE
#----------------------------------------------------------------

#----------------------------------------------------------------
# PHYSICAL DRIVES
#----------------------------------------------------------------
UUID=16fd56b1-1af3-4832-b584-e15c71d16b10 / ext4 noatime,lazytime,rw 0 1
MichaIng commented 2 years ago

Okay. I'll try to replicate with NanoPi R4S (same SoC), also with plain Armbian image, an in case report the bug. Important to test with an original Armbian image first, since they are strict regarding bug reports and support in general.

Joulinar commented 2 years ago

another issue with the latest Armbian kernel. Looks like USB not working correctly https://dietpi.com/phpbb/viewtopic.php?t=10195

MichaIng commented 2 years ago

Hmm, NFS mounting works very well here on NanoPi R4S (same SoC). @JaScoMa can you retry with latest DietPi release and the now implemented NFS scan done by dietpi-drive_manager?

JaScoMa commented 2 years ago

Hello.. Downloaded the latest DietPI version from the website and did a fresh installation. Can confirm that the issue still occurs with kernel Linux RockPRO64 5.15.25-rockchip64 #22.02.1, but roll-back to kernel Linux RockPRO64 5.10.63-rockchip64 #21.08.2, everything works fine.

During the process, I copied the text from the terminal window and pasted into the attached text files. Hope this helps.

Fresh_DietPI_Install.txt

Joulinar commented 2 years ago

Your NFS server is using NFS v3 or NFS v4? At least you could pin the kernel to avoid the automatically update while runningdietpi-update

JaScoMa commented 2 years ago

Your NFS server is using NFS v3 or NFS v4? At least you could pin the kernel to avoid the automatically update while runningdietpi-update

Hello.. Confirmed that only NFS v4 is enabled on the NAS device.

Yes, I've ran ;apt-mark hold linux-image-current-rockchip64 linux-dtb-current-rockchip64; to keep the system from updating the kernel. Thank you.

MichaIng commented 2 years ago

Okay, so with the new kernel, showmount shows the exports as expected, but mount fails (which is why also the fstab entries are not added), right?

I'm wondering what the difference is to my NanoPi R4S, which uses the same kernel, but mounting NFS works fine. This is not something that is related to the exact SBC/hardware, but a pure kernel feature, so the only difference I can think of is the NFS server.

I used another DietPi system as NFS server. You use a QNAP NAS. Is there a way to get the exact /etc/exports content for those two NFS shares from the QNAP, so that I can try to apply the exact same on my DietPi NFS server? Also do you know which NFS server version it is? I do not mean the protocol version (4.x or what), but e.g. on Debian Bullseye it's the nfs-kernel-server version 1.3.4: https://packages.debian.org/bullseye/nfs-kernel-server That has not changed for a long time (same on Stretch and Buster), but on Bookworm e.g. it's suddently v2.6.1 (what a jump!). Probably that one is closer to the QNAP NFS server version.

Joulinar commented 2 years ago

@MichaIng we have the user inside the forum having issues to mount NFS vor from Qnap as well. But the user is running RPI however it failing as well on latest version. Solution there was to downgrade RPI kernel or to specify NFS version on the mount.

MichaIng commented 2 years ago

Jep, I remember. Seems suspiciously similar indeed. So it looks like something between Linux v5.10 and v5.15 changed which affects QNAP NFS mounts in particular: https://dietpi.com/phpbb/viewtopic.php?t=10238

So another test with current kernel, to verify that both issues are the same, would be add nfsvers=4 as mount option. In /etc/fstab:

192.168.1.4:/MediaServer /mnt/nfs_QNAP/MediaServer nfs nofail,noauto,x-systemd.automount,nfsvers=4
192.168.1.4:/EmbyUpdates /mnt/nfs_QNAP/EmbyUpdates nfs nofail,noauto,x-systemd.automount,nfsvers=4

Or for a quick manual mount test:

mount -t nfs -o 'nfsvers=4' '192.168.1.4:/MediaServer' '/mnt/nfs_QNAP/MediaServer'
JaScoMa commented 2 years ago

Hello.. Can confirm that adding the ",nfsvers=4" at the end of the entry of my fstab worked. It's working properly once again, with the latest kernel.

I first added the ",nfsver=4" within my fstab and rebooted; drives mapped properly after reboot. I then marked unhold the kernel updates; "apt-mark unhold linux-image-current-rockchip64 linux-dtb-current-rockchip64" Then performed an "apt update" which downloaded the latest kernel updates and then I rebooted.

After reboot was able to list the mount points and see everything correctly. To confirm, performed a media library scan with Emby and this completed without any issues.

Thank you again for all your help.

putty.log

Joulinar commented 2 years ago

I did some test with a Bookworm VM acting as NFS server and a armv7 RPi3B+ as client. For me this was working well on NFS 4.2. 

root@DietPi3:~# mount -t nfs 192.168.0.18:/mnt/dietpi_userdata /mnt/nfs_client
root@DietPi3:~# nfsstat -m
/mnt/nfs_client from 192.168.0.18:/mnt/dietpi_userdata
 Flags: rw,relatime,vers=4.2,rsize=262144,wsize=262144,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.0.12,local_lock=none,addr=192.168.0.18
root@DietPi3:~# uname -a
Linux DietPi3 5.15.32-v7+ #1538 SMP Thu Mar 31 19:38:48 BST 2022 armv7l GNU/Linux
root@DietPi3:~# df -h
Filesystem                         Size  Used Avail Use% Mounted on
/dev/root                           15G  2.9G   11G  22% /
devtmpfs                           455M     0  455M   0% /dev
tmpfs                              487M     0  487M   0% /dev/shm
tmpfs                              195M  3.0M  192M   2% /run
tmpfs                              5.0M     0  5.0M   0% /run/lock
tmpfs                              1.0G     0  1.0G   0% /tmp
tmpfs                               50M     0   50M   0% /var/log
/dev/sda1                          127M   51M   76M  41% /boot
192.168.0.18:/mnt/dietpi_userdata  9.9G  769M  8.7G   8% /mnt/nfs_client
root@DietPi3:~# ls -la /mnt/nfs_client/
total 24
drwxrwxr-x 6 dietpi dietpi 4096 Jan  7 15:18 .
drwxr-xr-x 7 root   root   4096 Apr 11 18:52 ..
drwxrwxr-x 2 dietpi dietpi 4096 Jan  7 15:18 Music
drwxrwxr-x 2 dietpi dietpi 4096 Jan  7 15:18 Pictures
drwxrwxr-x 2 dietpi dietpi 4096 Jan  7 15:18 Video
drwxrwxr-x 2 dietpi dietpi 4096 Jan  7 15:18 downloads
MichaIng commented 2 years ago

So it looks like it is actually an issue with the particular QNAP NFS server in combination with Linux 5.15 on the client. It works well on all out tests, and it does not work with two Linux 5.15 client systems, but very different SoC and kernel build (RPi vs Armbian RK3399), with QNAP being the only thing in common.

Joulinar commented 2 years ago

one more https://dietpi.com/phpbb/viewtopic.php?t=10281

JaScoMa commented 2 years ago

Hello.. Noticed there was a new firmware released today, and I updated to the latest version. The workaround (,nfsvers=4) from the previous firmware still works with this latest update.

Will try re-moving then re-adding the NFS share without the ,nfsvers=4 and see if it works properly.

Thank you, Jeff

JaScoMa commented 2 years ago

OK, same issue as before; when re-adding a new NFS share with the QNAP NAS, it just sits there and doesn't do anything.

After cancelling, checked and the mnt directory is created but that is all; no entries in the fstab.

Manually adding the entries into the fstab with the ",nfsvers=4", remounts the shares without any issues.

Joulinar commented 2 years ago

Noticed there was a new firmware released today

you mean the new DietPi version 8.5? This is not a firmware. It's an update of our DietPi bash scripts (only). Not sure what the plan of @MichaIng is and if he like to address this from DietPi side. Actually it's an issue dedicated to QNAP NAS only.