fuse_internal_do_getattr: cache incoherent on /usr/local! Buggy FUSE server detected.

[grahamperrin]

Observed during a selective upgrade of a set of packages (https://github.com/nomadbsd/NomadBSD/issues/118#issuecomment-1253290764) – the lines after 08:00:

Sep 21 07:35:20 NomadBSD su[2875]: nomad to root on /dev/pts/1
Sep 21 07:36:01 NomadBSD kernel: pid 2880 (pkg), jid 0, uid 0, was killed: out of swap space
Sep 21 07:36:47 NomadBSD pkg[2972]: htop-3.2.1 installed
Sep 21 07:37:46 NomadBSD kernel: pid 3005 (pkg), jid 0, uid 0, was killed: out of swap space
Sep 21 07:56:04 NomadBSD kernel: pid 4239 (pkg), jid 0, uid 0, was killed: out of swap space
Sep 21 08:00:05 NomadBSD pkg[4487]: libxcb upgraded: 1.14_1 -> 1.15 
Sep 21 08:00:07 NomadBSD pkg[4487]: mpdecimal-2.5.1 installed
Sep 21 08:00:08 NomadBSD pkg[4487]: libffi upgraded: 3.3_1 -> 3.4.2 
Sep 21 08:00:09 NomadBSD pkg[4487]: readline upgraded: 8.1.0 -> 8.1.2 
Sep 21 08:00:10 NomadBSD pkg[4487]: pciids upgraded: 20210223 -> 20220518 
Sep 21 08:01:23 NomadBSD kernel: fuse_internal_do_getattr: cache incoherent on /usr/local!  Buggy FUSE server detected.  To prevent data corruption, disable the data cache by mounting with -o direct_io, or as directed otherwise by your FUSE server's documentation
Sep 21 08:01:40 NomadBSD pkg[4487]: libX11 upgraded: 1.6.12,1 -> 1.7.2,1 
Sep 21 08:01:40 NomadBSD pkg[4487]: mesa-libs-20.2.3 deinstalled
Sep 21 08:02:56 NomadBSD kernel: fuse_internal_do_getattr: cache incoherent on /usr/local!  Buggy FUSE server detected.  To prevent data corruption, disable the data cache by mounting with -o direct_io, or as directed otherwise by your FUSE server's documentation

The relevant line, wrapped:

Sep 21 08:01:23 NomadBSD kernel: fuse_internal_do_getattr: cache incoherent on /usr/local! Buggy FUSE server detected. To prevent data corruption, disable the data cache by mounting with -o direct_io, or as directed otherwise by your FUSE server's documentation

[Marietto2008]

I'm getting the same error,but no one replied to this,yet ? that's so bad since it is a severe problem,because,at least in my case,nomadbsd stops working totally and I should reboot. AND,any modification did when the error occurs will not be saved.

[mrclksr]

The best solution is to use a newer version of unionfs-fuse which builds with libfuse3. Our fork uses libfuse2. The problem is that the port sysutils/fusefs-libs3 does not provide the static library which is required to run unionfs without /usr/local. The port needs just a little patch (I'm looking at you, @grahamperrin :-))

diff --git a/sysutils/fusefs-libs3/Makefile b/sysutils/fusefs-libs3/Makefile
index dbe8f4073e4a..843557b5c010 100644
--- a/sysutils/fusefs-libs3/Makefile
+++ b/sysutils/fusefs-libs3/Makefile
@@ -1,5 +1,6 @@
 PORTNAME=  fusefs-libs3
 DISTVERSION=   3.12.0
+PORTREVISION=  1
 CATEGORIES=    sysutils
 MASTER_SITES=  https://github.com/libfuse/libfuse/releases/download/fuse-${DISTVERSION}/
 DISTNAME=  fuse-${DISTVERSION}
@@ -14,7 +15,8 @@ LICENSE_FILE= ${WRKSRC}/LICENSE

 USES=      compiler iconv meson tar:xz
 USE_LDCONFIG=  yes
-
+MESON_ARGS+=   --default-library \
+       both
 LDFLAGS+=  -L${LOCALBASE}/lib ${ICONV_LIB}

 PLIST_SUB= VERSION=${PORTVERSION}
@@ -25,7 +27,7 @@ PLIST_SUB=    VERSION=${PORTVERSION}
 EXTRA_PATCHES= ${FILESDIR}/extra-patch-test_meson.build
 .endif
 .if ${OPSYS} == DragonFly
-IGNORE=    fusefs has not been implemented on DragonFly
+IGNORE=        fusefs has not been implemented on DragonFly
 .endif

 .if ${CHOSEN_COMPILER_TYPE} == gcc
diff --git a/sysutils/fusefs-libs3/pkg-plist b/sysutils/fusefs-libs3/pkg-plist
index b466b066262d..7d3e1516d1f4 100644
--- a/sysutils/fusefs-libs3/pkg-plist
+++ b/sysutils/fusefs-libs3/pkg-plist
@@ -4,6 +4,7 @@ include/fuse3/fuse_common.h
 include/fuse3/fuse_log.h
 include/fuse3/fuse_lowlevel.h
 include/fuse3/fuse_opt.h
+lib/libfuse3.a
 lib/libfuse3.so
 lib/libfuse3.so.3
 lib/libfuse3.so.%%VERSION%%

[Marietto2008]

I realized that this bug even prevents my FreeBSD to load correctly the nvidia driver. Infact when I add this parameter "kld_list="fusefs" the nvidia driver won't load. Don't know why.

[grahamperrin]

… The port needs just a little patch (I'm looking at you, @grahamperrin :-)) …

Better make a bug report in Bugzilla.

The column to the right at https://reviews.freebsd.org/p/grahamperrin/ lists memberships in alphabetical order, maybe misleading. I'm primarily a committer to the documentation tree (separate from the ports tree).

There's a link out to https://www.freebsd.org/news/newsflash/#2022-06-01:1, which says (doc), but that's probably not clear enough.

I'll edit my profile, in the meantime there's a link to my wiki page, which makes things clearer. Thanks.