Unable to load KBFS Fuse kext on MacOS Big Sur (Intel) bootup.

[Danrancan]

I am getting the following error when starting up my Intel Mac running Big Sur 11.7.10.

We were unable to load KBFS (Fuse kext). This may be due to a limitation in MacOS where there aren't any device slots available. Device slots can be taken up by apps such as VMWare, VirtualBox, anti-virus programs, VPN programs and Intel HAXM.

Please run `keybase log send` to report the error.

My Log ID is: 85cf2e9c2938299ed0c5f31c

Please help me fix this.

[GwynethLlewelyn]

Well... unfortunately, there is no easy way to fix that.

There is a very kludgy workaround which I'm using myself everyday (see #25830), since I really really really needed to get Finder to work seamlessly with the Keybase FileSystem again, and was willing to do anything except recompiling the Keybase client from scratch (mostly because I don't have Xcode installed, and don't wish to).

The "trick" is simple: use a non-Keybase macFUSE (which works flawlessly) and force Keybase to use it. Keybase actually has an option to use a non-Keybase macFUSE — it's just that the way the client app was designed, it forces Keybase's own broken kext to load. And the Keybase devs did lots of small little tricks to guarantee that the configuration that loads the kext is always regenerated from scratch — and gives no option but to use Keybase's own fork, functional or not.

What my very long process does is simply to make the AgentLauncher file for the non-Keybase macFUSE unable to be overridden/overwritten/deleted/appended or changed in any way, and get the non-Keybase macFUSE kext launched no matter what, as soon as I log in with my user. Keybase's own mechanism, when launching in GUI mode, will try to ignore whatever ¨kexts` are running, but simply attempt to overwrite any AgentLauncher configuration it finds.

By marking my own configuration as immutable, Keybase cannot override it. It will give an error about not being able to install it's own fork. But it will continue to work, and that was my (allegedly simple) way of pushing Keybase to do what I wanted instead, without the need to recompile anything.

[spoike]

I can confirm I get this message now when I recently updated to Sonoma 14.2.

Log ID: 3e91b3025f8be070cb40721c

[Danrancan]

Well... unfortunately, there is no easy way to fix that.

There is a very kludgy workaround which I'm using myself everyday (see #25830), since I really really really needed to get Finder to work seamlessly with the Keybase FileSystem again, and was willing to do anything except recompiling the Keybase client from scratch (mostly because I don't have Xcode installed, and don't wish to).

The "trick" is simple: use a non-Keybase macFUSE (which works flawlessly) and force Keybase to use it. Keybase actually has an option to use a non-Keybase macFUSE — it's just that the way the client app was designed, it forces Keybase's own broken kext to load. And the Keybase devs did lots of small little tricks to guarantee that the configuration that loads the kext is always regenerated from scratch — and gives no option but to use Keybase's own fork, functional or not.

What my very long process does is simply to make the AgentLauncher file for the non-Keybase macFUSE unable to be overridden/overwritten/deleted/appended or changed in any way, and get the non-Keybase macFUSE kext launched no matter what, as soon as I log in with my user. Keybase's own mechanism, when launching in GUI mode, will try to ignore whatever ¨kexts` are running, but simply attempt to overwrite any AgentLauncher configuration it finds.

By marking my own configuration as immutable, Keybase cannot override it. It will give an error about not being able to install it's own fork. But it will continue to work, and that was my (allegedly simple) way of pushing Keybase to do what I wanted instead, without the need to recompile anything.

I read your long tutorial all the way through and really appreciate you taking the time to write it! But I feel that this is just a bit too hacky and risky for me as I have updated to MacOS Sonoma, and this hack seems untested by other users on it. That being said, how hard would it be to compile the code from source with the hack modified into the source code? Im not very familiar with modifying source code so would probably seek outside help.

What are the chances you might be willing to write up a tutorial for building keybase from source code with this hack included, or just making a keybase fork that is easily compilable and just works?

Just a thought since you seem like a very knowledgeable lady!

Regardless, thanks for all of your help and posts!

Dan

P.S. Your Second Life character is really hot! Lol

[GwynethLlewelyn]

I thank for your totally misdirected but well-intended compliment :)

No, I'm not really "knowledgeable" — I'm not even a professional programmer — I just happen to dig as deep as I can into things that I really like until I figure out how they work :) Sadly, I keep getting distracted with things outside my alleged sphere of work — such as Keybase, which I (still) find a fascinating solution.

Forking Keybase's open-source code is the least of the problems, as well as getting it to compile properly. In fact, on a Linux/Unix environment, all you need is a functional Go development environment (and that's just a simple download away).

But on macOS, well... it's pretty easy to get the CLI components operational. The trouble starts with two culprits: on one side, there is the GUI. Hidden beneath the GUI is the Keybase-compiled FUSE component. That's far trickier to tackle, for several reasons:

• Compiling GUI applications for the Mac really, really require having the whole Apple development environment installed, and being familiar enough with it — which I'm not. Although most of Keybase's UI is based on a platform-neutral development framework, the whole packaging — in order to get a "valid", installable package — definitely requires the Apple framework.
• On that note — whatever is installed through the Mac's UI requires (these days) to be signed with a developer certificate issued by Apple. This has a yearly cost. On top of that, there is a submission process, during which Apple evaluates all the code, and counter-signs. Otherwise, you get all those popups warning the user that this is an unsigned application from an unknown developer and therefore dangerous to install. Not exactly what you wish to see on an application which makes high claims about how secure it is!
• The FUSE component is an utter nightmare — but you need it to be able to use your Keybase disk space as just another network folder — which is one of the key advantages of the Keybase ecosystem. The main problem is really that the version that Keybase ships is based on ancient code, which was fully open-source, and worked by installing a kernel extension. Over the years, Apple has frowned more and more upon kernel extensions, and limited the number of developers still allowed to sign kernel extensions (and get them counter-signed as well). By sheer chance (or perhaps not!), that same developer has retained that privilege. What he did (I suppose to make his life easier) was to ship 99% of the code as open-source as before, except a tiny bit which contains the actual kernel extension — and that is properly signed. It's not a question of the price — that binary can also be obtained for free — but not its source code. Thus, Keybase opted to continue to use the old, open-source version — since the process of including a signed binary from a third-party, which, in turn, needs to be counter-signed by Keybase and by Apple... well, according to the Keybase developers, that was far too much trouble. Instead, they relied on the good, old, deprecated but open-source version of the original code.
• Unfortunately, that code doesn't work at all in the most recent versions of macOS, since Apple has definitely "pulled the plug" on the old-style kernel extensions. That is code that definitely needs to be rewritten from scratch — and even so it might not work at all! That, unfortunately, is way, way beyond my abilities...
• One solution, if Keybase were willing, would be to simply remove the FUSE component from the "standard distribution", so to speak, and just ship the rest. At the point when the GUI asks for installing this component (which, currently, is never "named" at all, in order to simplify the procedure for the less-tech-inclined Mac users), it would just give a link to the current version of macFUSE and tell the user to install it first, reboot their system, and install the Keybase app again.
• Also note that many other applications out there already install macFUSE for their own purposes, and it's not unlikely that the current procedure will actually install an additional kernel extension, which might not even be necessary! But Keybase assumed (probably correctly) that most of its users do not have any FUSE component pre-installed, and therefore installs its own.
• Although Keybase didn't change anything on their macFUSE fork (except for the name, to avoid conflicts), and all their code is designed to work with either library (i.e. their own or an external one), there is a catch: all the code was written for the old API. The new, contemporary versions of macFUSE requires using a new API, which, incidentally, is much more closely aligned with the original code of the FUSE project, which originated in Linux. For developers, this means that there is a reasonably easy path to port the same code for (at least) Linux, FreeBSD, and Darwin/macOS. So long, that is, that the new API is used. (Note: contemporary Linux kernels all include the FUSE component by default, so Linux users have no issue with that!)
• And, in fact, Keybase under Linux does use the "standard", modern FUSE API; it's just the Mac version that uses the "old" API; theoretically, therefore, it should be easy to stitch both together; in practice, I foresee a long period of code refactoring to achieve that, and Keybase doesn't seem to be willing to spend that amount of time.

Thus, my "ugly hack" — a "quick fix", requiring some manual intervention, but which allows Keybase to work with a different macFUSE version, and not their own. This works quite nicely.

But I understand that beyond Ventura nothing will ever work again as before, as Apple ruled out all possibilities of installing kernel extensions at all (at least, that's what I read). So... I'm afraid that there are no tricks nor hacks to get Keybase to mount remote volumes using FUSE.

Well, not with a kernel-based version of FUSE, of course. Apple, in fact, is not averse to have people mounting whatever they wish on their systems — so long as it runs on userland, and not in the kernel. And, indeed, the whole point of FUSE is to allow just that! The only issue is that in order to enable userland drivers to work, there needs to be at least a tiny bit of help from the kernel itself. There is simply no workaround to that.

The ideal, of course, would be to Apple to ship their kernels already with the FUSE component built-in, just like Linux does. But that's hardly something that they're willing to do; they simply don't see enough interest in that technology to burden their own developers with it. I mean, it would be easy for Apple to do that, but that's not going to happen.

Mac developers, nevertheless, are very resourceful, so there is a way to deal with this (perhaps there are more than one, but I'm just aware of one). It's called FUSE-T. The developer noticed that, although it gets harder and harder to access the macOS kernel, there is no need for that at all. Apple already allows a way to mount remote folders using a few non-Apple protocols (SMB-CIFS for Windows comes immediately to mind). By some strange reason, they decided to invest a lot in the oldest remote file sharing protocol — NFS (literally, "Network File System"), which was actually developed by Sun Microsystems... in 1984 (!). It is an IETF standard, fully open, not encumbered by any licenses or patents; but in the Windows ecosystem, for historic reasons, it was never widespread. Why? Well, because DOS/Windows hardly ever used Internet protocols to communicate — that was what Unix used. DOS users relied upon network services based on protocols developed by Novell, and, later, the ubiquitous SMB protocol introduced with Windows for Workgroups. Microsoft, for many decades, refused to accept the mere existence of the Internet at all. When they finally made a 180º change in their policy, and rebranded themselves as "the Internet company", fully embracing the popular Internet Protocols, it was 1995 — with the release of Windows 95.

1995, ironically, was when NFS version 3 came out. It is perhaps the most widespread version of NFS in use (1 was merely experimental; 2 is way too old and insecure). One might wonder why Microsoft didn't use it back then. Or, for all purposes, Apple. I suppose that it all comes from the stigma of being "a Unix network protocol", for Unix machines to mount remote directories of other Unix machines, and therefore not really interesting for users of personal computers. And, in any case, Novell and Microsoft (and a few others) already had their own network protocols, and a legacy of services and applications working on top of those.

NFS was, IMHO, simply sort of "forgotten" or merely considered "way too old and insecure" for the modern world. That might, indeed, be true of the ancient version 2 protocol. 3 was already quite acceptable by 1995. And further developments were introduced in version 4 (ca. 2000), namely, a better permissions mechanism for allowing access to remote files and directories.

Let me just open another parenthesis here. NFS was designed for the LAN of the mid-1980s — which could deliver 10 Mbps at best — and so it had to be especially efficient in such a scenario. At the current rates — from 1 Gbps to 10 Gbps — NFS effectively allows for delivery speeds equivalent to a local (rotary) hard disk.

In other words, it's fast.

Very fast!

In fact, so fast, that when I recently switched the remote network mounts to my Synology NAS at home, from the "usual" SMB-CIFS to NFS... I couldn't believe my eyes. There is simply no perceptible difference between writing to disk or to a remote folder. I can run applications from the NAS's disks as if they were running locally on my machine. It's mind-boggling.

But, then again, it shouldn't come as a surprise:

• NFS was designed to be fast even on a 10 Mbps Ethernet of the 1980s; obviously, as the LAN speeds increased, as well as everything else (CPU, memory access, disk access...), NFS kept its minimalistic, no-frills, bare-bones, ultra-fast protocol. It just became better and better — and faster and faster! — as legions of programmers, working on the protocol for the past 40 years (!), just kept doing a better and better job at it; and
• Apple is boasting of the incredible performance of their implementation of NFSv4. Why exactly they're doing that is not clear to me. Will Apple ultimately drop SMB-CIFS (as they have dropped their own AFS) and encourage users and vendors to embrace NFS instead — because it's so incredibly fast? "Nobody knows".

Nevertheless, I'm glad they did it. Because this is where the next-generation FUSE-T steps in :)

The author, Alex Fishman (aka @macos-fuse-t), did something quite clever: he added a "conversion layer" between the FUSE library and NFS! What that means is that the userland application that uses the FUSE library (which would be Keybase, in our case) simply "talks FUSE" to FUSE-T; from its perspective, it's "talking to the kernel" and requesting to mount a directory and so forth. But then FUSE-T takes these commands and turns them into NFS commands! Thus, NFS is used to mount a volume that is actually being requested by FUSE. The kernel has no idea — it just accepts the NFS requests and mounts a perfectly normal NFS filesystem. It gets managed exactly like any other NFS mount, and shows up as a NFS mount everywhere — including, of course, Finder. From the other side of the connection — i.e., Keybase — it just prepares access to the remote folder (using Keybase's own API), and communicates these to the FUSE library API. Both sides never talk to each other directly; neither is aware of what the other is doing. Keybase just thinks it's calling the "usual" FUSE library and feeding it whatever is required to access a remote file on their infrastructure; while the Mac kernel only knows that it has now a freshly-mounted NFS filesystem, which it accesses — and manages — exactly like any of the others.

All of this works so well because Apple's NFSv4 implementation is so fast! From the perspective of the user expecting almost-instant access to a remote folder on Finder, the whole FUSE-to-NFS translation is imperceptible. It has next-to-zero overhead. It's incredibly efficient. And, of course, it's how Apple intends things to work in the present (and the future!): keep your hands away from the kernel, use what we already provide you, feel free to be creative in your userland sandbox.

More importantly — Apple may even change the kernel in the future (not merely upgrading macOS every year!), or making everything surrounding the kernel completely different, from the perspective of a developer requiring to interface with the kernel. But they will always keep the NFSv4 implementation fully operational and compatible with the standards, no matter how they actually implement it. As such, FUSE-T will only need to "talk NFS" to the system; it doesn't need to worry about any "deprecated" APIs with the kernel, for instance. After all, NFS is with us for over four decades and likely to continue to be — exactly with the same API as it was defined in the past, even if the actual code may have changed drastically over so many decades. Thus, it's future-proof — and that means that calling the FUSE framework in this way becomes future-proof as well.

Personally, I think that this is the work of pure genius, thinking totally out of he box, and therefore neatly supporting the whole framework in a way that will never be questioned by Apple. In theory, Apple could even develop a completely new operating system — abandoning its BSD Unix codebase, for instance, and moving towards their own operating system, designed from scratch (the likelihood of that happening is effectively zero). FUSE-T couldn't care less about that; all it requires is that future versions of macOS merely expose the usual NFS API — which seems now to be something we can take for granted.

Now, back to our issues with Keybase... unfortunately, things are not that easy on thatside. As mentioned, there are several different ways of accessing the FUSE libraries. Those running Linux are lucky, because there is just one reference implementation to worry about, and it's already part of the kernel anyway. Under macOS, however, there have been several different implementations — each author trying to come up with something that was close to the Linux mindset, but could be implemented on a Unix (BSD) framework. Sometimes this meant drifting further and further apart from the reference implementation (although the current trend is to move again closer to the Linux reference, to make it easier on developers writing for both platforms).

As I've discussed elsewhere (I can't remember exactly where), macOS is essentially Darwin with a proprietary GUI on top of it, and Darwin is for all purposes Unix — more specifically, of the BSD flavour — with a Mach microkernel (thus setting it apart from, say, the FreeBSD/OpenBSD/NetBSD family, which use a monolithic kernel instead). Linux, by contrast, is "Unix-like" — Linus Torvalds wrote the kernel code to deliberately avoid infringing any patents or licenses, and that meant making Linux something different from Unix, while, at the same time, giving the application layer (mostly GNU tools, at the beginning) an interface that looks & feels like "Unix" — but it isn't. At the kernel level, or close to it, the non-Unix-ness of Linux becomes more and more apparent. Also, to make things even worse, Linux was vaguely inspired on the SysV flavour of Unix, not the BSD style used by Darwin, FreeBSD, and many others, which makes things even more annoying, as the two main Unix families do things considerably differently at the kernel level (again, due to licensing issues, Berkeley also had to rewrite the core Unix kernel from scratch, while still providing the same higher-level portability of most of its tools). Over the years, there has been some convergence — especially because Linux is the most used kernel in the world (think Android!), with BSD Unix as a runner-up (think iOS!), and SysV Unix does not even register a bleep on the market share graphs...

As a result, all the above means that Keybase would require a substantial amount of code refactoring to use FUSE-T. And you'd need to build a new package (so, the same issue would apply as above). Unless, of course, you just update the CLI that does the directory-redirecting magic, and make it available separately. That would work. Unfortunately, it's far beyond my meagre abilities...

---

P. S. I guess that's a built-in feature of Second Life — all characters tend to look great :-) And this picture is nothing special, I was just having fun tweaking settings to see what worked better :)

P. P. S. Kudos to you for knowing how Second Life avatars look like :)

[obscurerichard]

It would be kind of redundant to enter yet another issue for this, but I'm running MacOS Sonoma, 14.4.1 (23E224) and (obviously) can't load the FUSE extension.

I hope this gets fixed eventually!

my log id: 7e4a84bc2bd9461b12368b1c

[GwynethLlewelyn]

It would be kind of redundant to enter yet another issue for this, but I'm running MacOS Sonoma, 14.4.1 (23E224) and (obviously) can't load the FUSE extension.

I wonder, did you by any chance try out FUSE-T?

I'm just curious to learn if it works at all under Sonoma :)

That said, FUSE-T is not open source, it's just a free binary for personal use, and I'm not sure if it can be integrated into Keybase (which is technically a "commercial" product although I have no clue whatever Keybase does to earn money these days...)

[Danrancan]

I thank for your totally misdirected but well-intended compliment :)

No, I'm not really "knowledgeable" — I'm not even a professional programmer — I just happen to dig as deep as I can into things that I really like until I figure out how they work :) Sadly, I keep getting distracted with things outside my alleged sphere of work — such as Keybase, which I (still) find a fascinating solution.

Forking Keybase's open-source code is the least of the problems, as well as getting it to compile properly. In fact, on a Linux/Unix environment, all you need is a functional Go development environment (and that's just a simple download away).

But on macOS, well... it's pretty easy to get the CLI components operational. The trouble starts with two culprits: on one side, there is the GUI. Hidden beneath the GUI is the Keybase-compiled FUSE component. That's far trickier to tackle, for several reasons:

• Compiling GUI applications for the Mac really, really require having the whole Apple development environment installed, and being familiar enough with it — which I'm not. Although most of Keybase's UI is based on a platform-neutral development framework, the whole packaging — in order to get a "valid", installable package — definitely requires the Apple framework.

• On that note — whatever is installed through the Mac's UI requires (these days) to be signed with a developer certificate issued by Apple. This has a yearly cost. On top of that, there is a submission process, during which Apple evaluates all the code, and counter-signs. Otherwise, you get all those popups warning the user that this is an unsigned application from an unknown developer and therefore dangerous to install. Not exactly what you wish to see on an application which makes high claims about how secure it is!

• The FUSE component is an utter nightmare — but you need it to be able to use your Keybase disk space as just another network folder — which is one of the key advantages of the Keybase ecosystem. The main problem is really that the version that Keybase ships is based on ancient code, which was fully open-source, and worked by installing a kernel extension. Over the years, Apple has frowned more and more upon kernel extensions, and limited the number of developers still allowed to sign kernel extensions (and get them counter-signed as well). By sheer chance (or perhaps not!), that same developer has retained that privilege. What he did (I suppose to make his life easier) was to ship 99% of the code as open-source as before, except a tiny bit which contains the actual kernel extension — and that is properly signed. It's not a question of the price — that binary can also be obtained for free — but not its source code. Thus, Keybase opted to continue to use the old, open-source version — since the process of including a signed binary from a third-party, which, in turn, needs to be counter-signed by Keybase and by Apple... well, according to the Keybase developers, that was far too much trouble. Instead, they relied on the good, old, deprecated but open-source version of the original code.

• Unfortunately, that code doesn't work at all in the most recent versions of macOS, since Apple has definitely "pulled the plug" on the old-style kernel extensions. That is code that definitely needs to be rewritten from scratch — and even so it might not work at all! That, unfortunately, is way, way beyond my abilities...

• One solution, if Keybase were willing, would be to simply remove the FUSE component from the "standard distribution", so to speak, and just ship the rest. At the point when the GUI asks for installing this component (which, currently, is never "named" at all, in order to simplify the procedure for the less-tech-inclined Mac users), it would just give a link to the current version of macFUSE and tell the user to install it first, reboot their system, and install the Keybase app again.

• Also note that many other applications out there already install macFUSE for their own purposes, and it's not unlikely that the current procedure will actually install an additional kernel extension, which might not even be necessary! But Keybase assumed (probably correctly) that most of its users do not have any FUSE component pre-installed, and therefore installs its own.

• Although Keybase didn't change anything on their macFUSE fork (except for the name, to avoid conflicts), and all their code is designed to work with either library (i.e. their own or an external one), there is a catch: all the code was written for the old API. The new, contemporary versions of macFUSE requires using a new API, which, incidentally, is much more closely aligned with the original code of the FUSE project, which originated in Linux. For developers, this means that there is a reasonably easy path to port the same code for (at least) Linux, FreeBSD, and Darwin/macOS. So long, that is, that the new API is used. (Note: contemporary Linux kernels all include the FUSE component by default, so Linux users have no issue with that!)

• And, in fact, Keybase under Linux does use the "standard", modern FUSE API; it's just the Mac version that uses the "old" API; theoretically, therefore, it should be easy to stitch both together; in practice, I foresee a long period of code refactoring to achieve that, and Keybase doesn't seem to be willing to spend that amount of time.

Thus, my "ugly hack" — a "quick fix", requiring some manual intervention, but which allows Keybase to work with a different macFUSE version, and not their own. This works quite nicely.

But I understand that beyond Ventura nothing will ever work again as before, as Apple ruled out all possibilities of installing kernel extensions at all (at least, that's what I read). So... I'm afraid that there are no tricks nor hacks to get Keybase to mount remote volumes using FUSE.

Well, not with a kernel-based version of FUSE, of course. Apple, in fact, is not averse to have people mounting whatever they wish on their systems — so long as it runs on userland, and not in the kernel. And, indeed, the whole point of FUSE is to allow just that! The only issue is that in order to enable userland drivers to work, there needs to be at least a tiny bit of help from the kernel itself. There is simply no workaround to that.

The ideal, of course, would be to Apple to ship their kernels already with the FUSE component built-in, just like Linux does. But that's hardly something that they're willing to do; they simply don't see enough interest in that technology to burden their own developers with it. I mean, it would be easy for Apple to do that, but that's not going to happen.

Mac developers, nevertheless, are very resourceful, so there is a way to deal with this (perhaps there are more than one, but I'm just aware of one). It's called FUSE-T. The developer noticed that, although it gets harder and harder to access the macOS kernel, there is no need for that at all. Apple already allows a way to mount remote folders using a few non-Apple protocols (SMB-CIFS for Windows comes immediately to mind). By some strange reason, they decided to invest a lot in the oldest remote file sharing protocol — NFS (literally, "Network File System"), which was actually developed by Sun Microsystems... in 1984 (!). It is an IETF standard, fully open, not encumbered by any licenses or patents; but in the Windows ecosystem, for historic reasons, it was never widespread. Why? Well, because DOS/Windows hardly ever used Internet protocols to communicate — that was what Unix used. DOS users relied upon network services based on protocols developed by Novell, and, later, the ubiquitous SMB protocol introduced with Windows for Workgroups. Microsoft, for many decades, refused to accept the mere existence of the Internet at all. When they finally made a 180º change in their policy, and rebranded themselves as "the Internet company", fully embracing the popular Internet Protocols, it was 1995 — with the release of Windows 95.

1995, ironically, was when NFS version 3 came out. It is perhaps the most widespread version of NFS in use (1 was merely experimental; 2 is way too old and insecure). One might wonder why Microsoft didn't use it back then. Or, for all purposes, Apple. I suppose that it all comes from the stigma of being "a Unix network protocol", for Unix machines to mount remote directories of other Unix machines, and therefore not really interesting for users of personal computers. And, in any case, Novell and Microsoft (and a few others) already had their own network protocols, and a legacy of services and applications working on top of those.

NFS was, IMHO, simply sort of "forgotten" or merely considered "way too old and insecure" for the modern world. That might, indeed, be true of the ancient version 2 protocol. 3 was already quite acceptable by 1995. And further developments were introduced in version 4 (ca. 2000), namely, a better permissions mechanism for allowing access to remote files and directories.

Let me just open another parenthesis here. NFS was designed for the LAN of the mid-1980s — which could deliver 10 Mbps at best — and so it had to be especially efficient in such a scenario. At the current rates — from 1 Gbps to 10 Gbps — NFS effectively allows for delivery speeds equivalent to a local (rotary) hard disk.

In other words, it's fast.

Very fast!

In fact, so fast, that when I recently switched the remote network mounts to my Synology NAS at home, from the "usual" SMB-CIFS to NFS... I couldn't believe my eyes. There is simply no perceptible difference between writing to disk or to a remote folder. I can run applications from the NAS's disks as if they were running locally on my machine. It's mind-boggling.

But, then again, it shouldn't come as a surprise:

• NFS was designed to be fast even on a 10 Mbps Ethernet of the 1980s; obviously, as the LAN speeds increased, as well as everything else (CPU, memory access, disk access...), NFS kept its minimalistic, no-frills, bare-bones, ultra-fast protocol. It just became better and better — and faster and faster! — as legions of programmers, working on the protocol for the past 40 years (!), just kept doing a better and better job at it; and

• Apple is boasting of the incredible performance of their implementation of NFSv4. Why exactly they're doing that is not clear to me. Will Apple ultimately drop SMB-CIFS (as they have dropped their own AFS) and encourage users and vendors to embrace NFS instead — because it's so incredibly fast? "Nobody knows".

Nevertheless, I'm glad they did it. Because this is where the next-generation FUSE-T steps in :)

The author, Alex Fishman (aka @macos-fuse-t), did something quite clever: he added a "conversion layer" between the FUSE library and NFS! What that means is that the userland application that uses the FUSE library (which would be Keybase, in our case) simply "talks FUSE" to FUSE-T; from its perspective, it's "talking to the kernel" and requesting to mount a directory and so forth. But then FUSE-T takes these commands and turns them into NFS commands! Thus, NFS is used to mount a volume that is actually being requested by FUSE. The kernel has no idea — it just accepts the NFS requests and mounts a perfectly normal NFS filesystem. It gets managed exactly like any other NFS mount, and shows up as a NFS mount everywhere — including, of course, Finder. From the other side of the connection — i.e., Keybase — it just prepares access to the remote folder (using Keybase's own API), and communicates these to the FUSE library API. Both sides never talk to each other directly; neither is aware of what the other is doing. Keybase just thinks it's calling the "usual" FUSE library and feeding it whatever is required to access a remote file on their infrastructure; while the Mac kernel only knows that it has now a freshly-mounted NFS filesystem, which it accesses — and manages — exactly like any of the others.

All of this works so well because Apple's NFSv4 implementation is so fast! From the perspective of the user expecting almost-instant access to a remote folder on Finder, the whole FUSE-to-NFS translation is imperceptible. It has next-to-zero overhead. It's incredibly efficient. And, of course, it's how Apple intends things to work in the present (and the future!): keep your hands away from the kernel, use what we already provide you, feel free to be creative in your userland sandbox.

More importantly — Apple may even change the kernel in the future (not merely upgrading macOS every year!), or making everything surrounding the kernel completely different, from the perspective of a developer requiring to interface with the kernel. But they will always keep the NFSv4 implementation fully operational and compatible with the standards, no matter how they actually implement it. As such, FUSE-T will only need to "talk NFS" to the system; it doesn't need to worry about any "deprecated" APIs with the kernel, for instance. After all, NFS is with us for over four decades and likely to continue to be — exactly with the same API as it was defined in the past, even if the actual code may have changed drastically over so many decades. Thus, it's future-proof — and that means that calling the FUSE framework in this way becomes future-proof as well.

Personally, I think that this is the work of pure genius, thinking totally out of he box, and therefore neatly supporting the whole framework in a way that will never be questioned by Apple. In theory, Apple could even develop a completely new operating system — abandoning its BSD Unix codebase, for instance, and moving towards their own operating system, designed from scratch (the likelihood of that happening is effectively zero). FUSE-T couldn't care less about that; all it requires is that future versions of macOS merely expose the usual NFS API — which seems now to be something we can take for granted.

Now, back to our issues with Keybase... unfortunately, things are not that easy on thatside. As mentioned, there are several different ways of accessing the FUSE libraries. Those running Linux are lucky, because there is just one reference implementation to worry about, and it's already part of the kernel anyway. Under macOS, however, there have been several different implementations — each author trying to come up with something that was close to the Linux mindset, but could be implemented on a Unix (BSD) framework. Sometimes this meant drifting further and further apart from the reference implementation (although the current trend is to move again closer to the Linux reference, to make it easier on developers writing for both platforms).

As I've discussed elsewhere (I can't remember exactly where), macOS is essentially Darwin with a proprietary GUI on top of it, and Darwin is for all purposes Unix — more specifically, of the BSD flavour — with a Mach microkernel (thus setting it apart from, say, the FreeBSD/OpenBSD/NetBSD family, which use a monolithic kernel instead). Linux, by contrast, is "Unix-like" — Linus Torvalds wrote the kernel code to deliberately avoid infringing any patents or licenses, and that meant making Linux something different from Unix, while, at the same time, giving the application layer (mostly GNU tools, at the beginning) an interface that looks & feels like "Unix" — but it isn't. At the kernel level, or close to it, the non-Unix-ness of Linux becomes more and more apparent. Also, to make things even worse, Linux was vaguely inspired on the SysV flavour of Unix, not the BSD style used by Darwin, FreeBSD, and many others, which makes things even more annoying, as the two main Unix families do things considerably differently at the kernel level (again, due to licensing issues, Berkeley also had to rewrite the core Unix kernel from scratch, while still providing the same higher-level portability of most of its tools). Over the years, there has been some convergence — especially because Linux is the most used kernel in the world (think Android!), with BSD Unix as a runner-up (think iOS!), and SysV Unix does not even register a bleep on the market share graphs...

As a result, all the above means that Keybase would require a substantial amount of code refactoring to use FUSE-T. And you'd need to build a new package (so, the same issue would apply as above). Unless, of course, you just update the CLI that does the directory-redirecting magic, and make it available separately. That would work. Unfortunately, it's far beyond my meagre abilities...

---

P. S. I guess that's a built-in feature of Second Life — all characters tend to look great :-) And this picture is nothing special, I was just having fun tweaking settings to see what worked better :)

P. P. S. Kudos to you for knowing how Second Life avatars look like :)

This is by far the most epic post I have ever encountered on the internet in all my life! Who are you? Jeeeeeez louize you are polished with backgrounds of everything! You are so incredibly knowledgable and articulate that my mind is literally blown! Holy cow this is a lot to soak in and think about! How long did this take you to write? It covers literally everything! Im shocked! Lol. Anyways, thank you so much for putting your knowledge and prowess out here on the internet for public consumption. Kowledge is power and you have made me more powerful today. So thank you. I suppose it seems like there really isnt anything that comprimises developer time constraints and workload with the perfect most wanted solution, so I suppose I will have to just hope for the best, or start eying up other apps that could be cooking up in the open source community that might provide similar functionality. Either way, im happy i posted this issue strictly because i feel like I gained a wealth of knowledge from your posts. You write so elegantly! So thank you for empowering me and slightly overwhelming me with your wealthy working brain!

thanks a ton! You are a treasure to the internet!

P.S. My appologies for the misdirected compliment.

P.P.S. I had no idea what second life characters looked like. I stalked your github profile and checked out your website and i think facebook you have listed because I found you to be such an impressive person. All signs indicated you were a big second life fan! Lol. Anyways, cheers again and thanks again!