search

keybase / client

Keybase Go Library, Client, Service, OS X, iOS, Android, Electron
BSD 3-Clause "New" or "Revised" License
8.91k stars 1.23k forks source link

Enable Keybase in Finder doesn't work in macOS Big Sur 11.0.1 #24366

Closed jburnett closed 3 years ago

jburnett commented 4 years ago

After upgrading to macOS Big Sur 11.0.1, Keybase client is not working with Finder, and enabling does not work. In main app, select Files tab; "Enable Keybase in Finder?" should show at the top. Click to accept "I understand that closed source...", and push "Yes, enable" button.

Result: After working for a few seconds, nothing changes and "Yes, enable" button is enabled again. Attempting to navigate to any keybase managed files gives permission denied.

Keybase client for macOS 5.5.0-20200526170801+139bb348af (installed via brew). kbfuse version is 10.11 with symlinks for higher versions.

❯ ll /Library/Filesystems/kbfuse.fs/Contents/Extensions/
total 0
drwxr-xr-x@ 3 root  wheel    96B May 26 13:19 10.11/
lrwxr-xr-x  1 root  wheel     5B May 26 13:19 10.12@ -> 10.11
lrwxr-xr-x  1 root  wheel     5B May 26 13:19 10.13@ -> 10.11
lrwxr-xr-x  1 root  wheel     5B May 26 13:19 10.14@ -> 10.11
lrwxr-xr-x  1 root  wheel     5B May 26 13:19 10.15@ -> 10.11
GwynethLlewelyn commented 1 year ago

I've still got to try out some of the above suggestions (thanks, guys, for so many helpful comments!), but I've noticed something that might be a deal breaker.

I'm on a venerable MacBook Pro (Retina, 15-inch, Mid 2014), which can only run Big Sur (still supported by Apple at the time of writing), unless I go crazy and crack it with the OpenCore patcher... which, at this time, I'm as yet reluctant to do.

That said, I'm using Homebrew 3.6.16-2-gf679990, which has installed keybase version 6.0.3-20221212202006+608e46df72.

All worked reasonably well until very recently, when Apple decided to bump up macOS Big Sur to 11.7.3 (20G1102). I believe that the Homebrew team also updated Keybase at around the same time.

The result? Well, it seems that there is no 11.X kernel extension any longer:

$ ll /Library/Filesystems/kbfuse.fs/Contents/Extensions/
total 0
drwxr-xr-x 4 root wheel 128 Dec 12 20:49 ./
drwxr-xr-x 7 root wheel 224 Dec 12 20:49 ../
drwxr-xr-x 3 root wheel  96 Dec 12 20:49 12/
lrwxr-xr-x 1 root wheel   2 Dec 21 13:07 13 -> 12/

(the directory for macOS 12.X, however, seems to be quite complete).

And naturally enough, this means that Keybase.app will give the following installation error:

Keybase error message about failed kext install

At first, I thought it really was a conflict between kernel extensions, but it's really just the macOS 11 kext that is missing. The logs are vast and show things such as:

~/Library/Logs/keybase.kbfs.log

2022-12-21T12:05:01.999280Z ▶ [WARN kbfs mounter.go:41] 0ab KBFS failed to FUSE mount at /Volumes/Keybase (gwyneth): exit status 2
2022-12-21T12:05:01.999314Z ▶ [ERRO kbfs mount_interrupter.go:52] 0ac Mounting the filesystem failed: exit status 2

~/Library/Logs/Keybase.app.log

Action,2022-12-20T18:37:09.798Z,type: waiting:batchChangeWaiting:  {"payload":{"changes":[{"error":{"message":"ERROR CODE 218 - context deadline exceeded i
n method keybase.1.SimpleFS.simpleFSStat","name":"GENERIC","stack":"Error: ERROR CODE 218 - context deadline exceeded in method keybase.1.SimpleFS.simpleFS
Stat\n    at new RPCError (file:///Applications/Keybase.app/Contents/Resources/app/desktop/dist/main.bundle.js:1:3860882)\n    at convertToRPCError (file:/
//Applications/Keybase.app/Contents/Resources/app/desktop/dist/main.bundle.js:1:3861769)\n    at convertToError (file:///Applications/Keybase.app/Contents/
Resources/app/desktop/dist/main.bundle.js:1:3861828)\n    at file:///Applications/Keybase.app/Contents/Resources/app/desktop/dist/main.bundle.js:1:2347958\
n    at file:///Applications/Keybase.app/Contents/Resources/app/desktop/dist/822.bundle.js:2:324318\n    at Deferrals._call (file:///Applications/Keybase.a
pp/Contents/Resources/app/desktop/dist/822.bundle.js:2:417479)\n    at file:///Applications/Keybase.app/Contents/Resources/app/desktop/dist/822.bundle.js:2
:417671\n    at trampoline (file:///Applications/Keybase.app/Contents/Resources/app/desktop/dist/822.bundle.js:2:417207)\n    at Deferrals._fulfill (file:/
//Applications/Keybase.app/Contents/Resources/app/desktop/dist/822.bundle.js:2:417624)\n    at ret (file:///Applications/Keybase.app/Contents/Resources/app
/desktop/dist/822.bundle.js:2:416617)","code":218,"fields":null,"desc":"context deadline exceeded","details":"Error code 218: GENERIC in method keybase.1.S
impleFS.simpleFSStat"},"increment":false,"key":"fs:stat"},{"error":{"message":"ERROR CODE 218 - context deadline exceeded in method keybase.1.SimpleFS.simp
leFSStat","name":"GENERIC","stack":"Error: ERROR CODE 218 - context deadline exceeded in method keybase.1.SimpleFS.simpleFSStat\n    at new RPCError (file:
///Applications/Keybase.app/Contents/Resources/app/desktop/dist/main.bundle.js:1:3860882)\n    at convertToRPCError (file:///Applications/Keybase.app/Conte
nts/Resources/app/desktop/dist/main.bundle.js:1:3861769)\n    at convertToError (file:///Applications/Keybase.app/Contents/Resources/app/desktop/dist/main.
bundle.js:1:3861828)\n    at file:///Applications/Keybase.app/Contents/Resources/app/desktop/dist/main.bundle.js:1:2347958\n    at file:///Applications/Key
base.app/Contents/Resources/app/desktop/dist/822.bundle.js:2:324318\n    at Deferrals._call (file:///Applications/Keybase.app/Contents/Resources/app/deskto
p/dist/822.bundle.js:2:417479)\n    at file:///Applications/Keybase.app/Contents/Resources/app/desktop/dist/822.bundle.js:2:417671\n    at trampoline (file
:///Applications/Keybase.app/Contents/Resources/app/desktop/dist/822.bundle.js:2:417207)\n    at Deferrals._fulfill (file:///Applications/Keybase.app/Conte
nts/Resources/app/desktop/dist/822.bundle.js:2:417624)\n    at ret (file:///Applications/Keybase.app/Contents/Resources/app/desktop/dist/822.bundle.js:2:41
6617)","code":218,"fields":null,"desc":"context deadline exceeded","details":"Error code 218: GENERIC in method keybase.1.SimpleFS.simpleFSStat"},"incremen
t":false,"key":"fs:stat"}]},"type":"waiting:batchChangeWaiting"}

~/Library/Logs/keybase.kbfs.log

2022-12-21T13:07:57.450543Z ▶ [DEBU keybase socket_nix.go:135] 053 - SocketInfo#dialSocket(unix:/Users/gwyneth/Library/Group Containers/keybase/Library/Caches/Keybase/keybased.sock) -> ERROR: dial unix /Users/gwyneth/Library/Group Containers/keybase/Library/Caches/Keybase/keybased.sock: connect: no such fileor directory [time=94.709µs]
2022-12-21T13:07:57.450571Z ▶ [DEBU keybase socket_nix.go:105] 054 + SocketInfo#dialSocket(unix:/Users/gwyneth/Library/Caches/Keybase/keybased.sock)
2022-12-21T13:07:57.450582Z ▶ [DEBU keybase socket_nix.go:134] 055 | net.Dial(unix:/Users/gwyneth/Library/Caches/Keybase/keybased.sock)
2022-12-21T13:07:57.450661Z ▶ [DEBU keybase socket_nix.go:135] 056 - SocketInfo#dialSocket(unix:/Users/gwyneth/Library/Caches/Keybase/keybased.sock) -> ERROR: dial unix /Users/gwyneth/Library/Caches/Keybase/keybased.sock: connect: no such file or directory [time=78.124µs]
2022-12-21T13:07:57.450678Z ▶ [DEBU keybase socket.go:93] 057 | DialSocket -> ERROR: There were multiple errors: dial unix /Users/gwyneth/Library/Group Containers/keybase/Library/Caches/Keybase/keybased.sock: connect: no such file or directory; dial unix /Users/gwyneth/Library/Caches/Keybase/keybased.sock: connect: no such file or directory
2022-12-21T13:07:57.450686Z ▶ [WARN kbfs connection.go:612] 058 (CONN KeybaseDaemonRPC 2545f8b3) Connection: error dialing transport: There were multiple errors: dial unix /Users/gwyneth/Library/Group Containers/keybase/Library/Caches/Keybase/keybased.sock: connect: no such file or directory; dial unix /Users/gwyneth/Library/Caches/Keybase/keybased.sock: connect: no such file or directory
2022-12-21T13:07:57.450724Z ▶ [DEBU kbfs connection.go:827] 059 (CONN KeybaseDaemonRPC 2545f8b3) RetryNotify operation result: There were multiple errors:dial unix /Users/gwyneth/Library/Group Containers/keybase/Library/Caches/Keybase/keybased.sock: connect: no such file or directory; dial unix /Users/gwyneth/Library/Caches/Keybase/keybased.sock: connect: no such file or directory
2022-12-21T13:07:57.450730Z ▶ [WARN kbfs keybase_daemon_rpc.go:362] 05a KeybaseDaemonRPC: connection error: "There were multiple errors: dial unix /Users/gwyneth/Library/Group Containers/keybase/Library/Caches/Keybase/keybased.sock: connect: no such file or directory; dial unix /Users/gwyneth/Library/Caches/Keybase/keybased.sock: connect: no such file or directory"; retrying in 2s
2022-12-21T13:07:58.450203Z ▶ [WARN kbfs keybase_daemon_rpc.go:402] 05b Background keep alive hit an error: context deadline exceeded
2022-12-21T13:07:58.450319Z ▶ [DEBU kbfs connection.go:766] 05c (CONN KeybaseDaemonRPC 2545f8b3) Connection: getReconnectChan
2022-12-21T13:07:58.450350Z ▶ [DEBU kbfs connection.go:723] 05d (CONN KeybaseDaemonRPC 2545f8b3) Connection: waitForConnection; status: 1
2022-12-21T13:07:59.479849Z ▶ [DEBU kbfs connection.go:825] 05e (CONN Chat ebf468dd) RetryNotify attempt
2022-12-21T13:07:59.479885Z ▶ [DEBU kbfs connection.go:606] 060 (CONN Chat ebf468dd) Connection: dialing transport
2022-12-21T13:07:59.479897Z ▶ [DEBU keybase socket.go:68] 061 + GetSocket
2022-12-21T13:07:59.479866Z ▶ [DEBU kbfs connection.go:825] 05f (CONN KeybaseDaemonRPC 2545f8b3) RetryNotify attempt
2022-12-21T13:07:59.479900Z ▶ [WARN kbfs keybase_daemon_rpc.go:402] 062 Background keep alive hit an error: context deadline exceeded
2022-12-21T13:07:59.479907Z ▶ [DEBU keybase context.go:221] 063 - GetSocket -> ok [time=981ns]
2022-12-21T13:07:59.479924Z ▶ [DEBU kbfs connection.go:766] 065 (CONN KeybaseDaemonRPC 2545f8b3) Connection: getReconnectChan
2022-12-21T13:07:59.479927Z ▶ [DEBU keybase socket.go:78] 066 | empty socket wrapper; need a new one
2022-12-21T13:07:59.479917Z ▶ [DEBU kbfs connection.go:606] 064 (CONN KeybaseDaemonRPC 2545f8b3) Connection: dialing transport
2022-12-21T13:07:59.479935Z ▶ [DEBU kbfs connection.go:723] 067 (CONN KeybaseDaemonRPC 2545f8b3) Connection: waitForConnection; status: 1
2022-12-21T13:07:59.479939Z ▶ [DEBU keybase socket_nix.go:105] 068 + SocketInfo#dialSocket(unix:/Users/gwyneth/Library/Group Containers/keybase/Library/Caches/Keybase/keybased.sock)
2022-12-21T13:07:59.479939Z ▶ [DEBU keybase socket.go:68] 069 + GetSocket
2022-12-21T13:07:59.479948Z ▶ [DEBU keybase socket_nix.go:134] 06a | net.Dial(unix:/Users/gwyneth/Library/Group Containers/keybase/Library/Caches/Keybase/keybased.sock)
2022-12-21T13:07:59.479952Z ▶ [DEBU keybase context.go:221] 06b - GetSocket -> ok [time=482ns]
2022-12-21T13:07:59.480108Z ▶ [DEBU keybase socket_nix.go:135] 06c - SocketInfo#dialSocket(unix:/Users/gwyneth/Library/Group Containers/keybase/Library/Caches/Keybase/keybased.sock) -> ERROR: dial unix /Users/gwyneth/Library/Group Containers/keybase/Library/Caches/Keybase/keybased.sock: connect: no such fileor directory [time=159.4µs]
[... etc, same error repeats over and over again...]

~/Library/Logs/keybase.service.log

2022-12-21T12:04:57.149181Z ▶ [DEBU keybase fuse_status_darwin.go:47] 014 No kext info available (kext not loaded)
2022-12-21T12:04:57.149225Z ▶ [DEBU keybase fuse_status_darwin.go:137] 015 Loading plist: /Library/Filesystems/kbfuse.fs/Contents/Info.plist
[...]
12.21.2022 12:04:57.617 KBTask:73[DEBG] Task (out): {
  "version": "4.4.1",
  "bundleVersion": "4.4.1",
  "kextID": "com.github.kbfuse.filesystems.kbfuse",
  "path": "/Library/Filesystems/kbfuse.fs",
  "kextStarted": false,
  "installStatus": 4,
  "installAction": 1,
  "mountInfos": [],
  "status": {
    "code": 0,
    "name": "OK",
    "desc": "OK",
    "fields": null
  }
}

12.21.2022 12:04:59.628 KBInstaller:45[INFO] Install complete
12.21.2022 12:04:59.628 KBInstaller:49[INFO] Privileged Helper: Installed, Bundle Version: 1.0.47
Version: 1.0.47
12.21.2022 12:04:59.628 KBInstaller:49[INFO] Fuse: Install Error: KextManager failed to load with status: -603947007
Installed, Version: 4.4.1
Kext ID: com.github.kbfuse.filesystems.kbfuse
Kext Loaded: No
Path: /Library/Filesystems/kbfuse.fs
12.21.2022 12:04:59.628 Installer:97[INFO] Exit(4)
2022-12-21T12:04:59.632358Z ▶ [ERRO keybase install_darwin.go:612] 620c Error installing KBFuse: exit status 4

FYI, keybase fuse status outputs:

{
  "version": "4.4.1",
  "bundleVersion": "",
  "kextID": "com.github.kbfuse.filesystems.kbfuse",
  "path": "/Library/Filesystems/kbfuse.fs",
  "kextStarted": false,
  "installStatus": 4,
  "installAction": 0,
  "mountInfos": [],
  "status": {
    "code": 0,
    "name": "OK",
    "desc": "OK",
    "fields": null
  }
}

... which, as said, doesn't contain an extension for macOS 11 (Big Sur). This is actually missing from the bundled installer itself:

$ ls -la /Applications/Keybase.app/Contents/Resources/KeybaseInstaller.app/Contents/Resources/kbfuse.bundle/Contents/Extensio
ns/
total 0
drwxr-xr-x 4 gwyneth admin 128 Dec 12 20:49 .
drwxr-xr-x 7 gwyneth admin 224 Dec 12 20:49 ..
drwxr-xr-x 3 gwyneth admin  96 Dec 12 20:49 12
lrwxr-xr-x 1 gwyneth admin   2 Dec 21 13:07 13 -> 12

Granted, I can retrieve the 'old' version (11) from previous versions, but I haven't tried it out. BTW, the bundled kbfuse is clearly meant to be used with macOS 12.3 and higher, as can be seen inside its Info.plist: 

[...]
        <key>LSMinimumSystemVersion</key>
        <string>12.3</string>

(see the Apple documentation on LSMinimumSystemVersion)

This is a recent change (https://github.com/keybase/client/pull/25338 and https://github.com/keybase/client/commit/4c62f42b5c0730086277f8edbe001e91f1da8e16), which not even the Homebrew maintainers have noticed: when upgrading to macfuse 4.4.1, @mmaxim bumped the supported version to macOS 12, and unfortunately dropped support for Big Sur (even though macfuse itself supports 10.9 to 13.

JPry commented 1 year ago

@GwynethLlewelyn That seems like a different issue since your version of macOS is older.. You should open a new issue instead of replying to this one to get the best visibility.

GwynethLlewelyn commented 1 year ago

@JPry — no, actually it's always the same issue :-) It has just been reported at least 20 times (as far as I've managed to count so far) for different versions of macOS, a few of which have been marked as 'closed'. I posted on this thread because it explicitly mentioned macOS Big Sur in the title!

But I believe a full answer to this should require a 'meta-thread' — since it's something that will happen over and over again.

GwynethLlewelyn commented 1 year ago

@jburnett please re-open this issue that you prematurely closed two years ago. The issue is not — I repeat, not — fixed. Just because there are some workarounds (namely, going back a few versions or compiling the Keybase Client from scratch) it doesn't mean that the mainline Keybase works.

On the other hand, I found something that the Keybase team might find interesting: FUSE-T, a kext-less way to access a FUSE filesystem. The authors on their page note that it is getting incredibly harder to support kext-base solutions on macOS, and with each new release, old kexts get broken and become harder and harder to support. This is most definitely the case with Keybase!

Their alternative? Add the FUSE API over a layer of Apple's NFS driver. Whatever requires FUSE to work will have a drop-in compatible library; while on the filesystem side, there is no need to use any kernel functionality at all to work with the filesystem. Instead, everything gets routed through NFS. NFS, as the authors so well point out, is blindingly fast1 — that's by design, from the 1980s, when Ethernet could only support 1 MBps and Sun workstations had to do their best with whatever they had. Of course, NFS is (unjustly) blamed for all sorts of security issues (which mostly happen because, unlike other remotely-mounted filesystems, NFS is hard to configure, and people tend to be careless about how they set up their permissions), but, in this scenario, there is no reason for worrying: everything happens locally, after all.

Using FUSE over NFS is both future-proof and legacy-proof. Apple will continue to support NFS 'forever', and they already have a very optimised implementation. NFS is a pretty standard protocol and will not unexpectedly change from one day to the other (not even if Apple wants that change!).

So... Keybase devs... any thoughts on this? Mind you, you're already doing your own port/fork of an old version of macFUSE anyway.

FYI, I did check the kbfs code, and I did make a few experiments running keybase + kbfs manually (from bash), trying to force kbfs to use whatever FUSE is installed. Unfortunately, the code provided by https://github.com/bazil/fuse will just check for OSXFUSE version 2.0 or 3.0, based on the default paths used by those versions. This is sort of 'built-in' (although it can be changed — in fact, the same mechanism is used by Keybase to use their own, tweaked version of FUSE, by 'feeding' its own directories pointing to the patched version of FUSE). Because FUSE-T works differently, kbfs cannot find it anywhere in the system (although it is there); I imagine that it will require a different way of initialising things for FUSE-T. At the very least, there needs to be a way for checking for the presence of 'something' that provides a valid FUSE interface but doesn't rely upon /dev/fuse existing (because it won't) neither on the 'usual' directories from which OSXFUSE/macFUSE are launched (because these won't exist).

I'm sure there must be a way to accomplish that!

1. In fact, I was utterly surprised when mounting remote shares on my humble Raspberry Pi, first via Samba, then via NFS. The purpose was to stream video from the Raspberry Pi to a not-very-smart TV; because the Pi doesn't have much memory, I needed to mount a share from my home NAS; by default, those shares are made via Samba/SMB but there is an option to do them via NFS as well. With Samba... well, I got hiccups all the time, and eventually the Pi would give up on the attempt, and sever the connection. Even when it worked — over Wi-Fi, mind you! — video stuttered and gasped, struggling to flow through the wireless connection. After switching to NFS, not only there was an insane speed improvement, but all the stuttering magically disappeared! Talk about UDP for tackling unpredictable wireless transmissions... it actually deals with them so much better than Samba's TCP connections!

rjurney commented 1 year ago

Yeah this ticket kind of stinks. It's a big problem and ignored. I get it, Keybase FUSE isn't a priority. It still hurts a lot of people's sore little piddys. At least leave it open so people can figure out workarounds without having to look so hard for it.

GwynethLlewelyn commented 1 year ago

[...] It's a big problem and ignored. I get it, Keybase FUSE isn't a priority. It still hurts a lot of people's sore little piddys. At least leave it open so people can figure out workarounds without having to look so hard for it.

Exactly!

Every other week or so I take a look at this, and it seems to me that there is no easy way around it. The best that one can hope to do is, well, clone this repository, add some tweaks to use the above-mentioned kext-less FUSE-T solution (it's sooooooo easy to use...), grab XCode, figure out Keybase's signing key, change, oh, perhaps two lines of code, get rid of the whole KBFUSE subsystem (which is an installer app by itself), and let it compile... which on my decade-old Mac will take perhaps 10 or 20 hours, I guess :-P

The only reason for that is to avoid the Keybase GUI installer to automatically install KBFUSE. The Go apps have no issue whatsoever (also, Go compiles blindingly fast anyway) — they expect either KBFUSE or OSXFUSE 3.0 (an obsolete but still fully open-source version) to be installed, and won't work otherwise. However, with a single line of code, you can change the check and accept OSXFUSE 29 — weirdly enough, that's what FUSE-T it reports when questioned about its version. That's all that the underlying subsystem needs. At the frontend, well, that's another story — especially at the installer level, of course, which needs a few changes. Not many. A few. After all, from the perspective of the Keybase application(s), FUSE is FUSE, they just cannot be so picky is wanting one very specific version and failing otherwise :-)

I might give the Go part a try (because, well, the GUI/Installer bits require XCode to be installed — no thanks!). With a little bit of luck, I may be able to get at least the redirector launching without complaining, and with the redirector happy, Finder should be happy too. The Keybase app by itself may be hopelessly confused after that, but who cares... let it remain confused :-) — so long as Finder is not confused and works, I don't care!

GwynethLlewelyn commented 1 year ago

bumps @jburnett to reopen this issue