Open hippietrail opened 1 year ago
A more-or-less equivalent Rust program shares the behaviour of Zig, but Swift does not. This makes me think the dir walker code in each language's libraries does in fact need to handle firmlinks specifically and that Swift must do do. This despite claims they should be transparent to existing code.
Here's my Swift code's output:
dirwalker / LLVM
******** / ********
Library/Frameworks/Xamarin.iOS.framework/Versions/15.10.0.5/LLVM
Users/hippietrail/.vscode-insiders/extensions/ms-vscode.cpptools-1.5.1/LLVM
Users/hippietrail/.vscode/extensions/ms-vscode.cpptools-1.12.4-darwin-arm64/LLVM
Applications/Xcode.app/Contents/Applications/Instruments.app/Contents/PlugIns/DTLLVMBinaryAnalysisPlugin.xrplugin
---- all done ----
And the Swift code itself. You can see there's no special code for firmlinks:
import Foundation
import AppKit
let fileManager = FileManager.default
let resKeys : [URLResourceKey] = [.isDirectoryKey, .fileSizeKey, .isSymbolicLinkKey]
let startURL: URL = URL(string: fileManager.currentDirectoryPath)!
guard CommandLine.arguments.count == 3 else {
print("** usage: dirwalker path string")
exit(1);
}
let pathArg = CommandLine.arguments[1]
let matchArg = CommandLine.arguments[2]
if let path = pathArg.addingPercentEncoding(withAllowedCharacters: .urlQueryAllowed) {
if let url = URL(string: path) {
let en = fileManager.enumerator(at: url,
includingPropertiesForKeys: resKeys,
options: [.producesRelativePathURLs],
errorHandler: { (url, error) -> Bool in
return true }
)!
mainloop: for case let fileURL as URL in en {
do {
let rv = try fileURL.resourceValues(forKeys: Set(resKeys))
if let d = rv.isDirectory, d {
let filename: String = fileURL.lastPathComponent;
if filename.contains(matchArg) {
print(fileURL.relativePath)
}
}
} catch {
print("** error 2:", error)
}
}
}
}
print("done")
I did research this issue a while back, but here are my findings/notes in case it might help someone who wants to implement it or comment their opinion on it:
firmlinks can only apply on directories.
While there is an internal flag for identifying firmlinks, it is not set or used on APFS (see here)
What some projects implements like rofl0r/ncdu (with--exclude-firmlinks
) is checking the reference attribute to see if it points to the same location (see here).
GETPATH_NOFIRMLINK (102)
when using fcntl
call to show the firmlink reference and compare it.
This is only documented in XCode man pages.
This would look similar fs.realpath
or os.realpath
darwin functions (getFdPath
).
https://github.com/ziglang/zig/blob/a1aa55ebe52636f28335fd7ab30321fc52f48775/lib/std/os.zig#L5218-L5229Synthetic firmlinks can be created since MacOS Catalina via synthetic.conf
(Although i am not sure whether it actually creates a firmlink or an identifiable symlink with flag, needs further checking).
For system firmlinks, /usr/share/firmlinks
contains the list of firmlinks.
What's interesting is that MacOS's /bin/realpath
does not resolve firmlinks.
Here are way that I think can be used to solve this is:
when calling nextDarwin
and finding a Directory,
double check the reference/realpath and set entry kind depending on the result.
https://github.com/ziglang/zig/blob/a1aa55ebe52636f28335fd7ab30321fc52f48775/lib/std/fs.zig#L391
Documenting that Walker does not recognize/classify firmlinks and let the user do this check manually and/or maybe add a Firmlink check function?
Check with /usr/share/firmlinks
and synthetic.conf
(this can apply for any of the above).
I am not sure what's the best approach here as there aren't many system firmlinks and it's uncommon to synthesize (if that's even possible).
Note: I could be missing some other info, but this is what I have found.
Zig Version
0.10.0-dev.4324+c23b3e6fd
Steps to Reproduce and Observed Behavior
This is probably not a fully minimal example but I'm not fluent in Zig yet:
Run it with a starting path and a substring to search for. It will walk the directory and print out all paths that contain the substring after a
>
symbol and then its resolved canonical path after a»
symbol on the next line.I can only see the wrong behaviour when running in the root directory
/
as macOS has several directories from/System/Volumes/Data/x
linked to/x
using "firmlinks", which is a feature of the APFS filesystem going back to Sierra. Example firmlinked directories./Library
->/System/Volumes/Data/Library
/Users
->/System/Volumes/Data/Users
/Applications
->/System/Volumes/Data/Applications
I can't find Apple official documentation on firmlinks. Apparently they're supposed to be transparent and not affect most code. I also don't know how to get them to show up using
ls
. Thels
commandline I know that will show the most extra info macOS-specific info isls -alO@ /
but that doesn't seem to reveal them unlesssunlnk
means this and only this:Sample commandline:
Actual output:
Expected Behavior
Directories should only be listed at their true locations as is the case with symlinks, not also at the locations they're firmlinked to. I believe that would look like this: