hardlink in APFS macOS High Sierra 10.13

[karthikeyan-mac]

harlink is not working in APFS macOS High Sierra.

It shows "Error: Operation not permitted". Do we have any other way to create hardlink for directories in APFS .

Thanks & Regards, Karthikeyan

[BenjaminHCCarr]

Thanks for letting people know, one more reason to stay on 10.11.6!

Though I desperately want APFS to work on OSX/macOS. The early shipped not-turned on ZFS was so frustrating. TensCompliment tried so hard, glad OpenZFS and OpenZFS on OS X kept it going.

I would guess there is an undocumented hard-linking, if dedupe is implemented they kind of have to have something in the backend, one would think. They are handling sym- and hard-links effectively underneath, just not activated to user.

I see no support in @timsutton gist of the tools available: https://gist.github.com/timsutton/e4a5b5948b78a9f1c52d0769b5498d71

... yet ...

[luckman212]

_From Apple's APFS FAQ:_

Does Apple File System support directory hard links?

Directory hard links are not supported by Apple File System. All directory hard links are converted to symbolic links or aliases when you convert from HFS+ to APFS volume formats on macOS.

Grr, I lost some data due to having a few directory hardlinks on my HFS+ volume before upgrading to 10.13 - during the automatic conversion to APFS, I was somehow left with 0 copies of my actual original files, and just a bunch of broken symlinks.

[jamespayne]

If they claim it converts them to symbolic links and you've lost data, you should report it to them immediately. Thanks for the warning.

[karthikeyan-mac]

Hardlink will convert as soft link but the files/folders will be in /.HFS+ Private Directory Data?. You will have the folders like dir_** in /.HFS+ Private Directory Data? in the root drive.

[jamespayne]

So is the only solution to stay on HFS+ if you want to retain your hard links? Is there an option to not upgrade the file system?

[luckman212]

No option was presented to me on the installs I did. It just went right ahead and forklifted my filesystem.

According to this thread on MacRumors, you can skip the conversion by manually launching the installer with a --converttoapfs NO argument:

/Applications/Install\ macOS\ High\ Sierra.app/Contents/Resources/startosinstall --converttoapfs NO

[jamespayne]

Thanks a lot @luckman212 Did you end up finding your files?

[luckman212]

Files were destroyed but luckily I did find them in a recent Time Machine backup. So, no actual data loss but I can see how some might not be so lucky. I expect we'll hear some horror stories coming out, luckily manually-created hardlinks are quite rare on macOS.

[jamespayne]

Good to hear you got your files back. I guess if Apple don't implement some sort of solution for this hln is dead for OS X on AFPS :( Did you have a chance to check @bewithkarthi 's information he provided or was it too late at that stage?

[luckman212]

Yes I did check that actually. I have a folder at root / called .HFS+ Private Directory Data and inside there is a folder called dir_56395352 but it is completely empty. I confirmed from Terminal as root.

[jamespayne]

Great. Thanks for taking the time to share the info.

[jamespayne]

I've added some documentation to cover this discussion in this pull request https://github.com/selkhateeb/hardlink/pull/36

[yangm97]

Funny. Their “Documents and Desktop” on iCloud used to rely on hardlinks. I wonder what’s going on now.

[lishengting]

@bewithkarthi , thanks a lot. I found all (?) my files inside that directory. But why they named it ".HFS+ Private Directory Data?" The '?' is '^M' actually. WTF!!!!!!! They used WINDOWS to name the folder?

[lishengting]

NOOOO! @bewithkarthi , I only find half of my folders in that directory. Where are the others? The disk spaced showed they are must be somewhere on the disk (>100G missing).

[karthikeyan-mac]

I had a different observation today. If we leave the hard link as its is and upgrade to High Sierra, the directories are at least moving to ".HFS+ Private Directory Data?" but if you unlink the directory and upgrade the source directory changes to symbolic link and the data is completely lost.

[lishengting]

@bewithkarthi And the space is NOT released!!!! I can't find my data anywhere but the space of the hard disk is still occupied!!!!

[diimdeep]

Is it confirmed that APFS does not support hard link directories ?

[thelvis4]

@diimdeep Yes, see the comment above.

[sandipb]

I have tried to create a DMG file formatted into HFS, but for some reason I cannot find out, hln is giving an error "operation not permitted" even on this mounted volume. Any ideas?

[eliot-akira]

Reportedly, bindfs allows hard-linking directories in macOS >= 10.13. (Source)

[elvey]

Current docs on clones: https://developer.apple.com/documentation/foundation/file_system/about_apple_file_system Bloody hell. I miss hard links. TimeMachine relies (relied?) heavily on hard links. I guess it's using clones now on APFS.

HARDLINKS LIVE?: Other documentation - the APFS reference - mentions multiple "extended field"s that are "used only for hard links". And later, there's plain English:

"Hard links that all refer to the same inode are called siblings. Each sibling has its own identifier thatʼs used instead of the shared inode number when siblings need to be distinguished. For example, some Carbon APIs in macOS use sibling identifiers. The sibling whose identifier is the lowest number is called the primary link. The other siblings copy various properties of the primary link, as discussed in j_inode_val_t. You use sibling links and sibling maps to convert between sibling identifiers and inode numbers. Sibling-link records let you find all the hard links whose target is a given inode. Sibling-map records let you find the target inode of a given hard link."

I recall that I was using a different tool to create directory hard links on HFS+, but I can't find/recall it and I guess it's facing the same challenges.
https://jak-linux.org/projects/hardlink/ doesn't seem useful, though brew offers to install it.
As for https://formulae.brew.sh/formula/bindfs which Eliot mentions above... it relies on FUSE and may not be a solution anyway, and I'm too tired to delve into why, but as I recall, there's still an open source fork of FUSE, but the main branch is no longer compatible, per https://github.com/Homebrew/homebrew-core/pull/64491...
Sorry if this is more bugspammy than useful. Hoping it's the latter.

[kirbyzhou]

hardlink-osx is very useful to manipulate the disk of time machine. Could somebody put it back?

[rgaufman]

Argh, any solution to this? - I have a Mac mini with a 256GB SSD, this is unusable. I tried to:

ln -s /Volumes/Storeator/iCloud ~/Library/Mobile\ Documents

But as soon as I turn iCloud on, it removes my symlink and insists on using internal storage! - Any solution?