"standard system SDK" way to replace existing file doesn't work on ZFS datasets

[JMoVS]

I use Marked2.app to create a pdf from a markdown file currently. Exporting the file works the first time. But if the file already exists, it naturally asks me whether I want to replace it. Clicking "replace" doesn' actually touch the file at all (no change of modified date). If I do the same on an HFS+ volume, it works.

I reached out to the Dev, a little bit of details can be found at the bottom of this page (which I by now marked private due to further unrelated discussions with the dev in the thread). I will link him to this issue report so he maybe himself can chime in.

A screenshot of the relevant part is:

[ttscoff]

Hi, developer of Marked 2 here. When saving the paginated PDF to disk, Marked uses the (Cocoa) print system to write a temporary file, reads it into a PDFDocument, and uses PDFDocument's writeToURL:withOptions method. This works fine on all HFS volumes, but I've had problems reported with overwriting existing files on both NFS and ZFS systems (no error, just a failure to write). I'm not sure why, and would be open to any suggestions in that area.

[ilovezfs]

@ttscoff I'm just hoping it's not using this under the hood https://github.com/openzfsonosx/zfs/issues/49

[ttscoff]

It shouldn't, but I'll dig into the API docs and see if I need a workaround.

-Brett

On Apr 24, 2016, 10:56 AM -0500, ilovezfsnotifications@github.com, wrote:

@ttscoff(https://github.com/ttscoff)I'm just hoping it's not using this under the hood#49(https://github.com/openzfsonosx/zfs/issues/49)

— You are receiving this because you were mentioned. Reply to this email directly orview it on GitHub(https://github.com/openzfsonosx/zfs/issues/478#issuecomment-213988315)

[lundman]

Nothing too suspicious in here. dtruss on "Marked 2" just lists the usual fstat and getattrlist on the exported pdf filename. Does also call getattrlistbulk - but that is handled by VFS compat layer for us afaik.

It does not call vnop_exchange, nor anything that returns any obvious syscall errors.

But since it's an API, it most likely is passed onto another process to handle, like nsurlstoraged or filecoordination etc. So I need to dtrace deeper. ZFS only gets:

Apr 25 15:08:02 icgi-vip kernel[0]: -vnop_lookup 0 : dvp 4 'qwe.pdf'
Apr 25 15:08:04 icgi-vip kernel[0]: +readdir: 0xffffff80222cca50
Apr 25 15:08:04 icgi-vip kernel[0]: -readdir 0 (nument 28)

So, it looks it up (fstat) and one directory listing.

[lundman]

dtracing between replacing on HFS and ZFS has not yet revealed any insight, including the other daemons in play. Most peculiar.

@ttscoff I am guessing that Marked 2 would give some indication if writeToURL:withOptions returned an error? Ie, right now its failing silently, and returning success? Are you using any interesting withOptions ?

[ttscoff]

The only option passed is a quartz filter, nothing that would affect the write process.

I'll put some debug code in and see if there are any exceptions slipping by. I'm not a zfs user, so first step would be to set up a drive unless someone can suggest an easier way for debugging purposes…

-Brett

On Apr 27, 2016, 3:13 AM -0500, Jorgen Lundmannotifications@github.com, wrote:

dtracing between replacing on HFS and ZFS has not yet revealed any insight, including the other daemons in play. Most peculiar.

@ttscoff(https://github.com/ttscoff)I am guessing thatMarked 2would give some indication ifwriteToURL:withOptionsreturned an error? Ie, right now its failing silently, and returning success? Are you using any interestingwithOptions?

— You are receiving this because you were mentioned. Reply to this email directly orview it on GitHub(https://github.com/openzfsonosx/zfs/issues/478#issuecomment-215006745)

[lundman]

You don't have to use a real physical harddisk, if you have some space, create a large empty file (mkfile command will do nicely) and make a pool on it.

# mkfile 200M /var/tmp/poolfile.img
# zpool create BOOM /var/tmp/poolfile.img

Use the export command when you no longer want the volume around.

# zpool export BOOM

If you have to reboot or similar, and you want the volume back, import it again (or you can re-create if you want)

# zpool import -d /var/tmp BOOM

[emory]

I think this behavior also presents itself with Fission from Rogue Amoeba -- editing an audio file or batch encoding would end up leaving a lot of cruft behind.

I don't know if it's related or a different issue.

I also use Marked and can try to replicate if helpful @Lundman

[ilovezfs]

@ttscoff I'm curious whether if you compile with writeToFile instead of writeToURL if it exhibits the same behavior.

[ttscoff]

I can test that. I'll try to get an image set up, replicate, then debug over the weekend.

-Brett

On Apr 28, 2016, 9:22 PM -0500, ilovezfsnotifications@github.com, wrote:

@ttscoff(https://github.com/ttscoff)I'm curious whether if you compile with writeToFile instead of writeToURL if it exhibits the same behavior.

— You are receiving this because you were mentioned. Reply to this email directly orview it on GitHub(https://github.com/openzfsonosx/zfs/issues/478#issuecomment-215612771)

[JMoVS]

@ttscoff Any news on this issue? Can we help you with the setup or any other way?

[lundman]

I actually want to make a post about this to Apple, I never get to use those support tickets anyway, but needed to confirm that writeToURL:withOptions is not returning an error.

[ilovezfs]

Can we just try a hello world using the method before resorting to that? lol

[lundman]

Tried a small sample code to see if I can replicate it more controlled.

First I tried:

PDFDocument pdf;
[pdf writeToURL withOptions]

Which saves first time, and returns error the second time (File exists).

NSData *data;
[data writeToURL: options: error: ]

which saves every time correctly, even if the file is there.

I have not yet found a way to make it say it saved a file, but leaving the file untouched.

[JMoVS]

@ttscoff Slight bump here. Any news on this front from your side?

[ttscoff]

Sorry, no, it's been a rough few months. I'll try to get back to this as soon as I can.

[JMoVS]

sorry to hear that. Get better soon!