Ability to read and write MacOS extended attributes on CIFS volume

[deepcoder]

I was wondering of there is any way to use pyxattr to access extended attributes used by MacOS on CIFS mounted drive. I am able to use pyxattr to read and write these attributes on a local ext4 volume. The attribute names used by MacOS seem to often contain the colon ":" character. I have set the smb.conf to allow the colon character in the main file stream names with options shown below, however these do not seem to apply to the extended attribute streams:

If I try to read the attributes from a file on Samba share that has MacOS attributes set, they are not retrieved by pyxattr code on remote machine, however same code on local file works fine.

If I try to write a attribute on the CIFS volume with a colon in it's name, the pyxattr module returns: error '[Errno 22] Invalid argument'

example of attribute I am trying to write, works fine on local ext4 volume, however fails on CIFS volume:

xattr.setxattr(zpath, b'user.DosStream.AFP_AfpInfo:$DATA', 'blab')

smb.conf

vfs objects = catia fruit streams_xattr
fruit:aapl = yes
fruit:encoding = native
fruit:locking = none
fruit:metadata = stream
fruit:resource = file
fruit:model = MacSamba

mangled names = no

[iustin]

I am not sure, but I'd be surprised if samba (server) and cifs client can transport xattrs well (or even at all). So the first question is if you can set these using command line tools.

If the command line tools work, then the next step would be to get an equivalent of strace (not sure how it's called under MacOS) for the failure.

[deepcoder]

Thanks for your work on this project. Again, your program works fine when run on a MacOS machine, it fails when run on a Ubuntu machine writing to a local Ubuntu ext4 volume. This seems to work for you according to your documentation. Can you point me to a resource on how it use strace on Ubuntu, and perhaps I can share some better information with you.

What perplexes me is if I run the listed program below on MacOS 12.6 with it creating the file either on a local volume or a SMB attach Linux 2004.1 volume and setting the attributes as listed, it works. on the Mac CLI, I can use xattr to see and set the attributes and on the Ubuntu machine (with the file created by the MacOS machine on the Ubuntu machine filesystem), I can see the attributes with the linux getfattr command

On the Ubuntu machine can see the attributes with getfattr but I am not able to set the attributes with setfattr.

If i run the same program on the Ubuntu machine, it fails with the error below. And using the linux setfattr CLI program, this fails to create any attributes.

Same errors and experience when trying the test on Ubuntu 22.04.1 LTS x64 machine. Although on 22.04.1 the attributes get created when run on the MacOS machine, however they display differently (less attributes) on the command line, but the MacOS GUI Finder still sees the attribute set.

user@hp-600-g1-dm-01:~/pyxattr$ ./test03.py
Traceback (most recent call last):
  File "./test03.py", line 19, in <module>
    xattr.setxattr("test5.txt", b"com.apple.FinderInfo", b"\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0E\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x2A\xD9\xB9\x32\x80\x00\x00\x00")
OSError: [Errno 95] Operation not supported

on the linux machine, with the file created from the MacOS machine:

user@hp-600-g1-dm-01:~/pyxattr$ getfattr -d test5.txt
# file: test5.txt
user.DOSATTRIB=0sAAAEAAQAAABRAAAAIAAAALm3J4Rm3tgBubcnhGbe2AE=
user.DosStream.AFP_AfpInfo:$DATA=0sQUZQAAAAAQAAAAAAgAAAAAAAAAAAAAAAAA4AAAAAAAAAAAAAAAAAACrZuTKAAAAAAAAAAAAAAAAAAAAAAA==
user.DosStream.com.apple.metadata_kMDItemUserTags:$DATA=0sYnBsaXN0MDChAVhPcmFuZ2UKNwgKAAAAAAAAAQEAAAAAAAAAAgAAAAAAAAAAAAAAAAAAABMA

user@hp-600-g1-dm-01:~/pyxattr$

#! /usr/bin/env python3

# test03.py
# 202210121106
#
# test setting macos tags from linux python
# https://github.com/iustin/pyxattr
#

import xattr
from pathlib import Path

# create a new blank file or update access time of file if it exists
Path("test5.txt").touch()

# com.apple.metadata:_kMDItemUserTags
# set tag to orange

xattr.setxattr("test5.txt", b"com.apple.FinderInfo", b"\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0E\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x2A\xD9\xB9\x32\x80\x00\x00\x00")

# xattr.setxattr("test5.txt", b"com.apple.metadata:_kMDItemUserTags", b'bplist00\xa1\x01XOrange\n7\x08\n\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x13\x00')
a = b'\x62\x70\x6C\x69\x73\x74\x30\x30\xA1\x01\x58\x4F\x72\x61\x6E\x67\x65\x0A\x37\x08\x0A\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x13'

xattr.setxattr("test5.txt", b"com.apple.metadata:_kMDItemUserTags", a)

[iustin]

Just a quick note (without investigating too deep): attributes, on Linux, need to be set on a namespace, and there are a few known/supported namespaces: security, system, trusted and user (see this man page). What I suspect here is that you’re missing the “user” namespace, so - from Linux - you need always use xattr.setxattr(“test5.txt”, b”user.com.apple.metadata:…”, a) instead of plain com.apple…..

Now, I’m not sure why it works on Mac - maybe there the user namespace is implicit, or Mac doesn’t name namespaces? It would be interesting to see the output of xattr.listxattr on a file with attributes from both mac and linux. That should show whether the two OSes differ in the namespace aspect.

[iustin]

Also, this could simply be a bug/incomplete support for MacOS, or just different OS APIs that are not easy to work around. I’ll try to take a deeper look in the next couple of weeks, but it does sound like an interesting issue.

[deepcoder]

Your tip about prepending the "user.com" at the beginning of the attribute name allows the program to run from both MacOS and Ubuntu on top of the Ubuntu ext4 directory, the MacOS program also still works on top of the MacFS directory locally.

Interesting lesson in 'name space' for me.... by using a combo of your print(xattr.listxattr("test5.txt")) call and the getfattr -d -e hex test5.txt cli utility I see these some kind of remapping happening. Using the example below, I am now able to add the MacOS GUI finder attributes that I want to a file on a ext4 file system for Ubuntu from a python3 app running on Ubuntu 20.04.

output of xattr.listattr run from Ubuntu on Ubuntu directory , attributes added by routine running on Ubuntu as shown below

[b'user.DOSATTRIB', b'user.DosStream.AFP_AfpInfo:$DATA', b'user.DosStream.com.apple.metadata\xef\x80\xa2_kMDItemUserTags:$DATA']

output of xattr.listattr run from MacOS on Ubuntu directory, attributes added by python app running on Ubuntu routine below

[b'com.apple.FinderInfo', b'com.apple.metadata:_kMDItemUserTags']

output of getfattr cli utility run on Ubuntu

user@hp-600-g1-dm-01:~/pyxattr$ getfattr -d -e hex test5.txt
# file: test5.txt
user.DOSATTRIB=0x000004000400000051000000200000004401f9cb91ded8014401f9cb91ded801
user.DosStream.AFP_AfpInfo:$DATA=0x414650000000010000000000800000000000000000000000000e00000000000000000000000000002ad9b9328000000000000000000000000000000000
user.DosStream.com.apple.metadata_kMDItemUserTags:$DATA=0x62706c6973743030a101584f72616e67650a37080a0000000000000101000000000000000200000000000000000000000000000013

#! /usr/bin/env python3
#test06.py
#202210121649
#
#test setting macos tags from linux python
#https://github.com/iustin/pyxattr
#

import xattr
from pathlib import Path

#create a new blank file or update access time of file if it exists
Path("test5.txt").touch()

#set file tag to orange

#getfattr -d -e hex test5.txt
#print(bytes.fromhex()

a = b'\x00\x00\x04\x00\x04\x00\x00\x00Q\x00\x00\x00 \x00\x00\x00D\x01\xf9\xcb\x91\xde\xd8\x01D\x01\xf9\xcb\x91\xde\xd8\x01'

xattr.setxattr("test5.txt", b"user.DOSATTRIB", a)

a = b'AFP\x00\x00\x00\x01\x00\x00\x00\x00\x00\x80\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0e\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00*\xd9\xb92\x80\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

xattr.setxattr("test5.txt", b"user.DosStream.AFP_AfpInfo:$DATA", a)

a = b'\x62\x70\x6C\x69\x73\x74\x30\x30\xA1\x01\x58\x4F\x72\x61\x6E\x67\x65\x0A\x37\x08\x0A\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x13'

xattr.setxattr("test5.txt", b"user.DosStream.com.apple.metadata\xef\x80\xa2_kMDItemUserTags:$DATA", a)