Most filesystems and operating systems these days support extended attributes (Linux, FreeBSD, MacOS, & Windows all do) . It would be nice if gsutil cp and gsutil rsync were able to preserve these attributes when uploading & downloading.
And to make sure there's no confusion, I'm not referring to POSIX attributes, which are already supported, but extended attributes, which are arbitrary key/value pairs.
For example on linux I can do setfattr -n user.foo -v bar myfile.txt to set user.foo=bar on myfile.txt. The idea is that when this file is copied to google storage, these attributes would be copied as custom object metadata.
Note though, that on some implementations of file extended attributes, the attributes have to be namespaced. For example on linux you have 4 namespaces, security, system, trusted, and user, hence why in the above example, it was user.foo. Thus to support this, gsutil will need to pick a specific namespace to map metadata into. Meaning if an object in google storage has the key foo, you can't just set the extended attribute foo, it has to be user.foo. And since this mapping has to apply when downloading objects, I would recommend that the user. prefix be stripped off when uploading.
jameslittle
commented
1 year ago
Most filesystems and operating systems these days support extended attributes (Linux, FreeBSD, MacOS, & Windows all do) . It would be nice if
gsutil cpandgsutil rsyncwere able to preserve these attributes when uploading & downloading.And to make sure there's no confusion, I'm not referring to POSIX attributes, which are already supported, but extended attributes, which are arbitrary key/value pairs.
For example on linux I can do
setfattr -n user.foo -v bar myfile.txtto setuser.foo=baronmyfile.txt. The idea is that when this file is copied to google storage, these attributes would be copied as custom object metadata.Note though, that on some implementations of file extended attributes, the attributes have to be namespaced. For example on linux you have 4 namespaces,
security,system,trusted, anduser, hence why in the above example, it wasuser.foo. Thus to support this,gsutilwill need to pick a specific namespace to map metadata into. Meaning if an object in google storage has the keyfoo, you can't just set the extended attributefoo, it has to beuser.foo. And since this mapping has to apply when downloading objects, I would recommend that theuser.prefix be stripped off when uploading.