Support for the metadata API?

[simonvanderveldt]

First of all, thanks for this package! Helps a lot with neatly automating my setup :)

I was wondering if Jack's metadata API can be used through jackclient-python? I couldn't find any references to it in the the current code, so maybe some work needs to be done to enable it? If so, what would that be? Maybe I can be of help.

[mgeier]

TBH, I wasn't aware of this API and therefore it's currently not supported. But I don't see a reason why not to implement this.

Before thinking about the implementation, we should think about the potential API on the Python side. Could you please make some suggestions how this could look like? It would be nice to see some hypothetical code examples for this.

[simonvanderveldt]

@mgeier I'm prototyping a way to treat separate jack clients as a group, i.e. create them, create the correct connections and present them to the user as a "stack", similar to how for example Ardour or non-mixer show plugins.

To be able to do this I was looking at a way to add some metadata/tags to clients and low and behold there's a metadata API in jack :)

The method's as they are available in the Jack API look good enough to me, though it would obviously be nice to have some higher lever/object oriented method's as well, for example to allow adding the metadata when creating a client.

[mgeier]

I don't really understand why some functions take a jack_client_t* and others don't. I was thinking about creating new methods on jack.Client objects, but if not all functions take a jack_client_t*, that doesn't really make sense, does it?

I also don't really understand how different data types are supposed to be handled, nor do I understand what a "subject" is supposed to be.

Can you please provide some concrete (but still hypothetical) examples that could help clarify this?

[simonvanderveldt]

I don't really understand why some functions take a jack_client_t and others don't. I was thinking about creating new methods on jack.Client objects, but if not all functions take a jack_client_t, that doesn't really make sense, does it?

Looking at the API docs, apparently getters don't need a jack_client_t, whereas all setters do. As to why that is I don't know. Probably easiest to ask in #lad, #jack or on the lad mailing list?

I also don't really understand how different data types are supposed to be handled, nor do I understand what a "subject" is supposed to be.

If I understand it correctly the subject is the client you want to set/update the metadata for, identified by its UUID.

Can you please provide some concrete (but still hypothetical) examples that could help clarify this?

I don't have enough knowledge of the Jack API to answer that question. Only thing I can give you is some examples of how it's being used https://github.com/search?q=jack_set_property&type=Code&utf8=%E2%9C%93

[mgeier]

Yes, asking the JACK folks is a good idea. You could also ask at the jack-devel mailing list (http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org) and suggest to add a bit more documentation.

I had a quick look at a few usage examples on Github, and I only saw it used for port properties. It also seems that the JACK API only provides a jack_uuid_t for ports. The function jack_get_uuid_for_client_name() returns the UUID as a string.

What kind of data type would you like to use for property values? It looks like data has to be provided as null-terminated C string + MIME type. We should probably make sure that the byte string that is supposed to be stored doesn't contain any null bytes. I don't know if we'll have to add a terminating null character or not. We can probably make a copy of the data which should take care of adding the terminating null character (but doesn't check for null characters in the input):

data = ffi.new('char[]', userdata)

[SpotlightKid]

Any news on this? I'm considering switching jack-matchmaker from using the included "homegrown" ctypes-based JACK bindings to jackclient-python, but would like to add support for port meta data to the program as well.

I would be appreciate it, if support for this was added to jackclient-python.

BTW, support for the meta data API only landed in jack2 in the last release (Dec 2018) and the jack_property command line program is still only in the development version.

[simonvanderveldt]

@SpotlightKid I haven't had any need for it recently so didn't spend any time on it.

[SpotlightKid]

Ok. I'm adding support for the meta data API to my Jack bindings now. If this works out, I'll try to submit a PR to jackclient-python. No promise yet, though.

[SpotlightKid]

I've now implemented support for the JACK meta data API in my ctypes-based JACK bindings. If you want to have a look:

https://github.com/SpotlightKid/jack-matchmaker/blob/master/jackmatchmaker/jacklib.py#L1649

(Most of the new code concerned with meta data is below this line. A few enum (L206ff.), struct (L275ff.) and callback function type (L319f.) definitions are near the top of the file, as well as some constants for meta data property keys (L82ff.).

Cffi-bindings for jackclient-python could probably look very similar.

As to your questions from 2017:

I don't really understand why some functions take a jack_client_t* and others don't.

Functions, which only retrieve properties, do not need to pass a jack_client_t pointer, only those which want to set or delete properties or subscribe to changes.

I also don't really understand how different data types are supposed to be handled, nor do I understand what a "subject" is supposed to be.

A subject is anything with a UUID, i.e. a client or a port, to which properties can be attached.

Properties have a key, data (value) and an optional type. The key is an URI, so it's always an ASCII string (I think). I treat it as an UTF-8 encoded string. type can also be an URI or a mime-type, but it's not entirely clear, whether it is restricted to those two options, when it is set.

If type is not set, value is also a null-terminated string, but type can be used to indicate that it is to be interpreted in another way. The Jack API does not provide a way to retrieve an arbitrarily long property value string with nulls embedded, though, so storing, for example, binary image data in a property is impractical (I need to get confirmation from the Jack devs on this). It is also not clear whether value can be encoded in anything other than ASCII (or UTF-8).

Here's some info on Jack meta data, which I just recovered from the old (pre Apr 2014) jackaudio.org web site:

https://github.com/jackaudio/jackaudio.github.com/wiki/JACK-Metadata-API

There's also some discussion going on atm about documenting more standard meta data property key URIs in this jack2 PR:

https://github.com/jackaudio/jack2/pull/436

[mgeier]

Thanks @SpotlightKid!

It seems to me that an API based on UUIDs makes sense for the Python bindings. This way, no separate functions for ports and clients have to be provided.

However, the strange thing is that port UUIDs seem to be integers, while client UUIDs are strings (see jack_get_uuid_for_client_name(), I've also mentioned that above: https://github.com/spatialaudio/jackclient-python/issues/40#issuecomment-304826369). What's that all about?

@SpotlightKid Are you using client properties at all?

Another thing that's bugging me is the lifetime of the returned data. It seems wasteful to make copies of everything, but OTOH, since the data is given as null-terminated strings, I think it isn't really possible to create a buffer from it with CFFI without copying.

Regarding str vs bytes, I think it would be best to return a str if type is empty, but a bytes object if type is given. key and type themselves could be str, though?

I don't really like to involve any encoding in the API. Users can decode their bytes objects on their own, if they want to.

The Jack API does not provide a way to retrieve an arbitrarily long property value string with nulls embedded, though, so storing, for example, binary image data in a property is impractical

I think the idea is that non-text data should be stored (for example) in base64 encoding. At least that's suggested in the docs for jack_property_t::type.

[SpotlightKid]

However, the strange thing is that port UUIDs seem to be integers, while client UUIDs are strings

Apparently, you need to call jack_uuid_parse on the result of jack_get_uuid_for_client_name. At least that's what jack_property does.

Are you using client properties at all?

jack-matchmaker only uses port "pretty-names", and my jacklib bindings do not wrap jack_uuid_parse yet (but I will add it now).

key and type themselves could be str, though?

Yes, I'm pretty sure about that now. key should always be a URI, so only ASCII is allowed. type should be either a URI or a mimetype, which also means ASCII only, so the C bytes string can just be decode()d.

Many clients set the type for the http://jackaudio.org/metadata/pretty-name property to text/plain, so in my code I decode the value to a string if the property type starts with text/ for convenience.

[SpotlightKid]

I've implemented a get_client_properties function now, which uses jack_get_uuid_for_client_name and jack_uuid_parse to get the client UUID. Here's an example script that uses it: list_all_client_properties.py

[mgeier]

@SpotlightKid I've played around a bit and I don't seem to get anything to work!

All port UUIDs are 0, jack_uuid_parse() doesn't write to the given output parameter, and the metadata setter functions don't seem to work.

BTW, support for the meta data API only landed in jack2 in the last release (Dec 2018)

AFAICT, the last release (1.9.12) was actually in Dec 2017: https://github.com/jackaudio/jack2/releases

Are you using a not-yet-released version?

$ python3 list_port_info.py 
Port name: system:capture_1
UUID: 0
Aliases: alsa_pcm:hw:0:out1

Port name: system:capture_2
UUID: 0
Aliases: alsa_pcm:hw:0:out2

Port name: system:playback_1
UUID: 0
Aliases: alsa_pcm:hw:0:in1

Port name: system:playback_2
UUID: 0
Aliases: alsa_pcm:hw:0:in2

[SpotlightKid]

Maybe I got it wrong that meta data support on JACK2 is already in a released version. But you can test it with the current Git repo. Or with JACK1.

[SpotlightKid]

I checked and, unfortunately, meta data support has been added to JACK2 only after the latest release (2017-12-13) on 2018-10-08 in https://github.com/jackaudio/jack2/commit/81bddd49bccaad1498270b51f8d887ffa873cd87 (plus some later fixes).

[mgeier]

Thanks for the info @SpotlightKid!

I've tried it with JACK1 and it works fine.

I've made a first attempt of implementing this in #64, what do you think about that?

[SpotlightKid]

I left some review comments on #64.