Best practices for colors

[bltzr]

Exploring several OSCQuery implementations while trying to implement compatibility for colors in libossia, I came across some questions about color management:

Since there is a type for it in OSC, I guess the simple/straightforward way to handle this is with "r" type tag, which according to the OSC specs, is a 32-bit value describing 4 8-bit RGBA values. Period.

That's what all implementations I found seemed to do (Vezér and OSCQuery Browser when emitting, MadMapper when receiving). libossia doesn't yet, but we're working on it.

The only example I found actually holding color values was MadMapper (BTW @mrRay what about exposing VDMX parameters to OSCQuery - is this on the roadmap, or you're considering leaving VDMX OSCQuery-client-only ?)

Though, as a side note, I noticed that, even though MadMapper receives "regular" OSC-compliant 4bytes colors correctly, It actually announces their values as an array of 4 floats in the range [0. 1.]. such as this:

  "rgba" : {
    "ACCESS" : 3,
    "DESCRIPTION" : "Rgba Color",
    "FULL_PATH" : "\/medias\/Grid Generator\/parameters\/Color\/Back Color\/rgba",
    "TYPE" : "r",
    "VALUE" : [
      0.2352941185235977173,
      0.3333333432674407959,
      1,
      1
    ]
  }

I see the interest of this (better color precision), while I'm wondering if that's not precisely an example where OVERLOADS would be useful.

Well, anyhow, I guess we should have a correct way to represent those values in the JSON files. Maybe as in the in graphic design/web/proce55ing), by using the hex format, such as 0xCC006699. What do you think ?

[mrRay]

"The only example I found actually holding color values was MadMapper (BTW @mrRay what about exposing VDMX parameters to OSCQuery - is this on the roadmap, or you're considering leaving VDMX OSCQuery-client-only ?)"

• it is conceptually/theoretically impossible for any of the "helper" apps i wrote to "hold" color values, because this would require the helper app to have knowledge of the address space of the app it's helping. to the best of my knowledge, this is not doable.
• vdmx's support for publishing OSCQuery values is already complete- anything in a control surface is automatically published. since VDMX isn't a bridge app, it is capable of holding color values- and like MadMapper, it uses an array of four floating-point values to communicate a single RGBA value.

"Though, as a side note, I noticed that, even though MadMapper receives "regular" OSC-compliant 4bytes colors correctly, It actually announces their values as an array of 4 floats in the range [0. 1.]. such as this:"

i can't speak for MM's implementation, but when i had to write this i made my implementation use an array of four numbers because JSON doesn't have a native type for "color", and sending a hex string seemed like just as much of a hack as using an array to describe the color with none of the perks. you're absolutely right- this does need to be clarified, thanks for bringing this up. the way i see it, we have two choices:

• we can tell devs to use strings containing hex values describing an RGBA structure compliant with OSC (8 bits per channel, four channels per pixel, ordered RGBA) when they need to describe a color in JSON
• we can tell devs to use an array of four normalized floating-point values when they need to describe a color in JSON

anybody care to weigh in on this? my vote's for using an array of normalized floats to communicate color values via JSON.

one more note- the JSON blob you posted has a type tag string of "r", but its "VALUE" is an array with four values. i would expect its VALUE array to only have a single value, as its type tag string only has a single type in it, and this single value would either be a string (if we use hex strings to communicate RGBA values) or an array with four normalized floating-point values (if we use a four-member array to describe a color value).

[bltzr]

it is conceptually/theoretically impossible for any of the "helper" apps i wrote to "hold" color values

I didn't expect them to (though, wouldn't it be possible, if the "helped" app had a feedback port, mirroring all it's values ? But that's another subject entirely

vdmx's support for publishing OSCQuery values is already complete- anything in a control surface is automatically published.

sorry, I have never used VDMX, and thus don't know how to obtain such data: I just get an empty server, like this:

{
  "DESCRIPTION" : "root node",
  "FULL_PATH" : "\/"
}

Is there a tutorial somewhere on how to create control surfaces ? So I can add it to my test Apps...

the JSON blob you posted has a type tag string of "r", but its "VALUE" is an array with four values.

yes, that's just extracted from MM's reply to the whole namespace query.

i would expect its VALUE array to have (...) an array with four normalized floating-point values

isn't that the case ? (sorry if I'm missing something obvious)

As for the question of the format, I agree that an hex string is less than pretty, and I find the four values a little misleading, so I'm not really sure which I dislike the most. In any case, if we end up using 4 values, shouldn't they be an array of four 8-bit ints ? (which is probably the closest to what the data actually is)

[bltzr]

so I'm not really sure which I dislike the most

of course I don't have a better solution, so don't take this as a complaint (if it looks like one, that's maybe just because I'm French ;-p )

[dlublin]

Hey bltzr,

In VDMX there is a Workspace Inspector window, which has a plugins tabs, from there you can add a Control Surface plugin and add elements to it.

There is a quick demo of using a Control Surface / OSCQuery Client plugins in VDMX here: https://vimeo.com/282551062/7123060456

[bltzr]

thanks @dlublin I'll look into that!

[jcelerier]

we can tell devs to use an array of four normalized floating-point values when they need to describe a color in JSON

I'm voting for this - as this allows using better precision, in this age of 10-bit colors and various HDR gimmicks :p

[mrRay]

"isn't that the case ? (sorry if I'm missing something obvious)"

nope, but the difference is subtle and definitely easy to miss! an OSC node with the type tag string "r" expects to receive OSC messages with a single color-type value. because the OSC node only expects a single color-type OSC value, the OSCQuery protocol should return an array with a single member for its VALUE attribute (one element in the VALUE array per value in the type tag string).

...so if we use strings to represent color values, the VALUE attribute would look something like this:

VALUE: [ "FFFFFFFF" ]

...but if we use a single-array-with-four-floats to represent a color value, the VALUE attribute would look like this:

VALUE: [[ 1., 1., 1., 1. ]]

here's what MM is emitting:

VALUE: [ 1., 1., 1., 1. ]

...i would expect an OSC node with this VALUE attribute to have the type tag string "ffff". we already reported a very similar bug (it's probably the same issue and they probably already fixed it) to the MM devs very recently- once there's some sort of consensus on how the VALUE attribute should be handled for OSC color value types, we'll follow up with them on this (while simultaneously apologizing for the lack of clarity in the proposal).

"(if it looks like one, that's maybe just because I'm French ;-p )"

hah! no worries, we knew just what you meant :)

[jcelerier]

...but if we use a single-array-with-four-floats to represent a color value, the VALUE attribute would look like this:

couldn't "r" just be a synonymous for "ffff" ? likewise for "m" (midi message) and "iii".

[mrRay]

i'm sorry to be dense, but i don't fully understand what you mean by this:

"couldn't "r" just be a synonymous for "ffff" ? likewise for "m" (midi message) and "iii"."

could you please give a quick example that demonstrates the change you're suggesting? specifically, i'm not sure whether you're proposing a change in how the JSON is handled, or a change to the basic OSC types that are accepted, or something else entirely...

[jcelerier]

could you please give a quick example that demonstrates the change you're suggesting? specifically, i'm not sure whether you're proposing a change in how the JSON is handled, or a change to the basic OSC types that are accepted, or something else entirely...

sorry, it was a bit terse ! I was just saying that what MadMapper is emitting in the JSON for a TYPETAG: "r", that is, VALUE: [ 1., 1., 1., 1. ] (as RGBA right?), is what I naively expected too. That is, there is no difference in structure between TYPETAG: "r" and TYPETAG: "ffff" (or TYPETAG: "rr" and TYPETAG: "ffffffff" for what's it's worth) - but ultimately, as long as something is settled I would be happy to implement it, with my preferences being in order :

1. { TYPETAG: "r", VALUE: [ 1., 1., 1., 1. ] }
2. { TYPETAG: "r", VALUE: [ 4294967295 ] } (that is, 0xFFFFFFFF)... hard for the user but easy for the implementer :p
3. { TYPETAG: "r", VALUE: [ [1., 1., 1., 1.] ] }

1000 . { TYPETAG: "r", VALUE: [ "#FFFFFFFF" ] }

[bltzr]

That is, there is no difference in structure between TYPETAG: "r" and TYPETAG: "ffff"

I might have misunderstood, but the OSC spec states that:

r 32 bit RGBA color

Which is, 4 uint8_t values packed into a 32-bits value

Which is a completely different thing than TYPETAG: “ffff" Isn’t it ?

[mrRay]

i think this is the crux of the confusion- this is quoted from the current proposal:

VALUE The value stored with this string is always an array- this array contains one or more values (the number and type of values is described by the "TYPE" attribute). The values returned should be JSON representations of the corresponding OSC value types (if there's no corresponding JSON type- as is the case with the OSC type "infinity", for example- then a JSON "null" may be returned for that value).

...my intent here was for the VALUE object to be an array, and for that array to have one item per type in the type tag string. i think this is a simple, sensible arrangement that allows JSON objects to be used as ad-hoc data types where there is no native JSON type for a given OSC data type.

if we want to treat an "array of four floats" as a JSON replacement for an OSC color value, then the "array of four floats" needs to be inserted in the VALUE array- which would look like this:

VALUE: [[1., 1., 1., 1.]]

and if you have a node that accepts a color and a float, the relevant portions of the JSON would look like this:

TYPE: "rf",
VALUE: [[1., 1., 1., 1.], 1.]

That is, there is no difference in structure between TYPETAG: "r" and TYPETAG: "ffff" (or TYPETAG: "rr" and TYPETAG: "ffffffff" for what's it's worth)

• an OSC node with a type tag string of "r" is expecting a single 32-bit value that can be broken up into four 8-bit values. if you send that node an OSC message with four 32-bit floating point values, it probably won't be able to do anything with the data. from this perspective, there's a big difference between the typetags.
• if we go by the proposal's language then there's also a substantial difference in the structure of the JSON objects between "r" and "ffff"- it's the difference between [[1., 1., 1., 1.]] and [1., 1., 1., 1.].

[bltzr]

OK, thanks for clarifying, @mrRay ! The haze is getting thinner :-)

So, now that this is clear, I guess the 4 values array seems the best option, yes.

Maybe explaining this a little more in details in the specs (or maybe, in a wiki page placed as a link in the specs ?) would help, BTW ? Also maybe by taking the case of colors as an example, once we come an agreement/ find the "less worse“ solution on this one ?

Still, I'm still a little bit dubitative on the normalised float representation of a 4 x uint8_t value. My preference would go for something more like this:

TYPE: "r",
VALUE: [[255, 255, 255, 255]]

so it can reflect the fact that the OSC formatting will cause precision loss from float-based internal formats. Then we could use OVERLOADS to, well, overload this, something along this line:

TYPE: "r",
VALUE: [[255, 255, 255, 255]],
OVERLOADS:
    TYPE: "[ffff]",
    VALUE: [[1., 1., 1., 1.]],

NB: not sure this is a valid JSON or strictly following the specs, but you get the idea, right ? also, putting all four floats into an array is a proposal-shaped question: would that be more consistent to do it this way ? If it is, then, if we have a compound node such as the one you proposed, we could have:

TYPE: "rf",
VALUE: [[255, 255, 255, 255], 1.],
OVERLOADS:
    TYPE: "[ffff]f",
    VALUE: [[1., 1., 1., 1.], 1.],

Not sure at all this is relevant, though. Does that make any kind of sense ?

Anyway, if you guys find the float notation as default to be the best option, then I'll be just fine with it.

Opinions ?

[bltzr]

Then we could use OVERLOADS to, well, overload this, something along this line:

This would allow to send float-precision colors across OSCquery-compliant applications that support them, while keeping compatibility with more basic OSC implementations that stick with the 4x8-bit color representation. What do you think ?

[bltzr]

Going again through the thread, and even if that's @jcelerier 1000th choice, I find this one:

{ TYPETAG: "r", VALUE: [ "#FFFFFFFF" ] }

to be the most explicit and less confusion-prone (because it clearly states what the OSC port is expecting) - the hex-to-string is ugly, I have to admit, but, as @jcelerier said,

in this age of 10-bit colors and various HDR gimmicks

that, at least, brings us back to the actual limitations of the protocol we're using - 8 bit colors, that's what it does

so, maybe that will encourage us to move towards normalised units and so on...

[mrRay]

pascal, you make a good point. as much as i share jcelerier's distaste for its aesthetics, i have to admit that a string with four pairs of hex values is closest to the native OSC data type, sticks to the "one JSON value per OSC value" pattern in a less confusing manner than an array, and may at least have a chance of being recognized by someone who's going to be looking at the actual protocol traffic.

i like this ranked-choice thing, so here are my picks:

1. { "TYPETAG": "r", "VALUE": [ "FFFFFFFF" ] }
2. { "TYPETAG": "r", "VALUE": [ [1., 1., 1., 1.] ] }

[bltzr]

don't get me wrong, I also find this pretty ugly, but at least it's correct. So my ranking would be:

1. { "TYPETAG": "r", "VALUE": [ "#FFFFFFFF" ] }
2. { "TYPETAG": "r", "VALUE": [ "FFFFFFFF" ] }
3. { "TYPETAG": "r", "VALUE": [[255, 255, 255, 255] ] }
4. { "TYPETAG": "r", "VALUE": [ [1., 1., 1., 1.] ] }

And, again, I'm all for using OVERLOADS here, if one wants to keep the internal floating-point precision.

[jcelerier]

I suppose { "TYPETAG": "r", "VALUE": [ "FFFFFFFF" ] } is the choice with most acceptance then. I don't really see the point of adding a "#' : I'd wager that most people which will actually go look into the JSON will know that it is an OSC color type with the "r", but that not many people will go look directly at the json in the first place because introspection tools like @mrRay offers already offer a better way to introspect this. The relevant category would be implementors but I suppose they would at least read the spec at some point :p

Now, I think a good thing for the sake of better color precision would be to encourage implementations not to use the unofficial OSC "r" type (just checked, it's not even in the required types of the 1.1 spec) but instead to use N floats and a UNIT/EXTENDED_TYPE/whatever that semantically describes the set of values of the oscquery node. But that's outside the scope of this discussion.

[lov]

I would go with what @bltzr suggested, in that order. @jcelerier I would use the '#' too just to make clear we are expecting a hex color value.

Also, @jcelerier OSC "r" was always official, it is part of the 1.0 spec: http://opensoundcontrol.org/spec-1_0

[jcelerier]

@lov uh ? it says explicitely in your link that it is non-standard:

Some OSC applications communicate among instances of themselves with additional, nonstandard argument types beyond those specified above. OSC applications are not required to recognize these types; an OSC application should discard any message whose OSC Type Tag String contains any unrecognized OSC Type Tags.

[jcelerier]

For reference it isn't handled by liblo (https://github.com/radarsat1/liblo/blob/master/lo/lo_osc_types.h) which is AFAIK the reference OSC implementation

[lov]

@jcelerier Yeah, well, it is non-standard for sure, and you don't need to support that, but also "An application that does use any additional argument types must encode them with the OSC Type Tags in this table"

So I would say they are optional, (non-standard) parameters instead of unofficial, that is all! :)

[bltzr]

I would use the '#' too just to make clear we are expecting a hex color value.

yeah, I was about to write a precision about this, but then you guys sent in a bunch of messages.

So, the point, for me, of adding # (or it could 0x, I don't care at all, I just think we need something to make it clear it's hex) is not really visible in the example we gave, ie. { "TYPETAG": "r", "VALUE": [ "FFFFFFFF" ] } and is not even in such other values as e.g. { "TYPETAG": "r", "VALUE": [ "1A56F188" ] } because we have some octets > 8 but, for low-luminance values, such as { "TYPETAG": "r", "VALUE": [ "88888888" ] } this would become much more ambiguous, IMHO

While, adding a '#' removes any kind of ambiguity: { "TYPETAG": "r", "VALUE": [ "#88888888" ] } { "TYPETAG": "r", "VALUE": [ "#12658255" ] }

I agree that this is not everybody has to look at frequently, as you advocated for, @jcelerier . But we all also know that, when something goes wrong, we end up looking into the not-so-human-readable formats, and then it's always desirable to have as few sources of ambiguities as possible. Isn't it ?

[bltzr]

For reference it isn't handled by liblo (https://github.com/radarsat1/liblo/blob/master/lo/lo_osc_types.h) which is AFAIK the reference OSC implementation

Once we reach an agreement on this issue, shouldn't create an issue there advocating for them adding this support (and/or, maybe even better, making a pull request to add it) ?

[mrRay]

first of all- it seems like there's a general consensus that the hex notation is acceptable. i'll update the proposal accordingly today (and i'll include the # designator, so values will look like "#FFFFFFFF").

"just checked, it's not even in the required types of the 1.1 spec"

is there actually a 1.1 spec? i don't think there is- there used to be a conference paper that the official OSC site said "reflected the 1.1 vision", but the links to it are broken. i've read it, as i'm sure have all of you- here it is again in case you want to review it:

https://web.archive.org/web/20160817172811/http://cnmat.berkeley.edu/system/files/attachments/Nime09OSCfinal.pdf

...does that read like a spec, or does that read like a conference paper discussing a pending update to the spec? is it presented like an official spec on the site, or is it presented like a conference paper? take this passage from the paper- which explicitly mentions an official 1.1 spec that appears to be completely missing:

3.5.3 OSC 1.1 recommended and legacy optional types We have added more optional reserved types including ways to describe complex numbers, double precision floating point, matrices, vectors and units qualification. These may be found in the specification and are omitted here in the interests of brevity.

i don't think you can "skip over" important things like lists of data types "for the sake of brevity" in a spec...

"So I would say they are optional, (non-standard) parameters instead of unofficial, that is all! :)"

well put! as a side note, it seems like the OSC array type tags ("[" and "]") are also optional/non-standard.

"For reference it isn't handled by liblo (https://github.com/radarsat1/liblo/blob/master/lo/lo_osc_types.h) which is AFAIK the reference OSC implementation"

i know liblo has been around for a long time- but i have never heard it described as an "OSC reference implementation". i wasn't aware that any OSC lib had that designation...

"But that's outside the scope of this discussion."

i couldn't agree more- i wonder if any work's being done on OSC 2.0...

[bltzr]

i wonder if any work's being done on OSC 2.0...

I'm very very (very!) unsure about this! AFAIK Adrian Freed has retired quite recently, and he was leading this project (wasn't he ?), no real conversation has happenned for months (years ?) on the OSC_dev list, and the last mention I heard of a potential version 2.0 of OSC was in 2006 at NIME. Then there was this paper at NIME 2009, and also a meeting at IRCAM in 2010 (I was presenting Minuit there) where there seemed to be no real momentum from the people at CNMAT to move this forward. So I guess we'll have to make do with the current state of OSC 1.0 for a while from now on.

[bltzr]

Maybe once the dust has settled from this bunch of ongoing conversations, we should send a mail to the OSC_dev list to see if someone reacts ?

I sent an email announcing ongoing work on OSC Query exactly 3 years ago (3 years, duh !) on an email thread about queries. And BTW someone (salsaman) replied that:

Oh, OK. It would have been a nice courtesy to have notified people on this list so that we could have followed the discussion there.

Apparently the list server is dead, so I can't point you to it online. But not much was said then.

[mrRay]

sure, sounds good...for the record, i did notify them that i was working on this query protocol- i posted to the osc_dev list asking for input/feedback (they had some good suggestions!), and i even linked directly to this github repos in late january of 2014. they probably forgot or got busy, totally understandable...

[bltzr]

are we done with this one ?

that seems to be the case, since the proposal has been updated with all that was suggested here, AFAIK