API Call id

[ClosetGeek-Git]

I think it would be helpful for each call on the api to have a resulting request/call id. at this point it's impossible to line responses up with their respective caller. This becomes even more of an issue when considering UDP specific issues such as dropped packets and lack of guaranty in response packet order. This id would open paths to resolve these issues.

[ideoforms]

Each response includes its address and the identifier of the entity that it corresponds to (e.g. track and clip index), which make it possible to line up responses with their caller. Can you give an example of a scenario in which you're not able to achieve what you want with the current approach? What client software are you using?

[MartinSahlen]

I think what he means is just simply something that would allow to assign a given response to a given request - if so should be. in some cases, like when there is a listener callback, it would not exist, but in the cases where you issue a request it would be nice to be able to associate the response to the request.

[MartinSahlen]

I think this is not strictly necessary (it's fine as it is), but I also see that this would be a nice to have thing. the only slight issue I see is that it would probably warrant some larger change to the format to separate between the arguments and the request id since we dont have i.e headers that would be a natural place for such a thing

[ClosetGeek-Git]

I'm developing an event driven PHP client using ReactPHP and oscpack. I have Each API finished except Devices. In my case the possibility is real that a call can be made on the same address, potentially the same track/clip etc. This would leave two callbacks waiting for the the response of two calls, albeit on the same resources. In this case it only makes sense to use second response because it is likely to be the most recent.

Since none of the AbletonOSC API use a 'N' osc type as parameters I'd suggest maybe adding the ability to send data at the tail of each osc packet with a N field separating api parameters and custom fields that could follow. Checking for the null field wouldn't cause a loss in performance and it wouldn't break the current implementation from what I've seen.

[ideoforms]

I see now, thanks for the additional detail @ClosetGeek-Git — I can see how this would constrain high-contention scenarios with multiple different UI elements/processes accessing the same object.

I've just had a look through the OSC spec (and O2, which adds some extensions) to see if there are any standard solutions proposed, but there really isn't anything except O2's suggestion of including a "reply-to" address in a query, that would allow each individual query to return a response to a different OSC address. I don't really like this approach as it add a lot of overhead to the queries in general.

I think appending some custom data to the query that would be returned in the response is a pretty reasonable design decision. My reservation is how much complexity it might add to the AbletonOSC code - it's already quite difficult to parse the callback structures.

From a quick look, it looks like it would be trivial to add to get queries (just need to append *params to value on handler.py line 45). However, if we were going down this path, I would also want to add the same support to listeners for consistency — I could definitely imagine a scenario in which multiple UI elements want to track the same property of an object, which would mean introducing specifiable IDs to start_listen also, which is a bit more complicated.

An alternative might be to use something like HTML-style anchor suffixes, e.g.

/live/clip/get/name#fe03 0 0 would send the reply to /live/clip/get/name#fe03

But it's also quite nonstandard and would add complexity to the OSC parser.

[ClosetGeek-Git]

Made a dirty example of what I'm imagining here https://github.com/ClosetGeek-Git/AbletonOSC/tree/custom-fields-example, I'm not very familiar with python so forgive me if it could be more elegant. I read the O2 spec a while ago but just now found their reference implementation, it looks pretty promising.

[ClosetGeek-Git]

Being an RPC framework at it's core this is something of a requirement in IMO. Another issue would be ensuring error responses make it to the appropriate caller. If two requests for volume levels get their responses mixed it's not a real problem, but if one call causes an error it's important that it's response would be routed to the appropriate calling method.

[MartinSahlen]

Interesting, I looked at the relevant section too (https://www.hindawi.com/journals/wcmc/2019/8424381/). It seems that the authors don't fully see O2 or OSC as a true RPC system, but if such cases exists it should not be a strict dependency of the protocol, but it should facilitate it. Pretty good paper all in all.

[MartinSahlen]

This is probably a bigger refactor, but is there a reason why the list type is not being used to send and receive parameters? Feels that would make it more future proof, akin to how a rest API would be better to return {"data": [1,2,3]} vs [1,2,3].

[MartinSahlen]

And sorry for barging in on this thread, I see was even bringing up elements that was already answered regarding the O2 protocol. Re #82 - it would be great to have a discord or something to discuss (if it is not already set up?)

[ClosetGeek-Git]

I have a feeling their idea of a reply-to address is due to the distributed nature of many osc applications. For example, a serial device can have a midi sequencer connected to it, and the sequencer can have multiple synthesizer connected to it, etc. Same scenario would exist in lighting. In these cases routing replies/responses to the original point of the call would be more involved. The options would be either by routing backwards through the devices that forwarded the packet, something similar to the reverse path used in SMTP, or an routing scheme where a response could be sent to a specific point via an explicit address.

[ClosetGeek-Git]

I'm far from an expert with python, but while working with my custom vars concept I've found that in order to work with listeners that it needed to be moved into the global handlers in handler.py. At this point I have it working with both getters and listeners for track.py, I'm sure other apis will be similar. I'm also using a property in osc_server.py to hold each "last call" made into AbletonOSC (self.last_call = (message.address, message.params)) so that errors emitted on /live/error include track id and custom vars. Any hints would be nice, I will make a branch with my changes as well as a pull request when finished.