Should unique names be able to be generated by the TE?

[ghost]

Consider the scenario of a multiplexer that is incapable of decoding/encoding image data, but it can pass the full sequence to the host terminal. It runs multiple terminals, each of which will use image names that may conflict. The multiplexer has a couple options it might want to do:

1. Parse every GoodImageProtocol message (henceforth GIP) for the "name" parameter, and rewrite those for the application maintaining its own map of {TE name, application name} association.
2. Pass the GIP messages and responses directly to the TE, and let the application request unique names, assuming the application wants to do the "upload, then render" pathway and not the "upload and render" pathway.

The second is nicer, and makes for a multiplexer that doesn't even need to know ANYTHING about GIP, other than "passhru these sequences".

Also -- There is typo in table in section 11.3 image-x-offset used twice.

[wez]

FWIW, my two cents on this after implementing the kitty image protocol: I think it would be great if the GIP had a robust way for an application to avoid conflicting with other applications when it comes to naming/identifying images, and for that to be the only way to do it.

The kitty protocol has a simple way to deal with this: the uppercase-i I id parameter requests assignment of an unused internal ID that won't collide with existing images, and that can then be used to reference them. That technique wouldn't work in the multiplexer use case though, because two applications being passed through could still use the same I value and then have a bad time.

I think option 2 in the OP seems like a reasonable way to resolve this.

I would humbly suggest that the design used to resolve this should probably be the only way to resolve/identify images, so that the GIP doesn't allow for conflicts to exist. It would suck for that one app using the "dumb" approach to break the others.

[dankamongmen]

The kitty protocol has a simple way to deal with this: the uppercase-i I id parameter requests assignment of an unused internal ID that won't collide with existing images, and that can then be used to reference them. That technique wouldn't work in the multiplexer use case though, because two applications being passed through could still use the same I value and then have a bad time.

wait it's my understanding that you send the I, and what's returned is a true 'i'. i.e. the I is a request that you're not guaranteed to receive.

[dankamongmen]

The kitty protocol has a simple way to deal with this: the uppercase-i I id parameter requests assignment of an unused internal ID that won't collide with existing images, and that can then be used to reference them. That technique wouldn't work in the multiplexer use case though, because two applications being passed through could still use the same I value and then have a bad time.

wait it's my understanding that you send the I, and what's returned is a true 'i'. i.e. the I is a request that you're not guaranteed to receive.

The kitty protocol has a simple way to deal with this: the uppercase-i I id parameter requests assignment of an unused internal ID that won't collide with existing images, and that can then be used to reference them. That technique wouldn't work in the multiplexer use case though, because two applications being passed through could still use the same I value and then have a bad time.

so yeah i'm pretty sure (will verify) that two programs send I=420, and one or zero of them get back i=420 (both get back I=420, so they can correlate it with the request). checking now...

[dankamongmen]

yeah that's exactly how it works, which is the only way that scheme can work, right? i sent up an I=1 and displayed it. i then sent up a different image with i=1,I=1, and got back:

fGi=2,I=1,p=1;OK

with the new image loaded at i=2

[dankamongmen]

so long as the terminal atomically arbitrates these requests, basically as the kernel does file descriptors for a multithreaded process, i believe @wez that the kitty scheme is sufficient for this case. am i missing something?

[wez]

My read of the kitty protocol (which honestly wasn't super deep--shocking!) is that an application can keep using I and not look at the returned i and have the potential to collide that way.

For that multiplexer case, if two apps pick the same I value and one of them loses the race to use it, there's no clear guidance on how to successfully negotiate a winning I value aside from guess a new number and try, and repeat until you win.

In my mind there are two use cases for this sort of id allocation:

• Transmit and tell me an id in a response that I promise to read
• Transmit with an id that I provide, that "can't" collide with another app. I can't/won't read a response, and I need to reference this transmission in a later request

The second case is the one that feels potentially fraught with collision potential.

Maybe I've spent too long working on source control systems, but I'd feel happier using something more akin to eg: the content hash of the image data to reference an image, or perhaps something like a uuid as an alternative thing (which might be easier if you're streaming data and don't yet know the content hash) to minimize the chances of collision.

FWIW, I'd be totally fine with not allowing that second case. I think the argument for it is for simplistic clients that can't read the TE response from the first case, but in my mind, those clients are likely to be using transmit-and-place equivalent requests, rather than doing something more fancy that might need to use separate transmission and placement requests.

[dankamongmen]

My read of the kitty protocol (which honestly wasn't super deep--shocking!) is that an application can keep using I and not look at the returned i and have the potential to collide that way.

this seems a bug in the application?

which kinda implies "it is a bug to not use I and the resulting i unless you can be certain there is nothing else running, nor any preexisting images". which would be unfortunate, since then you need an RTT between uploading and presenting.

[wez]

The thing is that the spec allows for I (without the returned i) to be used to reference an earlier transmitted image, or at least, that's how I read it.

[dankamongmen]

The thing is that the spec allows for I (without the returned i) to be used to reference an earlier transmitted image, or at least, that's how I read it.

oh hrmm. i never read it that way. i'd consider that a flaw in the protocol, as it makes no sense. the obvious way it's supposed to be used is transmit I, get back i, use i henceforth.

i don't like that, though, because it means you can't reference the image again until there's been a RT. i'd say the way to do this is, rather than looking up an image identifier, request a process identifier at the beginning. or hell, use your PID as a namespace id (probably doesn't work on windows).

i don't use terminal multiplexers, and am not well-informed regarding their means of operation. are they in a position to know what process is talking to them, and proxy the request, transparently mapping conflicting ids so that they do not conflict? but then what if two processes are cooperating and want to refer to the same id?

[ghost]

i don't use terminal multiplexors, and am not well-informed regarding their means of operation. are they in a position to know what process is talking to them, and proxy the request, transparently mapping conflicting ids so that they do not conflict?

They are indeed in that position. From the POV of the application, it (ideally) has no means of knowing that it isn't speaking to pty with xterm on the other side. From xterm's POV, it has no idea that there are multiple panes/windows (e.g tmux) that each think they have their own xterm to talk to.

[dankamongmen]

They are indeed in that position. From the POV of the application, it (ideally) has no means of knowing that it isn't speaking to pty with xterm on the other side. From xterm's POV, it has no idea that there are multiple panes/windows (e.g tmux) that each think they have their own xterm to talk to.

then i would like to believe apps can assume they're the only thing on-screen, and multiplexers -- responsible for breaking that abstraction -- could include functionality to transparently broker exclusive IDs. the case i mentioned where two applications want to share an ID seems worth considering, maybe, but definitely pretty esoteric.

[wez]

then i would like to believe apps can assume they're the only thing on-screen, and multiplexers -- responsible for breaking that abstraction -- could include functionality to transparently broker exclusive IDs

A counterpoint to this: a recent thread in the wezterm matrix/element/whatever-it-is-called room last week was complaining about the hurdles required to compel tmux to pass through the iTerm2 image protocol OSC sequence to the TE on the other end of tmux (today, it swallows various "unrecognized" sequences, and needs the application to re-encapsulate them in order to pass through the mux). Teaching the multiplexer to actively interpret and re-encode both sides of the GIP feels like a bigger and less compelling request to make of the multiplexer author than just passing through data without modification.

[dankamongmen]

A counterpoint to this: a recent thread in the wezterm matrix/element/whatever-it-is-called room last week was complaining about the hurdles required to compel tmux to pass through the iTerm2 image protocol OSC sequence to the TE on the other end of tmux (today, it swallows various "unrecognized" sequences, and needs the application to re-encapsulate them in order to pass through the mux). Teaching the multiplexer to actively interpret and re-encode both sides of the GIP feels like a bigger and less compelling request to make of the multiplexer author than just passing through data without modification.

certainly. all of this is kinda abstract really, as tmux is resolutely opposed to allowing any kind of graphics as far as i'm aware.

[ghost]

tmux is opposed to having to do any kind of image management / pixel manipulation translation. nicm might be open to translating image names and screen coordinates parameters, so long as the image data itself can remain a direct passthru.

[dankamongmen]

cool, well being able to work at all is more important than transparent id-proxying atop working, so i withdraw my suggestion =].