Support for key/value attributes on Participant

[davidzhao]

This picks up after #728

Key/value attributes lets us:

• allow various components to provide more information about the participant (i.e. SIP including the phone number)
• allow Agents to communicate their own status to the end user via RTC
• enable applications to update specific fields without overriding others

[changeset-bot[bot]]

🦋 Changeset detected

Latest commit: a4d0f8917e98aa3b6976d4d603ad6d0be74e98e8

The changes in this PR will be included in the next version bump.

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

💥 An error occurred when fetching the changed packages and changesets in this PR

``` Some errors occurred when validating the changesets config: The package or glob expression "github.com/livekit/protocol" specified in the `fixed` option does not match any package in the project. You may have misspelled the package name or provided an invalid glob expression. Note that glob expressions must be defined according to https://www.npmjs.com/package/micromatch. ```

[paulwe]

for sip we need a way to copy information about the caller from the sip api to the participant. this data is created once when the participant is created and never updated.

for agents we need a way to change the job config after the job is dispatched ie. update the voice or language. this config is initially copied from the job description to the job and can be modified while the job is running. ideally this would work the same way for both room and publisher jobs.

for sip i'm not sure we've considered the privacy implications of broadcasting pii to everyone in the room. caller information should not be part of the participant api. it does not change so we do not need to retransmit it and we should deliver it only to privileged participants. similarly agent config updates should not be delivered to every participant in a room. any private information the agent needs would be publicly visible.

for agents the proposed change does not provide a way to update room jobs or to update one job specifically. developers could implement their own namespaces and try to avoid collisions but this is kludgy...

we should define a new message type SIPCallerInfo and add a repeated field to ParticipantUpdate to deliver it. ParticipantPermission should have a new field can_read_sip_caller_info that enables the session to receive these updates.

for agents we should add signal and agents api methods for updating jobs. this can work the same as the proposed changes to the update participant apis but specify job rather than participant subjects. updates can be delivered to the agent worker ws and dispatched to the running job by the agent client framework.

[boks1971]

Sorry, did not read @paulwe 's comments before bonking this. I think this needs more discussion after reading his comments.

[davidzhao]

I've synced with Paul. He had two concerns:

1. privacy of always exposing the caller's number to everyone in the room

This is a good callout. The decision of exposing the phone number vs not should not be up to us, but it should be up to the developer.

2. building specific APIs for each use case vs general APIs that are used for multiple use cases.

Paul was saying it would be more clear to have specific APIs for each use case. While I don't disagree, I think it becomes a burden on the client teams to implement a lot of different features. For that reason, I personally prefer general constructs.

---

@dennwc for 1, do you think this can be a SIP specific option? for example, a setting to indicate if a user's phone number should be anonymous. Perhaps it should be specified on the trunk or dispatch rule?

[dennwc]

for example, a setting to indicate if a user's phone number should be anonymous. Perhaps it should be specified on the trunk or dispatch rule?

Yes, we already have a hide_phone_number on Dispatch Rule, so we can reuse it. Will need to add the same setting on CreateSIPParticipant too.

[dennwc]

@davidzhao I added SIP-related attributes, should be good to go from my side.

Please check the attribute prefix for SIP, we won't be able to change it later.

[dennwc]

One more note, there's no way currently to set attributes on participant creation, right?

[davidzhao]

One more note, there's no way currently to set attributes on participant creation, right?

ah yeah.. I will add that in a few

[davidzhao]

@dennwc I've added it here. I'll leave the branch open until server support is integrated.

[dennwc]

Also worth adding a changeset.

[davidzhao]

Also worth adding a changeset.

will do!

[lukasIO]

One thing I'm unsure about is using the same permission for both metadata and attributes. From an application level perspective, I imagine it to be nice to be able to differentiate those two. E.g. metadata could be used for things the participant is allowed to update directly, but attributes might make more sense in some cases to be controlled by the application's backend.

[davidzhao]

One thing I'm unsure about is using the same permission for both metadata and attributes. From an application level perspective, I imagine it to be nice to be able to differentiate those two. E.g. metadata could be used for things the participant is allowed to update directly, but attributes might make more sense in some cases to be controlled by the application's backend.

I think for certain attributes, you do want the participant to update: i.e. their current state or preferences. Do you think it makes sense to have another permission canUpdateOwnAttributes ?

[lukasIO]

I think for certain attributes, you do want the participant to update: i.e. their current state or preferences.

yeah, totally agree, for certain use cases (esp. agents) that makes a lot of sense.

Do you think it makes sense to have another permission canUpdateOwnAttributes ?

that's what I was thinking, yeah. Not sure if it's worth it though, we'd probably want a separate signal message then as well just for the attributes...

[davidzhao]

that's what I was thinking, yeah. Not sure if it's worth it though, we'd probably want a separate signal message then as well just for the attributes...

I guess the question is.. do we see any use cases where it makes sense to allow participants to update one, but not the other? If so it might be worth it to separate.

[dennwc]

I tried integrating this change to SIP, and it looks like it will have to send attribute update at least once too. Some headers are available only later during the call, there's no way to bake them into the token. But all SIP attrs mentioned in this PR can be.

[davidzhao]

decided to keep the permissions the same, in order to avoid additional complexity without a use case that requires it. merging.