livekit / client-sdk-js

LiveKit browser client SDK (javascript)
https://livekit.io
Apache License 2.0
334 stars 143 forks source link

handleTranscription routes to wrong event channel #1176

Closed heilmela closed 1 month ago

heilmela commented 2 months ago

Describe the bug

Expectation

I expected the transcriptionReceived event to be received on the participant who produced the transcript, whether it's a remote or local participant.

Current Behaviour

Transcriptions for remote participants are attached to the agent when it's not the transcription for the local participant. This affects the useTranscription hook in the React library, making it seem impossible to receive transcriptions for any remote participant.

I was unable to get feedback on Slack, so I'm assuming this is not intended. I believe the routing of the packet here is incorrect for the RemoteParticipant: https://github.com/livekit/client-sdk-js/blob/c7b7f9e701b3b484390c423fc74bf0cf04001299/src/room/Room.ts#L1521-L1525 It should not use the RemoteParticipant passed in (which is the agent), but instead use the identity to find the correct remote participant as well.

If this routing is not intended, I'd be happy to submit a PR.

Reproduction

(reproduction is quite heavy and potentially not necessary. I would consider creating one when explicitly demanded)

Logs

No response

System Info

System:
    OS: macOS 14.4.1
    CPU: (12) arm64 Apple M2 Max
    Memory: 109.08 MB / 32.00 GB
    Shell: 5.9 - /bin/zsh
Binaries:
    Node: 20.12.2 
    Yarn: 1.22.19 
    npm: 10.2.1 
    pnpm: 8.15.7 
    Watchman: 2024.05.06.00 
 Browsers:
    Safari: 17.4.1

Severity

serious, but I can work around it

Additional Information

Workaround is basically hook into the EventEngine and route the event stream manually

lukasIO commented 2 months ago

Thanks for the report! Yeah, this is a bug! We have some internal protocol changes pending in that regard as well so I'll include the fix for that in this update and link to the issue here.