mbalamat / aiortc-broadcast-demo

Broadcast your camera's video feed to many peers.
Apache License 2.0
6 stars 4 forks source link

add a function to process audio track #1

Open LogWell opened 4 days ago

LogWell commented 4 days ago

Thank you for the sample code.

I want to do some simple transformations on audio and video.

There are sample codes for video operations: https://github.com/aiortc/aiortc/blob/main/examples/server/server.py#L22

I don't know how to implement the process function in AudioProcessingTrack(just for some simple operations) in this file: https://github.com/aiortc/aiortc/blob/main/examples/webcam/webcam.py#L83

【I tried something similar to the video, but the code doesn't work】

Will you do me a favor!

server.py.txt index.html.txt client.js.txt

LogWell commented 4 days ago

return frame is OK, but others are NG!

class AudioProcessingTrack(MediaStreamTrack):
    kind = "audio"

    def __init__(self, track):
        super().__init__()  
        self.track = track

    async def recv(self):
        frame = await self.track.recv()
        new_frame = await self.process(frame)
        return new_frame

    async def process(self, frame):
        # return frame  #! OK

        gain = 1.5
        for p in frame.planes:
            samples = np.frombuffer(p.to_bytes(), dtype=np.int16)
            samples = np.clip(samples * gain, -32768, 32767)
            p.update(samples.tobytes())

        new_frame = AudioFrame(format=frame.format, layout=frame.layout, samples=frame.samples)
        new_frame.pts = frame.pts
        new_frame.sample_rate = frame.sample_rate
        new_frame.time_base = frame.time_base
        return new_frame