ceaglest
commented
4 years ago Hi @astaranowicz,
We haven't rushed to support this use case because recording on the subscriber side results in the lowest quality video (compared to the media server or publisher perspective). If you want to manage the I420 format conversion then you can record remote video (please be sure to get user consent of course).
If you are using Vp8 simulcast then you must be in a Group Room and I would recommend our Recordings API to record the Vp8 video track on the media server. Server recordings are going to be better quality than subscriber recordings since the server always has the best simulcast layers and doesn't transcode the video. Your subscriber might experience packet loss that would be missing in the server recording, and there is no guarantee that you are receiving the top quality layers (you can use a bandwidth profile + high priority to make high quality more likely).
- According to #286, a custom render is created for the purpose of saving the video frames from the 2nd video track. However, it seems that I420 isn't supported. Using #123 as an example, a user was able to convert I420 into NV12 which then can be saved to the MP4 container (Would a MOV file also work?). But this process seems overly complex with a lot of overhead. Even in putting libyuv into the project seems a bit complex for a single function call. Since that issue was from a couple of years ago, is there any updates on this conversion taking place in Twilio? Is this functionality exposed in any way in from the Twilio SDK?
No we don't support converting from I420 -> NV12 natively. If you can stand some extra memory and CPU use then vImageConvert_420Yp8_Cb8_Cr8ToARGB8888 is built in and can convert I420 -> BGRA for AVAssetWriter with no 3rd party code.
- If the 2nd video track can be configured to use the H264 codec, then using the custom render from #286 would work out of the box. I don't see a way to force the 2nd video track to take on a different codec as the room was setup using the VP8 codec. Is there a way to configure the codec of the video track without doing anything to the room settings or interfering with the 1st video track?
Setting codec preferences on a per-Track basis isn't supported yet so I wouldn't recommend this approach. Doing a format conversion is way cheaper than encoding, sending and decoding a second video track.
Best, Chris
astaranowicz
Help Request Related Issues: #123 and #286 Scenario:
Local user starts up a room with a VP8 Simulcast Video Track with a remote user joining the room with the same connection settings. Local user wants to record the incoming video track. The remote user will start up a 2nd video track in which the local user will record into an MP4. The 2nd video track will not be viewed, it will all happen in the background with a notification going to the remote user to accept/deny the video recording. Currently both users will have a main video track that ensures high quality using the VP8 connection. The 2nd optional video track is created and destroyed on command. While the main video track keeps a high quality connection by having a reduced frame rate or frame size, the 2nd video track can be dynamic.
There are two paths: 1) According to #286, a custom render is created for the purpose of saving the video frames from the 2nd video track. However, it seems that I420 isn't supported. Using #123 as an example, a user was able to convert I420 into NV12 which then can be saved to the MP4 container (Would a MOV file also work?). But this process seems overly complex with a lot of overhead. Even in putting libyuv into the project seems a bit complex for a single function call. Since that issue was from a couple of years ago, is there any updates on this conversion taking place in Twilio? Is this functionality exposed in any way in from the Twilio SDK?
2) If the 2nd video track can be configured to use the H264 codec, then using the custom render from #286 would work out of the box. I don't see a way to force the 2nd video track to take on a different codec as the room was setup using the VP8 codec. Is there a way to configure the codec of the video track without doing anything to the room settings or interfering with the 1st video track?