Open pyu1538 opened 3 years ago
It is awesome that you integrated Lyra into WebRTC for real-time calls. However, those MOS scores seem very low compared to what we experience in our WebRTC calls. Does the audio on these calls sound reasonable compared to when running a file through it? I just want to rule out some integration issue. Regardless, in the meantime we are working on improving Lyra's quality and will share it as soon as we have something.
My subjective listening is good, the audio is clear and sound good. Is the Lyra algorithm not friendly with the PESQ algorithm or kind of this issue?
That is a great point! Generative models tend to perform poorly with objective metrics like PESQ, POLQA or VISQOL, even if perceptually they sound good, because they aren't optimized to maintain certain alignment or phase. And Lyra is no exception of this rule. Please see section 3.3 and table 1 on our 2017 paper.
Our listening tests show that the Lyra 1.2.0 release improves quality significantly, but I am guessing that the MOS score would still be low, since it is also based on a generative model.
@pyu1538 Hi, Yu, just curious how the Lyra2 runs on a phone call now, since it only supports 3 bitrates , 3200, 6000, 9200. Most of the calls have higher bitrates if opus is used. Thanks.
I do not have the MOS test box now, I will comment here if I figure out a way to measure it.
I am assuming you have seen our blog post, but just to summarize: From our MUSHRA listening tests, LyraV2@3.2k has similar quality than Opus@10k, LyraV2@6k has similar quality than Opus@13k, and LyraV2@9.2k has similar quality than Opus@14k. But I agree that many calls use Opus at higher bitrates, like 32kbps, and there LyraV2 as it is today won't be able to compete.
I have integrated Lyra into WebRTC and tested it on two Pixel 3 phones(Android 11), the MOS(measured by Malden red box) of p2p call is only about 2.2, it can't reach the VoIP call standard, do we have a plan to optimize its quality?