search

lynckia / licode

Open Source Communication Provider based on WebRTC and Cloud technologies
http://lynckia.com/licode
MIT License
3.1k stars 1.01k forks source link

A/V Desync REC #528

Open kekkokk opened 8 years ago

kekkokk commented 8 years ago

Hi guys, I'm experiencing a big (for my use case) problem. The recordings are quite often in desync with audio/video after a couple of minutes, sometimes it starts in desync.

I tried everything, last test scenario was: t2.medium EC2 ubuntu 14.04 fresh install licode cloned from upstream/master

tried 20 different recordings half with chrome (53 latest version) half with canary (55 latest version).

few recordings are perfect, few are in complete desync (up to 4 sec).

I'm using these recordings to broadcast to another media server the stream, but if I can't rely on the quality of those I'm lost.

I haven't found a pattern which broke the mechanism yet. Tomorrow I'll try to build some old commits, do you have an idea on where to start to debug? Thanks, Francesco

jnoring commented 8 years ago

Most likely issue is licode doesn't sync streams using RTCP feedback, so it's possible to have drift. To fix that, someone needs to add code to start correcting timestamps based on information in sender reports

kekkokk commented 8 years ago

Ok, I had this fear. I'm trying looking into this. Up to now I can intercept and read all the rtcp Sr packets and read these values. Do you have any suggestion about how to use these data to made a simple algorithm?

jnoring commented 8 years ago

For unrelated reasons i've started work on terminating RTCP completely, so yes. :) that said, it's not on a branch available and my local branch has deviated hugely from master licode, so.... :( I need to get with ging/licode people and work something out.

kekkokk commented 8 years ago

Nice, I’d love to know more about this issue! Would you be available to continue this conversation via email? My contact is work@francescodurighetto.com. Have a nice day

simonemazzoni commented 8 years ago

Hi, are there some news about this problem? I'm interested too. I tried to implement an RTPC processing to address the A/V sync problem, but it isn't completely working yet.

@jnoring any chance to get in touch with you or have a look at your implementation? It would be very useful to me.

@lodoyun any plan to address the problem?

Thanks, Simone

jnoring commented 8 years ago

@simonemazzoni I don't have an implementation of timestamp syncing based on RTCP. I do have code that correctly parses the necessary information out of RTCP sender reports. Is that what you're wanting?

simonemazzoni commented 8 years ago

@jnoring I have the code to analyze RTCP packets too (it is already committed in licode master if I'm not wrong). I am more interested in how to use them. I found several papers explaining theoretically how to use RTCP packets to address A/V sync problem, but none of them implemented it. I also had a look at chromium source code, which implements this mechanism, but obviously reverse the mechanism from code is not an easy task.

So, long story short, I am more interested in how to use that data to overcome this problem. How to do the math, to be clear :)

jnoring commented 8 years ago

I haven't done it yet so I'm not sure about that, but http://www.ciscopress.com/articles/article.asp?p=705533&seqNum=6 is the best article I've seen thus far?

simonemazzoni commented 8 years ago

Yes, I used that very same article as a bible, and I obtained a "pseudo-working" solution, but I'm not sure it is completely correct. The reason I asked you about that, is because I saw that you implemented the current recording on licode, so I guessed you are confident with that code.

jnoring commented 8 years ago

Unfortunately I've never been confident of the current implementation. I made it a lot better with the changes I made, but without RTCP info, it's definitely lacking.

simonemazzoni commented 8 years ago

By saying

I made it a lot better with the changes I made

you mean the changes you have in your branch, or the changes that you made time ago that are in the current master branch?

Anyway, is in your plans to update that code with RTCP info?

jnoring commented 8 years ago

The changes I made some time ago that are in the current master branch. Before it would constantly record completely corrupted video.

Ultimately yes, I'll fix recording with RTCP info, but I have no ETA on that. Right now I'm working on adding RTCP termination to licode, which I'm nearly done with.

simonemazzoni commented 8 years ago

Good!

Can I ask you to drop me a line when its done? Or you plan to pull request all to ging/licode?

BTW what you mean with

adding RTCP termination to licode ?

Feel also free to contact me if you want to discuss my current solution.

jnoring commented 8 years ago

Doubtful there will be a pull request into licode, because again, my local branch has deviated enough that it's nearly impossible.

Licode doesn't terminate RTCP, which means a publisher receives no feedback unless there's a subscriber present. That leads to all sorts of strange issues.

simonemazzoni commented 8 years ago

Doubtful there will be a pull request into licode, because again, my local branch has deviated enough that it's nearly impossible.

Ok, I understand the problem. It would be nice to have your feedback when you've done with RTCP and A/V sync.

Licode doesn't terminate RTCP, which means a publisher receives no feedback unless there's a subscriber present. That leads to all sorts of strange issues.

Now I understand what you said. This is a problem too. I planned to have a look also at it, but at the moment I have other priorities.

zevarito commented 8 years ago

@jnoring can you link to the commit in your local fork that fixes this issue?

2016-09-28 13:53 GMT-03:00 jnoring notifications@github.com:

Doubtful there will be a pull request into licode, because again, my local branch has deviated enough that it's nearly impossible.

Licode doesn't terminate RTCP, which means a publisher receives no feedback unless there's a subscriber present. That leads to all sorts of strange issues.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ging/licode/issues/528#issuecomment-250227592, or mute the thread https://github.com/notifications/unsubscribe-auth/AACgak7h5uqIo93BqZ86THxlr1u0jaGkks5qupt9gaJpZM4J-ATo .

Alvaro

kekkokk commented 8 years ago

@lodoyun

https://github.com/ging/licode/pull/532

So, I made a first implementation of what can be our rtcp a/v sync but I need help to find bugs, as I wrote in the description. I'm new to C. Maybe you can open a specific branch.

Also, It would be very nice to see @jnoring implementation of rtcp termination. Do you have any public fork or can you just give us some parts of code?

Thank you guys!

lodoyun commented 8 years ago

@kekkokk See my comments in the PR #532 . Thanks for the work you put into it! @jnoring We should work something out, RTP/RTCP termination is in our roadmap but still haven't got around to do it. Any suggestion on how to proceed?

jnoring commented 8 years ago

@lodoyun I'll look into getting you access to our git repo and/or sharing the code directly with you. I wish my local branch had not deviated so hugely, but....c'est la vie.

kekkokk commented 8 years ago

Ok, I found a much better implementation of the rtcp sync. I'll PL as soon as I can