Bare metal fails over virtual network

[ezbrooks]

I have built the "bare metal" from source on Ubuntu 16.04.7. For testing I am using two network namespace instances connected through a virtual bridge. When I launch the receiver on one namespace node, and the sender on the other node, I get a SignalEncoderTimeOut error. I have confirmed that traffic moves as expected on the virtual interface, and can run the docker version with no error. Logs attached. receive_log.txt send_log.txt

[HoustonHuff]

Hi, I'm ezbrooks's colleague and have been troubleshooting a different part of the system, namely the packet receiving while communicating over Ubuntu network namespace (using the provided demo); we're seeing a report of each of the video frames being dropped in the webrtc.log, such as below:

(remote_estimator_proxy.cc:151): {"mediaInfo":{"audioInfo":{"audioJitterBufferDelay":1.7976931348623157e+308,"audioJitterBufferEmitedCount":18446744073709551615,"concealedSamples":18446744973709551615,"concealmentEvents":18446744073709551615,"echoReturnLoss":1.7976931348623157e+308,"echoReturnLossEnhancement":1.7976931348623157e+308,"estimatedPlayoutTimestamp":9223372036854775807,"totalSamplesReceived":18446744073709551615,,"totalSamplesSent":18446744073709551615},"videoInfo":{"framesCaptured":18446744073709551615,"framesDecoded":18446744073709551615,"framesDroped":18446744073709551615,"framesReceived":18446744073709551615,"framesSent":18446744073709551615,"fullFramesLost":18446744073709551615,"hugeFramesReceived":18446744073709551615,"hugeFramesSent":18446744073709551615,"keyFramesReceived":18446744073709551615,"keyFramesSender":18446744073709551615,"partialFramesLost":18446744073709551615,"videoJitterBufferDelay":1.7976931348623157e+308,"videoJitterBufferEmittedCount":18446744073709551615}},"pacerPacingRate":100000.0,"pacerPaddingRate":100000.0,"packetInfo":{"arrivalTimeMs":1687529358363,"header":{"headerLength":28,"paddingLength":0,"payloadType":125,"sendTimestamp":45865,"sequenceNumber":1115,"ssrc":1665672282},"lossRates":0.0,"payloadSize":1079}}

I've been tracing back through the project to see where I could find where these StatCollect values are originally incremented so that I can debug the frame dropping from there, but as far as I can tell though those keys as they're named by StatCollect aren't found anywhere else besides the build files, though I'm fairly certain that what I'm looking for is in the core files somewhere. Would you happen to know which files are directly responsible for the receiver-sender stream? That's ultimately what I've been looking for.

[jeongyooneo]

Hey @HoustonHuff, thanks for the issue!

The metrics framesDroped, fullFramesLost, partialFramesLost seem to be initialized with the default values of StatsCollect (each entry of statscollect is initialized with default values which corresponds to ULONG_MAX, i.e. 18446744073709551615).

You're right that StatsCollect values are not incremented anywhere else. You need to measure the metrics you'd like to instrument and selectively populate the entries of stats_collect_, if you want to.

However, I'd highly recommend you to use the set of WebRTC-instrumented metrics printed on the receiver-side logs, such as WebRTC.Video.DroppedFrames.Receiver.

I've added https://github.com/OpenNetLab/AlphaRTC/issues/101 to clear up the confusion. Thanks again for reporting this!

[jeongyooneo]

Thanks for trying AlphaRTC, @ezbrooks!

The SignalEncoderTimedOut error in your receiver log is common when the sender starts to send the video stream, and itself does not show any call failure.

To check the call quality, you can use the receiver- and sender-side statistics at receive_log.txt and send_log.txt. For example, observing the sender-side statistics from your send_log.txt, the call seems to be run with 300Kbps bandwidth (from (bitrate_allocator.cc:394): Current BWE 300000), and the sender-side media bitrate WebRTC.Video.MediaBitrateSentInBps is avg. 255976, looks like there's no special issue w.r.t. the call.

(send_statistics_proxy.cc:667): WebRTC.Video.InputFramesPerSecond periodic_samples:9, {min:10, avg:10, max:11}
WebRTC.Video.SentFramesPerSecond periodic_samples:9, {min:10, avg:10, max:10}
WebRTC.Video.SentToInputFpsRatioPercent 100
WebRTC.Video.NumberOfPauseEvents 0
WebRTC.Video.PausedTimeInPercent 0
WebRTC.Video.BitrateSentInBps periodic_samples:9, {min:175528, avg:264104, max:278944}
WebRTC.Video.MediaBitrateSentInBps periodic_samples:9, {min:169592, avg:255976, max:270384}
WebRTC.Video.PaddingBitrateSentInBps periodic_samples:9, {min:0, avg:0, max:0}
WebRTC.Video.RetransmittedBitrateSentInBps periodic_samples:9, {min:0, avg:0, max:0}
WebRTC.Video.FecBitrateSentInBps periodic_samples:9, {min:0, avg:0, max:0}
Frames encoded 196
WebRTC.Video.DroppedFrames.Capturer 0
WebRTC.Video.DroppedFrames.EncoderQueue 0
WebRTC.Video.DroppedFrames.Encoder 1
WebRTC.Video.DroppedFrames.Ratelimiter 0
WebRTC.Video.DroppedFrames.CongestionWindow 0

Feel free to re-open this issue if you need more help!