search

EricssonResearch / bowser

A WebRTC browser for iOS developed in the open
BSD 2-Clause "Simplified" License
208 stars 65 forks source link

Performance bottleneck in image copying #36

Open ijsf opened 9 years ago

ijsf commented 9 years ago

I've been testing OpenWebRTC in combination with Bowser on an iPad Mini Gen 1 (armv7), and noticed that image buffer copying/moving takes up a significant portion of time on this particular device.

When profiling with Time Analyzer, a _platform_memmove call that originates from WebCore seems to take up at least 12% of CPU when displaying local video on demo.openwebrtc.io:38080 (not calling), and this only gets worse once a call is initiated.

I'm not quite sure why this bottleneck is appearing for local video, as the video screen is very small in comparison to the window size. Can anybody shine some light on this?

stefanalund commented 9 years ago

The current rendering in Bowser is a hack, really. The video element gets replaced by a repeatedly updated image element. This is highly inefficient as bitmap (I think) images are transferred over localhost.

The proper way to do this is by replacing the hack with OpenGL rendering, the same way native OpenWebRTC apps render video. We have a solution that is 90% complete.

ijsf commented 9 years ago

Great. Would be nice to test this so I can start evaluating and improving OpenWebRTC performance on armv7.

stefanalund commented 9 years ago

@pererikb has it locally. Can you please share?

pererikb commented 9 years ago

https://github.com/pererikb/openwebrtc/commits/overlay https://github.com/pererikb/bowser/commits/overlay

stefanalund commented 9 years ago

Thanks Per-Erik, looks pretty good!

ijsf commented 9 years ago

@pererikb Tried it here and forced the overlay to be used so I could test with demo.openwebrtc.io. Unfortunately, the local video frame no longer displays anything.

I get the following message in the output:

gldisplay gstgldisplay.c:182:gst_gl_display_new: Could not create display. user specified (NULL) (platform: (NULL)), creating dummy