Description \ Library for live video playback with ultra low latency (below 10ms) on android devices. Supports playback of .h264 encoded live video data transmitted via UDP encapsulated in RAW or RTP and simple file playback. \ Latency data (see example for more information)
Device | Encoder | avgTotalDecodingTime |
---|---|---|
Galaxy s9+ | rpi cam | 8.0ms |
Galaxy s9+ | x264 | 11.5ms |
Pixel 3 | rpi cam | 11.2ms |
Pixel 3 | x264 | 10.5ms |
HTC U11 | rpi cam | 8.6ms |
HTC U11 | x264 | 10.1ms |
Example App \
This library has been optimized for low latency and tested on a wide variety of devices, including those running FPV-VR for wifibroadcast. The example library also contains test cases that can be executed on the 'gooogle firebase test lab'. These tests include feeding the decoder with faulty NALUs, created by a lossy connection. (e.g. wifibroadcast).
The 2 most important factors for low latency are
However, all native code needed for creating, starting and stopping the decoding process are exposed via the JNI, so you can use the lib without writing c/c++ code. The VideoPlayer class purposely has no pause/resume functions,since this library is for live video playback. Playback works on a 'best effort' principle, e.g. as soon as a receiver is created nalus are forwarded to the LowLagDecoder, but no frames can be decoded until enough I-frame data was received. Make sure to use a low enough I-frame interval with your h264 encoder. When receiving corrupted data (e.g from a lossy connection) the decoder will still generate frames if possible.
Structure:
Setup Dependencies\ There are 2 ways to use VideoCore in your Project \ 1 Declaring Dependency via Jitpack: jitpack.io \ :heavy_plus_sign: Easy \ :heavy_minus_sign: cannot browse native libraries \ Gradle example:
allprojects {
repositories {
jcenter()
maven { url "https://jitpack.io" }
}
}
dependencies {
implementation 'com.github.Consti10:LiveVideo10ms:v1.1'
}
2 Forking the repo and including sources manually: \ :heavy_plus_sign: browse native libraries \ :heavy_plus_sign: modify code
include ':VideoCore'
project(':VideoCore').projectDir=new File('..\\LiveVideo10ms\\VideoCore')
and modify the path according to your download file
implementation project(':VideoCore')
See FPV-VR as an example how to add dependencies.