HebiRobotics / HebiCam

MATLAB utility for streaming video acquisition (h264, mjpeg, etc.)
Other
12 stars 5 forks source link
java matlab

HebiCam

Introduction

HebiCam is a MATLAB class that supports live video acquisition from a variety of sources. It is similar in functionality to MATLAB's IP Camera support package, but provides support for a wider range of formats. HebiCam uses JavaCV, and thus supports all formats that are supported by OpenCV and FFMpeg, including h264 and mjpeg streams. USB cameras are supported on Windows and OSX.

Prerequisites

HebiCam has been tested on Windows7/8, Ubuntu 16.04, and OSX 10.12 using MATLAB 2015a and 2016a. However, it should work on all MATLAB versions after 2009a. It does not require any particular Toolboxes.

Note that in our testing we found problems connecting to USB cameras on Ubuntu due to issues with Video4Linux2 drivers, so USB cameras are only supported on Windows and OSX.

Installation

Warning: Problems when used with other Java libraries

Typically the underlying Java libraries are loaded automatically at the first call. However, there is a limitation in MATLAB that prevents Java libraries to be loaded once any Java object is instantiated. Thus, if you are using other Java libraries, e.g., MatlabInput or the Hebi API, you will need to pre-load the libraries manually before using them.

HebiCam.loadLibs();

Sample Usage

Axis IP Camera (mjpeg)

% Replace url with your device specific stream
url = 'http://<ip address>/mjpg/video.mjpg?resolution=640x480';
cam = HebiCam(url);

Axis IP Camera (h264) with forced grayscale

% Replace url with your device specific stream
url = 'rtsp://<ip address>/axis-media/media.amp?videocodec=h264&resolution=640x480';
cam = HebiCam(url, 'ImageMode', 'Gray');

Windows USB Camera

clear cam; % make sure device is not in use
cam = HebiCam(1);
imshow(cam.getsnapshot());

Acquisition of a single frame

[image, frameNumber, timestamp] = getsnapshot(cam);

Live display of continuously acquired frames

figure();
fig = imshow(getsnapshot(cam));
while true
    set(fig, 'CData', getsnapshot(cam)); 
    drawnow;
end

Sample Use Cases

How it works

The computationally intensive nature of video decoding can be a problem for languages like MATLAB, which limit users to a single thread. MATLAB does offer bindings for other languages (e.g. Java) that enable background threading, but converting high-resolution images back into a MATLAB readable format (e.g. byte[][][]) can be prohibitively expensive. This project gets around these limitations by using Java/C++ to acquire images, and shared memory to get these images into MATLAB.

This enables accessing high quality (1080p h264) video streams with almost no overhead (<50us) to the main MATLAB thread. However, in practice we usually use 640x480 resolution images for any actual computer vision tasks. Synchronization of shared memory is done via Java locks.

Workflow

Installation from Source Code