IntelRealSense / RealSenseID

Intel® RealSense™ ID SDK
https://intelrealsense.com/facial-authentication/
Apache License 2.0
100 stars 58 forks source link

Provide low resolution preview for use on devices with limited cpu power #121

Open mdisg opened 3 years ago

mdisg commented 3 years ago

We want to display the preview image on our device which has a display pixel resolution of 800x480. Using the current preview modes MJPEG_1080P or MJPEG_720P causes high cpu load on that device (nearly 100%). The preview is very slow and has a big delay.

Would it be possible for you to provide a lower resolution e.g. 240p/360p/480p?

mdisg commented 3 years ago

I did some tests with the library and adjustments in the code and came up to a good solution which improves the performance on the device a lot.

For testing I added the following new function to StreamConverter.cc and replaced the Call to DecodeJpeg in Buffer2Image with CopyRawJpegBuffer.

bool StreamConverter::CopyRawJpegBuffer(Image* res, buffer frame_buffer)
{    
    ::memcpy(res->buffer, frame_buffer.data, frame_buffer.size);
    res->height = 1280;
    res->width = 704;
    res->stride = 0;
    res->size = frame_buffer.size;
    return true;
}
    case MJPEG:
        try
        {
            res->metadata = ExtractMetadataFromMDBuffer(md_buffer,true);
            //bool decode_success = DecodeJpeg(res, frame_buffer);
            bool decode_success = CopyRawJpegBuffer(res, frame_buffer);
            return decode_success;
        }

Now the JPEG is passed directly to my calling routine which can also pass it directly over to a HTTP MJPEG stream to be displayed on the device. CPU usage is a lot better than with the JPEG handling in DecodeJpeg.

Would be nice to get that kind of preview into one of your next releases so that MJPEG is passed from the camera to the preview callback of the user by adding a new PreviewMode like PreviewMode::MJPEG_720P_DIRECT for example.