SVO recording via API on machine without GPU

greenfieldvision commented 3 years ago

I have a ZED Mini camera and I'm trying to use the Python API to record an SVO file on an old laptop without GPU. Camera initialization fails with "NO GPU DETECTED" and there are further error printouts:

CUDA error at Camera.cpp:94 code=999(cudaErrorUnknown) "void cdecl sl::Camera::close(void)" CUDA error at Camera.cpp:148 code=999(cudaErrorUnknown) "void cdecl sl::Camera::close(void)" CUDA error at Camera.cpp:183 code=999(cudaErrorUnknown) "void cdecl sl::Camera::close(void)" CUDA error at sl_zed/CameraUtils.hpp:691 code=999(cudaErrorUnknown) "void __cdecl sl::ObjectsDetectorHandler::clear(void)" CUDA error at sl_zed/CameraUtils.hpp:707 code=999(cudaErrorUnknown) "void cdecl sl::ObjectsDetectorHandler::clear(void)" CUDA error at sl_zed/CameraUtils.hpp:713 code=999(cudaErrorUnknown) "void cdecl sl::ObjectsDetectorHandler::clear(void)" CUDA error at sl_zed/CameraUtils.hpp:716 code=999(cudaErrorUnknown) "void __cdecl sl::ObjectsDetectorHandler::clear(void)" CUDA error at sl_zed/CameraUtils.hpp:718 code=999(cudaErrorUnknown) "void cdecl sl::ObjectsDetectorHandler::clear(void)" CUDA error at sl_zed/CameraUtils.hpp:726 code=999(cudaErrorUnknown) "void cdecl sl::ObjectsDetectorHandler::clear(void)" CUDA error at Camera.cpp:198 code=999(cudaErrorUnknown) "void cdecl sl::Camera::close(void)"

I'm following the example from https://www.stereolabs.com/docs/video/recording/#using-the-recording-api , setting the compression mode to sl.SVO_COMPRESSION_MODE.LOSSLESS, which is supposed to be CPU based. Also setting depth mode to sl.DEPTH_MODE.NONE to make absolutely sure ZED doesn't attempt to use the GPU. The error seems to come from object detection, which AFAIU is not even available on ZED Mini. Can the code be updated to remove unnecessary dependencies?

Code snippets:

self.camera = sl.Camera()

init_params = sl.InitParameters()
init_params.depth_mode = sl.DEPTH_MODE.NONE
init_params.camera_resolution = sl.RESOLUTION.HD720
init_params.camera_fps = 15

error_code = self.camera.open(init_params)
if error_code != sl.ERROR_CODE.SUCCESS:
    print("failed to start the camera:", error_code)

recording_params = sl.RecordingParameters(
   video_filename=...,
   compression_mode=sl.SVO_COMPRESSION_MODE.LOSSLESS,
)

error_code = self.camera.enable_recording(recording_params)
if error_code != sl.ERROR_CODE.SUCCESS:
   print("failed to enable recording:", error_code)

The alternative is to use the ZED Explorer tool, which works on this old laptop, but is CPU intensive and cannot be customized to save less frequently, eg at 1 FPS.

adujardin commented 3 years ago

The ZED SDK can't be used without a CUDA-compatible GPU, even just for recording.

ZED Explorer is the only way, and even then it's advised to have an Nvidia GPU to be able to use the hardware accelerator to encode in H264/5. Otherwise, as you saw the image-based encoding is CPU heavy. The LOSSY (based on jpeg) should be faster to encode than lossless (png or zstd based) if that's an acceptable option.

github-actions[bot] commented 2 years ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment otherwise it will be automatically closed in 5 days

Fuzl commented 1 year ago

Hello @adujardin, It's been more than a year after this Issue but I have the same need : record from a ZED on a lightweight edge hardware that doesn't have GPU and postprocess it later on with GPU-enabled hardware.

The goal would be to be able to use the ZED with minimum video acquisition costs, and GPU hardware is costly.

Has there been any changes on that since the issue was closed ? When you say recording is CPU intense without GPU, could you be a bit more specific ? CPU cadences have gone a way up and with multicores and specific CPU-oriented frameworks, a lot has been made to alleviate the GPU burden/costs.

Following another lead, we are considering using the ZED on a GPU-enabled smartphone. Would you recommend that path and do you have any documentation/tutorial/dummy projects/good examples in this direction ? It could be Android or iOS or even better Flutter.

Many thanks !

adujardin commented 1 year ago

Hi @Fuzl,

There have been no major changes since then. The image-based compressions are heavy for both the CPU and storage as it as encodes fairly big images 1920x1080x2 for instance at 30fps, depending on the mode and platform in PNG, JPEG or ZSTD. The only way to record a compatible ZED SDK format is to record an SVO, either :

using the SDK with a CUDA-capable GPU on an X86 Desktop or an Nvidia Jetson, no other configurations are supported for hardware-accelerated encoding currently
or the ZED Explorer tool on any X86 machine (or Nvidia Jetson). In that case, it will try to detect the presence of a compatible Nvidia GPU and failover on the CPU encoder-based options.

Smartphones or other architecture will not work for SVO recording. It is also worth noting that acquiring the USB 3.0 stream at full speed may be very challenging on mobile devices if not impossible.

That said, it is technically possible to record a video of the side-by-side images with the compression of your choice and a custom implementation, but this will not be compatible with the ZED SDK.

Fuzl commented 1 year ago

Thanks for your prompt reply.

We require 5fps videos so CPU could be affordable I think. But we do need gyroscope/accelerometre so unless there is a non-gpu way to retrieve those we need the .svo format. I am going to try the ZED Explorer way.

adujardin commented 1 year ago

If you need the IMU data, you can have a look at this project https://github.com/stereolabs/zed-open-capture I don't think there's a recording feature implemented and it will not be compatible with the ZED SDK, but we will have access to all the sensors data + images. This project doesn't require CUDA at all

Fuzl commented 1 year ago

Ah, I didn't mention I was trying this on Raspberry Pi / ARM, and it seems the official setup stops when it detects ARM.

Is there any way around this ?
It would be faster if I didn't need to go to the zed-open-capture option right now.

Thanks for the help !

adujardin commented 1 year ago

We don't support any other ARM boards than Nvidia Jetsons, I guess if it's aarch64 you could try the jetson installer but I think a lot of specific libraries will be missing. If it's an option I think it may be easier to buy an Nvidia Jetson, like a Nano devkit, you'll then be able to run the ZED SDK on it. Alternatively, an entry-level intel NUC or a laptop can work well for recording SVOs.

Please note that you'll need an Nvidia GPU PC to process the SVO and run the ZED SDK, regardless of the compression mode used or the ZED SDK settings.

stereolabs / zed-python-api

SVO recording via API on machine without GPU #183