Multi-Femto Mega camera connecting issue from the network

[Anil-Bhujel]

First of all, thanks for the nice SDK. I have setup the environment and run the OrbeecSDK_ROS2 in the Ubuntu machine and working fine. However, I didn't see the launch file for connecting multiple Femto Mega camera from the network. There is a launch file for multi-camera connected through USB but couldn't see for network ones. Could you please suggest about it how we can get it? Also, I would like to know about the compressed images from the camera. What compression technique is used and can we set the compression level during launch time?

Anil Bhujel, Research Associate, Michigan State University

[jian-dong]

Hi @Anil-Bhujel

Thank you for using the Femto Mega camera. We're glad to hear that the OrbeecSDK_ROS2 is running smoothly on your Ubuntu machine.

Regarding your question about connecting multiple Femto Mega cameras over a network, I would first like to know how many cameras you plan to use and the resolution at which you plan to capture data. Our RGB streams use compression techniques such as MJPEG, H.264, and H.265, while depth streams are not compressed and are only available in Y16 format. Please note that H.264 and H.265 compression methods are only supported over the network.

We will also soon be able to give you an example of setting up multiple cameras over the network.

Best regards,

[Anil-Bhujel]

@jian-dong Thank you for your prompt response. Indeed, we are working to build a large computer vision dataset for Precision Livestock Farming (PLF) to attract computer vision and AI communities towards PLF. We do series of experiments with different combination of camera numbers. Currently, we have 6 in our laboratory but we could use it 2 or more at a time and plan to record Compressed color image and uncompressed depth topics from multiple cameras into single Rosbag2 file. I am happy to hear that you are willing to assist us. Overall and to your earlier response, we have following queries and requirements,

1. We have to synchronize the timestamp for all cameras within a network with the system clock or camera clock.
2. How we can select the compression techniques among MJPEG, H.264, and H.265 and which one you recommend along with compression ratios (if any). Please keep in mind that our experimental livestock is located in remote and there is limited space available to store long run videos with high frame rate. So plan to compress as much as possible while recording and revert to original quality for analysis in the lab.
3. Can we get back the compressed images to original quality during playback of Rosbag2 file for analysis using any third-party decoder algorithm?
4. Does OrbbecSDK_ROS2 utilizes the NVIDIA GPU similar to Isaac_ros_compression?
5. To optimize the recording spaces we plan to record videos at full fps when the animal is there with certain motion activities but at reduced fps (let's say 1 fps) when there is no movement. Is there any mechanism you have applied to detect motion or such kind of event and activates camera accordingly?

Sorry for a bit long queries, this is quite critical experiments and we are happy to use Orbbec camera. FYI, we set the IPs to each camera connected via network for single camera. When we connected all cameras in the network and run ros2 launch orbbec_camera femto_mega.launch.py enumerate_net_device:=true, it only launches single camera having the least IP address.

Once again thank you for your time and support.

[jian-dong]

Hi @Anil-Bhujel Thank you for your detailed feedback. Since some of the technical questions you've raised are quite complex, I will consult with more specialized colleagues to provide you with a comprehensive response. Meanwhile, to better assess your needs, please provide the following information:

1. Maximum Number of Cameras: How many cameras do you plan to use simultaneously at most during your experiments?
2. Resolution and Data Streams: What resolution and frame rate will you use? Do you need only color images, or also depth and IR data streams?
3. Recording Duration: How long do you plan to record during each experiment?
4. Desired Bag File Size: Considering your storage limitations, what is the maximum acceptable size for the bag file?

This information will help us conduct a more accurate assessment. Thank you for your cooperation, and we will get back to you as soon as possible.

[Anil-Bhujel]

Hi @jian-dong Please find the response here in italic text.

1. Maximum Number of Cameras: How many cameras do you plan to use simultaneously at most during your experiments? Mostly 2 but could be 4 at max.
2. Resolution and Data Streams: What resolution and frame rate will you use? Do you need only color images, or also depth and IR data streams? In animal movement time maximum possible frames with high quality is preferred (but depending upon the other constraints, at least 15 fps with HD quality) and without animal movement period HD frame with 1 fps is enough. We can compress the images before storing and minor loss in the reconstruction will be acceptable. Yes, we need all RGB, depth, and IR images along with other metadata (timestamp, frame rate, resolution, and other camera configuration).
3. Recording Duration: How long do you plan to record during each experiment? At least a week for an experiment but the bag file must be manageable with a minute or 5 or 10 minutes duration. Although we are going to record video throughout the animal cycle, unavoidable breaking in the video recording is acceptable.
4. Desired Bag File Size: Considering your storage limitations, what is the maximum acceptable size for the bag file? We are planning to record one-minute-long Bag file for easy processing. However, we can go longer than it (5-minutes, 10-minutes) as we have 100TB NAS device to store the data but not prefer larger single bag file than 10GB.

I hope it clarifies you. If you have further let me know.

[Anil-Bhujel]

@jian-dong Just in case of data stream, we can exclude IR stream if it increases the complexity and memory.