YOLO reshape array problem using ZED2

[ericlai0323]

I'm facing an issue when using ZED2. I've configured various parameters, but upon execution, I encounter an error. The problem seems to be related to the image format conversion when YOLO frontend receives the image. Could you please provide any solutions or insights regarding this error?

Thank you!!

Below is my config and launch file:

%YAML:1.0

num_threads: 0  # 0  Use the max number of threads of your device.
                #    For some devices, like HUAWEI Atlas200, the auto detected max number of threads might not equivalent to the usable numble of threads. (Some cores(threads) might be reserved for other usage(NPU) by system.)
                # x  It is proper that 1 < x < MAX_THREAD.
                # For now, this parameter is relevant to grid-detector to run in parallel. 

#common parameters
imu: 1
static_init: 0 # fix_depth should be set 1 if static_init is set 1
imu_topic: "/zed/zed_node/imu/data"
image_topic: "/zed/zed_node/rgb/image_rect_color"
depth_topic: "/zed/zed_node/depth/depth_registered"
output_path: "/home/ericlai/testdynamicvins_ws/src/Dynamic-VINS/output"

#RGBD camera Ideal Range
depth_min_dist: 0.3
depth_max_dist: 6
fix_depth: 0    # 1: feature in ideal range will be set as constant 

frontend_freq: 30 # It should be raised in VO mode(without IMU).
num_grid_rows: 7
num_grid_cols: 8

#camera calibration
model_type: PINHOLE
camera_name: camera
image_width: 640
image_height: 360
  #TODO modify distortion

distortion_parameters:
   k1: 0.0
   k2: 0.0
   p1: 0.0
   p2: 0.0
projection_parameters:
   fx: 277.9840393066406
   fy: 277.9840393066406
   cx: 326.0340576171875
   cy: 193.9207305908203

# Extrinsic parameter between IMU and Camera.
estimate_extrinsic: 0   # 0  Have an accurate extrinsic parameters. We will trust the following imu^R_cam, imu^T_cam, don't change it.
                        # 1  Have an initial guess about extrinsic parameters. We will optimize around your initial guess.
                        # 2  Don't know anything about extrinsic parameters. You don't need to give R,T. We will try to calibrate it. Do some rotation movement at beginning.
#If you choose 0 or 1, you should write down the following matrix.
#Rotation from camera frame to imu frame, imu^R_cam
extrinsicRotation: !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data: [0.9999,-0.0120,-0.0025,
         0.0120,0.9999,-0.0028,
         0.0025,0.0028,1.0000] # ZED2 SDK Camera-IMU Transform

#Translation from camera frame to imu frame, imu^T_cam
extrinsicTranslation: !!opencv-matrix
   rows: 3
   cols: 1
   dt: d
   data: [0.0017, 0.0230, -0.0002] # ZED2 SDK Camera-IMU Transform

#feature traker paprameters
max_cnt: 150           # max feature number in feature tracking. It is suggested to be raised in VO mode.
min_dist: 25           # min distance between two features
freq: 10                # frequence (Hz) of publish tracking result. At least 10Hz for good estimation. If set 0, the frequence will be same as raw image
F_threshold: 1.0        # ransac threshold (pixel)
show_track: 1           # publish tracking image as topic
equalize: 0             # if image is too dark or light, trun on equalize to find enough features
fisheye: 0              # if using fisheye, trun on it. A circle mask will be loaded to remove edge noisy points

#optimization parameters
max_solver_time: 0.04  # max solver itration time (ms), to guarantee real time
max_num_iterations: 8   # max solver itrations, to guarantee real time
keyframe_parallax: 10.0 # keyframe selection threshold (pixel); if system fails frequently, please try to reduce the "keyframe_parallax"  

#imu parameters       The more accurate parameters you provide, the better performance
acc_n: 0.5          # accelerometer measurement noise standard deviation. 
gyr_n: 0.3         # gyroscope measurement noise standard deviation.   
acc_w: 0.001         # accelerometer bias random work noise standard deviation. 
gyr_w: 0.0001       # gyroscope bias random work noise standard deviation.  

g_norm: 9.81       # gravity magnitude

#unsynchronization parameters
estimate_td: 1    ##########                  # online estimate time offset between camera and imu
td: 0.0                           # initial value of time offset. unit: s. readed image clock + td = real image clock (IMU clock)

#rolling shutter parameters
rolling_shutter: 0                      # 0: global shutter camera, 1: rolling shutter camera
rolling_shutter_tr: 0.0               # unit: s. rolling shutter read out time per frame (from data sheet).

#loop closure parameters
loop_closure: 0                    # start loop closure
fast_relocalization: 0            # useful in real-time and large project
load_previous_pose_graph: 0        # load and reuse previous pose graph; load from 'pose_graph_save_path'
pose_graph_save_path: "/home/ericlai/testdynamicvins_ws/src/Dynamic-VINS/output/pose_graph" # save and load path

#visualization parameters
save_image: 0                   # enable this might cause crash; save image in pose graph for visualization prupose; you can close this function by setting 0
visualize_imu_forward: 0        # output imu forward propogation to achieve low latency and high frequence results
visualize_camera_size: 0.4      # size of camera marker in RVIZ

#Only Consider Moving Objects
dynamic_label: ["person", "cat", "dog", "bicycle", "car","bus"]

<launch>
    <arg name="config_path" default="$(find vins_estimator)/../config/stereolabszed2/stereolabszed2.yaml" />
    <arg name="vins_path" default="$(find vins_estimator)/../config/../" />

    <remap from="/camera/color/image_raw" to="/zed/zed_node/rgb/image_rect_color" />
    <include file="$(find yolo_ros)/launch/yolo_service.launch">
    </include>

    <arg name="manager_name" default="nodelet_manager_pc" />
    <node pkg="nodelet" type="nodelet" name="$(arg manager_name)" args="manager" output="screen" />

    <node pkg="nodelet" type="nodelet" name="EstimatorNodelet" args="load vins_estimator/EstimatorNodelet $(arg manager_name)" output="screen">
        <param name="config_file" type="string" value="$(arg config_path)" />
        <param name="vins_folder" type="string" value="$(arg vins_path)" />
    </node>

    <!-- <node pkg="nodelet" type="nodelet" name="PoseGraphNodelet" args="load pose_graph/PoseGraphNodelet $(arg manager_name)" output="screen">
        <param name="config_file" type="string" value="$(arg config_path)"/>
        <param name="visualization_shift_x" type="int" value="0"/>
        <param name="visualization_shift_y" type="int" value="0"/>
        <param name="skip_cnt" type="int" value="0"/>
        <param name="skip_dis" type="double" value="0"/>
    </node> -->

</launch>

[ericlai0323]

This is the error message.

[ERROR] [1706936026.447987]: Error processing request: cannot reshape array of size 921600 into shape (360,640,3)
['Traceback (most recent call last):\n', '  File "/opt/ros/noetic/lib/python3/dist-packages/rospy/impl/tcpros_service.py", line 633, in _handle_request\n    response = convert_return_to_response(self.handler(request), self.response_class)\n', '  File "/home/ericlai/testdynamicvins_ws/src/yolo_ros/src/yolo_bridge/yolo_bridge.py", line 67, in request_handle\n    img0 = rnp.numpify(req.image)\n', '  File "/opt/ros/noetic/lib/python3/dist-packages/ros_numpy/registry.py", line 42, in numpify\n    return conv(msg, *args, **kwargs)\n', '  File "/opt/ros/noetic/lib/python3/dist-packages/ros_numpy/image.py", line 72, in image_to_numpy\n    data = np.fromstring(msg.data, dtype=dtype).reshape(shape)\n', 'ValueError: cannot reshape array of size 921600 into shape (360,640,3)\n']
[ERROR] [1706936026.449774164]: Service call failed: service [/yolo_service] responded with an error: error processing request: cannot reshape array of size 921600 into shape (360,640,3)

[jianhengLiu]

Hi, it seems that the width and height didn't set right. As 921600 looks like 480x640x3.