udayraghuvanshi03 / Pedestrian-Detection-using-RGB-Camera-and-LiDAR-sensor-fusion

4 stars 0 forks source link

Sensor fusion on matlab #2

Open alolcreed opened 11 months ago

alolcreed commented 11 months ago

I need to synchronize lidar and camera together to be able to detect distance of people and objects when I run the code the camera detects that am moving but the lidar is far behind or delayed how can I make that camera and lidar moving on the same pace % lidarFig = subplot(1, 2, 2); % Create subplot for lidar data countinueStreaming = true;

while countinueStreaming % Record the current timestamp before capturing the image

% Capture the image from the Jetson camera hardware. I = snapshot(cam);

% Record the timestamp after capturing the image

% Call to detect method [bboxes, scores, labels] = detect(detector, I); personIndices = find(labels == 'person'); personBboxes = bboxes(personIndices, :);

% Convert categorical labels to cell array of character vectors labels = cellstr(labels);

% Annotate detections in the camera image. img = insertObjectAnnotation(I, 'rectangle', bboxes, labels);

% Display camera image imshow(img); hold on;

% Read data from Velodyne Lidar pc = read(lidar);

% Remove ground plane groundPtsIndex = segmentGroundFromLidarData(pc, 'ElevationAngleDelta', 0, ... 'InitialElevationAngle', 0); nonGroundPts = select(pc, ~groundPtsIndex); imPts = projectLidarPointsOnImage(pc, intrinsics, LidarToCam); scatter(imPts(:, 1), imPts(:, 2), 10, 'g', 'filled');

% Display lidar data % pcshow(nonGroundPts.Location, 'Parent', lidarFig);

if ~isempty(personIndices) for i = 1:numel(personIndices) index = personIndices(i); personBbox = bboxes(index, :);

    % Process lidar data for each detected person
    [lidarBbox, ~, boxUsed] = bboxCameraToLidar(personBbox, nonGroundPts, intrinsics, ...
        camToLidar, 'ClusterThreshold', 2, 'MaxDetectionRange', [1, 10]);
    [distance, nearestRect, idx] = helperComputeDistance(personBbox, nonGroundPts, lidarBbox, ...
        intrinsics, LidarToCam);

    % Print distance information
    fprintf('DistOfPerson %d: %.2f meters\n', i, distance);

    % Update image with bounding boxes
    % im = updateImage(display, im, nearestRect, distance);
    % updateLidarBbox(display, lidarBbox)
end

else % Process lidar data for other objects [lidarBbox, ~, boxUsed] = bboxCameraToLidar(bboxes, nonGroundPts, intrinsics, ... camToLidar, 'ClusterThreshold', 2, 'MaxDetectionRange', [1, 10]); [distance, nearestRect, idx] = helperComputeDistance(bboxes, nonGroundPts, lidarBbox, ... intrinsics, LidarToCam);

% Print distance information
fprintf('DistOfObj: %.2f meters\n', distance);

% Update image with bounding boxes
% im = updateImage(display, im, nearestRect, distance);
% updateLidarBbox(display, lidarBbox)

end

hold off; end

alolcreed commented 11 months ago

i need it to be live as well and is the problem in matlab or my laptop and if I change to jetson nano it would be synchronized by itself