Bin-ze
commented
2 months ago The visualization code is offline, and pred_seg_path points to the inference result path on the nuscene dataset using debug_test.py. You can refer to：
https://github.com/Bin-ze/BEVFormer_segmentation_detection/issues/6#issuecomment-1650847977
I got an error IndexError: list index out of range when I visualize it with visual_det_seg.py running
File "visual_det_seg.py", line 528, in
render_sample_data(sample_token_list[id], pred_data=bevformer_results, out_path=f"/root/BEVFormer_segmentation_detection/visual_res_small/{sample_token_list[id]}", seg_list=seg_list)
IndexError: list index out of range
I don't know which path should I put in it as pred_seg_path. pred_seg_path = '/root/BEVFormer_segmentation_detection/pred_seg/'
Let me Know the way to solve this issue
----------------------------------------------------------------- Below my visual_det_seg.py that I am using ...............................................................................
Based on https://github.com/nutonomy/nuscenes-devkit
---------------------------------------------
Modified by Bin_ze
---------------------------------------------
import os import mmcv import cv2 from nuscenes.nuscenes import NuScenes from pyquaternion import Quaternion from nuscenes.utils.geometry_utils import view_points, box_in_image, BoxVisibility from nuscenes.eval.common.data_classes import EvalBoxes from nuscenes.eval.detection.data_classes import DetectionBox from nuscenes.eval.detection.utils import category_to_detection_name from nuscenes.eval.detection.render import visualize_sample from nuscenes.eval.common.utils import boxes_to_sensor import numpy as np import matplotlib.pyplot as plt from nuscenes.utils.data_classes import LidarPointCloud, Box from PIL import Image from matplotlib import rcParams
cams = ['CAM_FRONT', 'CAM_FRONT_RIGHT', 'CAM_BACK_RIGHT', 'CAM_BACK', 'CAM_BACK_LEFT', 'CAM_FRONT_LEFT']
pred_seg_path = '/root/BEVFormer_segmentation_detection/pred_seg/'
det_grid_conf = { 'xbound': [-51.2, 51.2, 0.68], 'ybound': [-51.2, 51.2, 0.68], }
map_grid_conf = { 'xbound': [-30.0, 30.0, 0.15], 'ybound': [-15.0, 15.0, 0.15], }
def padding_seg_to_det(path):
def visualize_sample(nusc: NuScenes, sample_token: str, gt_boxes: EvalBoxes, pred_boxes: EvalBoxes, conf_th: float = 0.30,) -> None: """ Visualizes a sample from BEV with annotations and detection results. :param nusc: NuScenes object. :param sample_token: The nuScenes sample token. :param gt_boxes: Ground truth boxes grouped by sample. :param pred_boxes: Prediction grouped by sample. :param nsweeps: Number of sweeps used for lidar visualization. :param conf_th: The confidence threshold used to filter negatives. :param eval_range: Range in meters beyond which boxes are ignored. :param verbose: Whether to print to stdout. :param savepath: If given, saves the the rendering here instead of displaying. """
def render_annotation( anntoken: str, margin: float = 10, view: np.ndarray = np.eye(4), box_vis_level: BoxVisibility = BoxVisibility.ANY, out_path: str = 'render.png', extra_info: bool = False) -> None: """ Render selected annotation. :param anntoken: Sample_annotation token. :param margin: How many meters in each direction to include in LIDAR view. :param view: LIDAR view point. :param box_vis_level: If sample_data is an image, this sets required visibility for boxes. :param out_path: Optional path to save the rendered figure to disk. :param extra_info: Whether to render extra information below camera view. """ ann_record = nusc.get('sample_annotation', anntoken) sample_record = nusc.get('sample', ann_record['sample_token']) assert 'LIDAR_TOP' in sample_record['data'].keys(), 'Error: No LIDAR_TOP in data, unable to render.'
def get_sample_data(sample_data_token: str, box_vis_level: BoxVisibility = BoxVisibility.ANY, selected_anntokens=None, use_flat_vehicle_coordinates: bool = False): """ Returns the data path as well as all annotations related to that sample_data. Note that the boxes are transformed into the current sensor's coordinate frame. :param sample_data_token: Sample_data token. :param box_vis_level: If sample_data is an image, this sets required visibility for boxes. :param selected_anntokens: If provided only return the selected annotation. :param use_flat_vehicle_coordinates: Instead of the current sensor's coordinate frame, use ego frame which is aligned to z-plane in the world. :return: (data_path, boxes, camera_intrinsic <np.array: 3, 3>) """
def get_predicted_data(sample_data_token: str, box_vis_level: BoxVisibility = BoxVisibility.ANY, selected_anntokens=None, use_flat_vehicle_coordinates: bool = False, pred_anns=None ): """ Returns the data path as well as all annotations related to that sample_data. Note that the boxes are transformed into the current sensor's coordinate frame. :param sample_data_token: Sample_data token. :param box_vis_level: If sample_data is an image, this sets required visibility for boxes. :param selected_anntokens: If provided only return the selected annotation. :param use_flat_vehicle_coordinates: Instead of the current sensor's coordinate frame, use ego frame which is aligned to z-plane in the world. :return: (data_path, boxes, camera_intrinsic <np.array: 3, 3>) """
def lidiar_render(sample_token, data, out_path=None): bbox_gt_list = [] bbox_pred_list = [] anns = nusc.get('sample', sample_token)['anns'] for ann in anns: content = nusc.get('sample_annotation', ann) try: bbox_gt_list.append(DetectionBox( sample_token=content['sample_token'], translation=tuple(content['translation']), size=tuple(content['size']), rotation=tuple(content['rotation']), velocity=nusc.box_velocity(content['token'])[:2], ego_translation=(0.0, 0.0, 0.0) if 'ego_translation' not in content else tuple(content['ego_translation']), num_pts=-1 if 'num_pts' not in content else int(content['num_pts']), detection_name=category_to_detection_name(content['category_name']), detection_score=-1.0 if 'detection_score' not in content else float(content['detection_score']), attribute_name='')) except: pass
def get_color(category_name: str): """ Provides the default colors based on the category names. This method works for the general nuScenes categories, as well as the nuScenes detection categories. """ a = ['noise', 'animal', 'human.pedestrian.adult', 'human.pedestrian.child', 'human.pedestrian.construction_worker', 'human.pedestrian.personal_mobility', 'human.pedestrian.police_officer', 'human.pedestrian.stroller', 'human.pedestrian.wheelchair', 'movable_object.barrier', 'movable_object.debris', 'movable_object.pushable_pullable', 'movable_object.trafficcone', 'static_object.bicycle_rack', 'vehicle.bicycle', 'vehicle.bus.bendy', 'vehicle.bus.rigid', 'vehicle.car', 'vehicle.construction', 'vehicle.emergency.ambulance', 'vehicle.emergency.police', 'vehicle.motorcycle', 'vehicle.trailer', 'vehicle.truck', 'flat.driveable_surface', 'flat.other', 'flat.sidewalk', 'flat.terrain', 'static.manmade', 'static.other', 'static.vegetation', 'vehicle.ego'] class_names = [ 'car', 'truck', 'construction_vehicle', 'bus', 'trailer', 'barrier', 'motorcycle', 'bicycle', 'pedestrian', 'traffic_cone' ]
print(category_name)
def render_sample_data( sample_toekn: str, with_anns: bool = True, box_vis_level: BoxVisibility = BoxVisibility.ANY, out_path: str = None, pred_data=None, seg_list=None ) -> None: """ Render sample data onto axis. :param sample_data_token: Sample_data token. :param with_anns: Whether to draw box annotations. :param box_vis_level: If sample_data is an image, this sets required visibility for boxes. :param axes_limit: Axes limit for lidar and radar (measured in meters). :param ax: Axes onto which to render. :param nsweeps: Number of sweeps for lidar and radar. :param out_path: Optional path to save the rendered figure to disk. :param underlay_map: When set to true, lidar data is plotted onto the map. This can be slow. :param use_flat_vehicle_coordinates: Instead of the current sensor's coordinate frame, use ego frame which is aligned to z-plane in the world. Note: Previously this method did not use flat vehicle coordinates, which can lead to small errors when the vertical axis of the global frame and lidar are not aligned. The new setting is more correct and rotates the plot by ~90 degrees. :param show_lidarseg: When set to True, the lidar data is colored with the segmentation labels. When set to False, the colors of the lidar data represent the distance from the center of the ego vehicle. :param show_lidarseg_legend: Whether to display the legend for the lidarseg labels in the frame. :param filter_lidarseg_labels: Only show lidar points which belong to the given list of classes. If None or the list is empty, all classes will be displayed. :param lidarseg_preds_bin_path: A path to the .bin file which contains the user's lidar segmentation predictions for the sample. :param verbose: Whether to display the image after it is rendered. :param show_panoptic: When set to True, the lidar data is colored with the panoptic labels. When set to False, the colors of the lidar data represent the distance from the center of the ego vehicle. If show_lidarseg is True, show_panoptic will be set to False. """ assert sample_toekn + '.png' in seg_list, '分割图必须存在！' lidar_img = lidiar_render(sample_toekn, pred_data, out_path=out_path)
if name == 'main': nusc = NuScenes(version='v1.0-mini', dataroot='/root/BEVFormer_segmentation_detection/data/nuscenes/', verbose=True)
render_annotation('7603b030b42a4b1caa8c443ccc1a7d52')