nutonomy / nuscenes-devkit

The devkit of the nuScenes dataset.
https://www.nuScenes.org
Other
2.19k stars 616 forks source link

How to deal with the problem 'Multipolygon'object is not iterable? #1012

Closed gracexmatin closed 6 months ago

gracexmatin commented 7 months ago

When using the function 'get_map_mask' in the class NuScenesMap, the problem in the title occurred. How to use that correctly?

whyekit-motional commented 7 months ago

@gracexmatin pls post a code snippet to reproduce the error

gracexmatin commented 7 months ago

PKK1CnnF2E fVEeodh9oR

gracexmatin commented 7 months ago

@whyekit-motional Could you take a look and pls give me the possible explanation?

whyekit-motional commented 7 months ago

@gracexmatin pls post a code snippet to reproduce the error (i.e. some code which I can copy and paste and run)

gracexmatin commented 7 months ago

import os
import numpy as np
import torch

import sys
cur_file_path = os.path.dirname(os.path.realpath(__file__))
sys.path.append(os.path.join(cur_file_path, '..'))
import src.datasets.nuscenes_utils as nutils

NUSC_MAP_SIZES = { # in meters (H x W)
            'singapore-onenorth' : [2025.0, 1585.6],
            'singapore-hollandvillage' : [2922.9, 2808.3],
            'singapore-queenstown' : [3687.1, 3228.6],
            'boston-seaport' : [2118.1, 2979.5]
        }

class NuScenesMapEnv(object):
    def __init__(self, map_data_path,
                       bounds=[-17.0, -38.5, 60.0, 38.5],
                       layers=['drivable_area', 'carpark_area', 'road_divider', 'lane_divider'],
                       L=256,
                       W=256,
                       device='cpu',
                       flip_singapore=True,
                       load_lanegraph=False,
                       lanegraph_res_meters=1.0,
                       lanegraph_eps=1e-6,
                       pix_per_m=4):
        '''
        :param map_data_path: path to the dataset e.g. /path/to/mini which contains the maps directory
        :param bounds: [low_l, low_w, high_l, high_w] distances (in meters) around location to crop map observations
        :param layers: name of the nusc layers to render
        :param L: number of pixels along length of vehicle to render crop with
        :param W: number of pixels along width of vehicle to render crop with
        :param device: the device to store the rasterized maps on. Note that for
                        5 pix/m resolution this required ~5GB of memory for road and divider layers.
        :param flip_singapore: if true, flips singapore maps about the x axis to change driving direction from 
                                left hand to right hand side
        :param load_lanegraph: if true, loads the lane graph as well
        :param lanegraph_res_meters: resolution at which to discretize lane graph
        :param lanegraph_eps:
        :param pix_per_m: resolution to discretize map layers
        '''
        super(NuScenesMapEnv, self).__init__()

        self.data_path = map_data_path
        self.nusc_maps = nutils.get_nusc_maps(map_data_path)
        if load_lanegraph:
            print('Loading lane graphs...')
            self.lane_graphs = {map_name: nutils.process_lanegraph(nmap, lanegraph_res_meters, lanegraph_eps) for map_name,nmap in self.nusc_maps.items()}
        self.map_list = list(self.nusc_maps.keys())
        self.layer_names = layers
        self.bounds = bounds
        self.L = L
        self.W = W
        self.device = torch.device(device)
        self.flip_singapore = flip_singapore

        self.road_list = ['drivable_area', 'road_segment', 'lane']
        self.num_layers = 0
        road_layers = [lay for lay in self.layer_names if lay in self.road_list]
        self.num_layers = 1 if len(road_layers) > 0 else 0
        nonroad_layers = [lay for lay in self.layer_names if lay not in self.road_list]
        self.num_layers += len(nonroad_layers)

        # map layer names to their channel index in returned crops
        self.layer_map = {}
        for lay in road_layers:
            self.layer_map[lay] = 0
        lay_idx = 1
        for lay in nonroad_layers:
            self.layer_map[lay] = lay_idx
            lay_idx += 1

        # binarize all the layers we need for all maps and cache for crop later
        print('Rasterizing nuscenes maps...')
        m_per_pix = 1.0 / pix_per_m
        self.nusc_raster = []
        self.nusc_dx = []
        max_H, max_W = -float('inf'), -float('inf')
        msize_list = []
        for midx, mname in enumerate(self.map_list):
            nmap = self.nusc_maps[mname]
            msize = np.array(NUSC_MAP_SIZES[mname])
            cur_msize1 = msize * pix_per_m
            cur_msize1 = np.round(cur_msize1).astype(np.int32)
            cur_dx = msize / cur_msize1
            self.nusc_dx.append(cur_dx)
            cur_msize1 = tuple(cur_msize1)
            if cur_msize1[0] > max_H:
                max_H = cur_msize1[0]
            if cur_msize1[1] > max_W:
                max_W = cur_msize1[1]
            msize_list.append(cur_msize1)

        for midx, mname in enumerate(self.map_list):
            nmap = self.nusc_maps[mname]
            cur_msize = msize_list[midx]

            # get binarized rasterization of full map
            map_layers = []
            # first road
            # draw both road layers in one channel
            road_layers = [lay for lay in self.layer_names if lay in self.road_list]
            if len(road_layers) > 0:
                road_img = nmap.get_map_mask(None, 0.0, road_layers, cur_msize)
                # collapse to single layer
                road_img = np.clip(np.sum(road_img, axis=0), 0, 1).reshape((1, cur_msize[0], cur_msize[1])).astype(np.uint8)
                map_layers.append(road_img)
                print(map_layers)
            # draw any other layers separately (e.g. walkway)
            other_layers = [lay for lay in self.layer_names if lay not in self.road_list]
            if len(other_layers) > 0:
                other_img = nmap.get_map_mask(None, 0.0, other_layers, cur_msize)
                map_layers.append(other_img)

            # Create single image
            map_img = np.concatenate(map_layers, axis=0)
            print(map_img.shape)

            # flip about x axis if desired (i.e. switch driving direction)
            if self.flip_singapore and mname.split('-')[0] == 'singapore':
                print('Flipping %s about x axis...' % (mname))
                map_img = np.flip(map_img, axis=1).copy()

                if load_lanegraph:
                    print('Flipping lane graph about x axis...')
                    # also need to flip lane graph
                    cur_lg = self.lane_graphs[mname]
                    # xys is (L, 2), reflect y
                    xys = cur_lg['xy']
                    mheight = NUSC_MAP_SIZES[mname][0]
                    xys[:,1] = mheight - xys[:,1]
                    self.lane_graphs[mname]['xy'] = xys
                    # negate diffy in edges [x0, y0, diff[0], diff[1], dist]
                    edges = cur_lg['edges']
                    edges[:,1] = mheight - edges[:,1]
                    edges[:,3] *= -1
                    self.lane_graphs[mname]['edges'] = edges

            # # NOTE: viz debug
            # import matplotlib.pyplot as plt
            # fig = plt.figure(figsize=(6, 6))
            # plt.imshow(map_img[0], origin='lower', vmin=0, vmax=1)
            # # plt.imshow(map_img[0], vmin=0, vmax=1)
            # plt.gca().set_aspect('equal', adjustable='box')
            # imname = 'check_%04d_%s_nopad_flipped.jpg' % (0, mname)
            # print('saving', imname)
            # plt.savefig(imname)
            # plt.close(fig)

            pad_right = max_W - cur_msize[1]
            pad_bottom = max_H - cur_msize[0]
            pad = torch.nn.ZeroPad2d((0, pad_right, 0, pad_bottom))
            padded_map_img = pad(torch.from_numpy(map_img).unsqueeze(0))[0]

            self.nusc_raster.append(padded_map_img)

        # pad each map to max for efficient cropping
        self.nusc_raster = torch.stack(self.nusc_raster, dim=0).to(device)
        self.nusc_dx = torch.from_numpy(np.stack(self.nusc_dx, axis=0)).to(device)
whyekit-motional commented 7 months ago

@gracexmatin from import src.datasets.nuscenes_utils as nutils, I see you have some sort of custom utility methods which are not included in your code snippet

On my end, I came up with a code snippet to see if I can reproduce your error:

from nuscenes.map_expansion.map_api import NuScenesMap

nusc_map = NuScenesMap(dataroot='/data/sets/nuscenes', map_name='singapore-onenorth')

road_layers = ['drivable_area']
cur_msize= (100, 100)
print(nusc_map.get_map_mask(None, 0.0, road_layers, cur_msize))

get_map_mask works for me in the above code snippet

Some things I can suggest you try on your end are: