Dataset and “size mismatch for agent_rnn.waypoint_predictor”

[LuserBen]

Hi, @Iftimie Thank you for sharing the code. I have two issues： （1）What dataset did you use? Is the dataset from Waymo available? And can you share the dataset link to us？ （2）I found a issue when I run “python main.py”. “size mismatch for agent_rnn.waypoint_predictor.conv1.0.weight: copying a param with shape torch.Size([1, 32, 3, 3]) from checkpoint, the shape in current model is torch.Size([3, 32, 3, 3])”. Maybe caused by ChauffeyrNet/network/models/SimpleConv.py line 97: self.conv1 = self.conv_block(Config.rnn_num_channels,3) The model size between pretrained model and defined model are different. I don't know if I missed something. So can you help me, thanks very much.

[Iftimie]

Hi @LuserBen Sorry for my late response. (1) I created the dataset. I created a simulator with a 3d camera, a world defined by some coordinates and lines, and the rendering is done with opencv. A bit overkill but it was easier to start prototyping. You can create a dataset (a run with the car) by swiching in main.py record=True and config.py linux_env = False(my linux server did not had a GUI). Also I updated the master so that you can clone and start recording record your driving with the mouse and keys. In the console will appear some instructions. The design of the code is a mess. I learned a lot from then, and hopefully I won't do the same mistakes again. (2) Not sure why that appears. I have just cloned the repository. The script downloads from google drive the weights and the world. On my computer it works. However I have prepared a virtual machine freezed with all of the configurations dependencies and other stuff. You can open it with Oracle VirtualBox https://drive.google.com/file/d/1DMRL77TWLXf_7GeSmP-mFwBt1TySTPZY/view If that again does not show anything, make sure in main.py record=False, do_train=False, just_test_network = True, and Simulator argument to_video = False. Also check in config.py linux_env = False. Tell me if it helped

[kargarisaac]

You generate all the data from your simulator? speed limits, traffic light states, routes, ...?

[Iftimie]

Hi @kargarisaac The world of the simulator consists of a simple set of lines. I designed it in Blender then exported and loaded into the python scripts. The world has only lane markings and traffic lights, no speed limits, or routes. The routes (GPS like) that you see in the gifs are actually the paths that I took when I recorded the dataset. It's true that I only overfitted the model on the this track. The straffic lights states are just simply swiching every 5 seconds or a similar number. The dataset only contains a .pkl file with the list of states of the objects in the world (car has orientation, speed, wheels angle and translation. traffic lights have only the color state). This state is rendered at train time (otherwise the dataset would be huge) I decided to build this simulator because I found the Carla simulator API a bit limiting ( It did not allowed me to render traffic lights as lines on the road, because I could not access the world elements and their 3D geometry)

[LuserBen]

Hi, @Iftimie Thanks for your reply. (1) Yes, I tried to control the vehicle with the mouse and key in the simulator, it's interesting. (2) I redownloaded the weights file directly from your google drive links, it works. I think my previous weights file did't match the code. Thanks again.

[kargarisaac]

@Iftimie Thank you for your answer. Are you going to continue to work on this project? And do you have any code or link to generate the same bird view images from Carla?

Thank you

[Iftimie]

@kargarisaac I won't work on this project. It was fun, but It was enough for 2 weeks of work. The gifs in README were not generated programatically. I just opened Carla simulator, played with it and recorded my screen.