search

carla-simulator / carla

Open-source simulator for autonomous driving research.
http://carla.org
MIT License
11.11k stars 3.58k forks source link

terminate called after throwing an instance of 'std::out_of_range' what(): _Map_base::at #3424

Closed kargarisaac closed 3 years ago

kargarisaac commented 3 years ago

Hi,

I get the following error when I want to set to autopilot mode using traffic manager. I moved from carla 0.9.6 to 0.9.10.1

terminate called after throwing an instance of 'std::out_of_range'
  what():  _Map_base::at

It works sometimes but most of the time I get the above error. I use the default tm port which is 8000. I tested others too but the same result. I also use `-carla-rpc-port=5000' for the simulator.

In cases that it runs successfully, I see sometimes that the car doesn't move at all.

Update: tests in 0.9.9.4:

I also tested in carla 0.9.9.4. I see better performance here. When I have only one ego car, it works most of the time and when I add other agents, I get the same error as above. Here, again I see sometimes the car doesn't move in autopilot mode.

Thanks

thytless commented 3 years ago

Hello! I am concerned about your problem since I am going to upgrade to 0.9.10 as well.

  1. Which map did you test on? Did you modify it? I have also experienced such exception when I got my Town06.xodr wrong modified.
  2. Or would you please run your client code with gdb and have an insight into the stack trace? It should be able to locate the problem in libcarla.
kargarisaac commented 3 years ago

Hello! I am concerned about your problem since I am going to upgrade to 0.9.10 as well.

  1. Which map did you test on? Did you modify it? I have also experienced such exception when I got my Town06.xodr wrong modified.

I tested on Town03. I also tried to use behavior_agent.py code instead of TM but again I see the problem. But in less cases. Especially for a high number of agents. I didn't modify anything. Just used the tar.gz file directly in Ubuntu 18.

  1. Or would you please run your client code with gdb and have an insight into the stack trace? It should be able to locate the problem in libcarla.

I didn't do it. I have to check it. Thanks

piraka9011 commented 3 years ago

Here's an example stack trace I got: Carla (Client and server) v0.9.10.1 Ubuntu 18.04

(gdb) run test.py
Starting program: /usr/local/bin/python test.py
warning: Error disabling address space randomization: Operation not permitted
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7f2b51189700 (LWP 596)]
[New Thread 0x7f2b50988700 (LWP 597)]
[New Thread 0x7f2b4c187700 (LWP 598)]
[New Thread 0x7f2b4b986700 (LWP 599)]
[New Thread 0x7f2b47185700 (LWP 600)]
[New Thread 0x7f2b44984700 (LWP 601)]
[New Thread 0x7f2b42183700 (LWP 602)]
[New Thread 0x7f2b3f982700 (LWP 603)]
[New Thread 0x7f2b3d181700 (LWP 604)]
[New Thread 0x7f2b3a980700 (LWP 605)]
[New Thread 0x7f2b3817f700 (LWP 606)]
[New Thread 0x7f2b3597e700 (LWP 607)]
[New Thread 0x7f2b3317d700 (LWP 608)]
[New Thread 0x7f2b3297c700 (LWP 609)]
[New Thread 0x7f2b3017b700 (LWP 610)]
pygame 1.9.6
Hello from the pygame community. https://www.pygame.org/contribute.html
connecting to Carla server...
[New Thread 0x7f2b100cf700 (LWP 611)]
[New Thread 0x7f2b0f8ce700 (LWP 612)]
[New Thread 0x7f2b0f0cd700 (LWP 613)]
[New Thread 0x7f2b0e8cc700 (LWP 614)]
[New Thread 0x7f2b0e0cb700 (LWP 615)]
[New Thread 0x7f2b0d8ca700 (LWP 616)]
[New Thread 0x7f2b0d0c9700 (LWP 617)]
[New Thread 0x7f2b0c8c8700 (LWP 618)]
[New Thread 0x7f2ae7fff700 (LWP 619)]
[New Thread 0x7f2ae77fe700 (LWP 620)]
[New Thread 0x7f2ae6ffd700 (LWP 621)]
[New Thread 0x7f2ae67fc700 (LWP 622)]
[New Thread 0x7f2ae5ffb700 (LWP 623)]
[New Thread 0x7f2ae57fa700 (LWP 624)]
[New Thread 0x7f2ae4ff9700 (LWP 625)]
[New Thread 0x7f2acffff700 (LWP 626)]
[New Thread 0x7f2acf7fe700 (LWP 627)]
WARNING: Version mismatch detected: You are trying to connect to a simulator that might be incompatible with this API 
WARNING: Client API version     = 784d9b9f 
WARNING: Simulator API version  = 0.9.10.1 
Carla server connected!
[New Thread 0x7f2aceffd700 (LWP 628)]
[Thread 0x7f2b3017b700 (LWP 610) exited]
[Thread 0x7f2b3297c700 (LWP 609) exited]
[Thread 0x7f2b3317d700 (LWP 608) exited]
[Thread 0x7f2b3597e700 (LWP 607) exited]
[Thread 0x7f2b3817f700 (LWP 606) exited]
[Thread 0x7f2b3a980700 (LWP 605) exited]
[Thread 0x7f2b3d181700 (LWP 604) exited]
[Thread 0x7f2b3f982700 (LWP 603) exited]
[Thread 0x7f2b42183700 (LWP 602) exited]
[Thread 0x7f2b44984700 (LWP 601) exited]
[Thread 0x7f2b47185700 (LWP 600) exited]
[Thread 0x7f2b4b986700 (LWP 599) exited]
[Thread 0x7f2b4c187700 (LWP 598) exited]
[Thread 0x7f2b50988700 (LWP 597) exited]
[Thread 0x7f2b51189700 (LWP 596) exited]
[New Thread 0x7f2b3017b700 (LWP 630)]
[New Thread 0x7f2b3297c700 (LWP 631)]
[Thread 0x7f2b3017b700 (LWP 630) exited]
[Thread 0x7f2b3297c700 (LWP 631) exited]
[New Thread 0x7f2b3017b700 (LWP 632)]
[New Thread 0x7f2b3297c700 (LWP 633)]
terminate called after throwing an instance of 'std::out_of_range'
  what():  _Map_base::at

Thread 38 "python" received signal SIGABRT, Aborted.
[Switching to Thread 0x7f2b3297c700 (LWP 633)]
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51  ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.

Looks like non-existent files aren't handled properly.

thytless commented 3 years ago

Thread 38 "python" received signal SIGABRT, Aborted. [Switching to Thread 0x7f2b3297c700 (LWP 633)] __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.

Please call bt command of gdb to see full stack, usually stack top does not show the critical infomation

glopezdiest commented 3 years ago

Hey @piraka9011 @kargarisaac. I've done some tests from my side and everything is working fine, do any of you have a script to reproduce it? It'd be of much help.

kargarisaac commented 3 years ago

@glopezdiest Here is an example to reproduce the error:

https://gist.github.com/kargarisaac/57324b76c57a84cba5c8b9ec7412ba51

When I set 'number_of_vehicles':100 in the params dictionary I get the error:

terminate called after throwing an instance of 'std::out_of_range'
  what():  _Map_base::at
jackbart94 commented 3 years ago

Hi,

I've been checking out the code, and I can see that Traffic Manager isn't set to synchronous mode by using self.tm.set_synchronous_mode(True). I would highly recommend to do so, especially since the world is set to synchronous, and this will fix your error for sure.

Also, there are a lot of other errors in the code that are happening once you fix this, but let me know if this fixes at least this one.

brysef commented 3 years ago

Confirmed that this seems to fix the issue for me -- since it's a bit of a race condition I can't be 100% sure but I've had 3 successful run-throughs after the proposed fix. Thanks @jackbart94!

It would be good to have a documentation bump so folks don't need to refer back to this issue. That function call doesn't exist on the latest documentation (https://carla.readthedocs.io/en/latest/python_api/#carlatrafficmanager).

zchrissirhcz commented 3 years ago

@brysef You may just create an PR.

jackbart94 commented 3 years ago

It would be good to have a documentation bump so folks don't need to refer back to this issue. That function call doesn't exist on the latest documentation (https://carla.readthedocs.io/en/latest/python_api/#carlatrafficmanager).

The documentation is officially updated!

kargarisaac commented 3 years ago

Hi,

I've been checking out the code, and I can see that Traffic Manager isn't set to synchronous mode by using self.tm.set_synchronous_mode(True). I would highly recommend to do so, especially since the world is set to synchronous, and this will fix your error for sure.

Also, there are a lot of other errors in the code that are happening once you fix this, but let me know if this fixes at least this one.

Thanks a lot. Now it seems it works. For the ego car, I use the behavior_agent.py code to be able to set the route. Is it the same as TM behavior? I want it to do takeover and lane change automatically. I don't see this behavior in my limited tests. I see that TM has a method to force lane changing but the behavior agent doesn't have it. I want to run Carla for several days and collect data and want to be sure about the driving quality. Do you have any suggestions?

Thanks

jackbart94 commented 3 years ago

@kargarisaac If you are using the behavior_agent.py, it does perform lane changes but only under specific conditions (changes to the left lane if a vehicle in front is slower, or moves to the right if a vehicle behind is faster). There isn't a way to manually execute a lane change for the Behavior agent, but there are these policies.

However, you must make sure that the PID values for the specific car model that you are using are still valid, or the vehicle will oscillate. I remember that the ones I set were working fine for the Audi TT, but recently all vehicle physics have been changed and the PID values might need some tuning.

If you need more info about this, please reach out on Discord or open a new issue, as it falls out of the scope of this issue!

kargarisaac commented 3 years ago

@jackbart94 Thank you. I close this issue.