KeyError: "transform" in agent state

[alexiskovi]

Hi, LGSVL team! In my script I'm trying to check EGO position every 0.2 seconds with following lines, using Python API:

x = self.sim.map_to_gps(self.ego.state.transform).easting
y = self.sim.map_to_gps(self.ego.state.transform).northing

After some random time interval (about 10-15 seconds) I got the following exception and EGO agent can not move, but stop lights are react to keyboard signals. Exception text:

File "/opt/PythonAPI/lgsvl/agent.py", line 105, in state
return AgentState.from_json(j)
File "/opt/PythonAPI/lgsvl/utils.py", line 53, in from_json
Transform.from_json(j["transform"]),
KeyError: 'transform'

Reducing frequency from 5 Hz to 1 Hz didn't help.

[hadiTab]

@alexiskovi would it be possible to share your code?

[martins-mozeiko]

@alexiskovi Note that Python API is designed to be called from single thread. In case you are calling functions from multiple threads, the calls must be serialized explicitly.

[alexiskovi]

@hadiTab, @martins-mozeiko, multithreading is the exact problem here, I got it. Thank you a lot!

[alexiskovi]

I have downgraded my script to a single thread but faced the same problem. My script is about getting EGO relative position to the end point:

import lgsvl 
import os
import threading
import time
#from apollo_features import ApolloFeatures

#def sim_thread(sim):
#    sim.run()

VEHICLE_NAME = "Lexus2016RXHybrid (Apollo)"
VEHICLE_NAME = "Lincoln2017MKZ (Apollo 5.0)"
LEVEL1d4STARTPOS = lgsvl.Vector(-51.8521118164063, -0.000195413827896118, -43.7105865478516)
LEVEL1d4STARTROT = lgsvl.Vector(-3.2442374049424e-06, 159.470977783203, 4.19762145611458e-05)
LEVEL1d4START = (215703.15, 2546231.91)
LEVEL1d4DEST = (215751.025127983, 2546084.5615921)

sim = lgsvl.Simulator(os.environ.get("SIMULATOR_HOST", "127.0.0.1"), 8181)
sim.load(scene="Shalun")
ego_state = lgsvl.AgentState()
ego_state.transform.position = LEVEL1d4STARTPOS
ego_state.transform.rotation = LEVEL1d4STARTROT
ego = sim.add_agent(name = VEHICLE_NAME, agent_type = lgsvl.AgentType.EGO, state=ego_state)
ego.connect_bridge(address="127.0.0.1", port=9090)

pwp = []
pwp.append(lgsvl.WalkWaypoint(position=lgsvl.Vector(-1.26881003379822, -0.000195264830836095, -43.7538833618164), idle=0, trigger_distance = 13.0))
pwp.append(lgsvl.WalkWaypoint(position=lgsvl.Vector(-6.12296199798584, 0.124001771211624, -29.0761013031006), idle=0))

#apollo = ApolloFeatures()

spawns = sim.get_spawn()

state = lgsvl.AgentState()
state.transform = spawns[0]
state.transform.position = pwp[0].position

p = sim.add_agent("Pamela", lgsvl.AgentType.PEDESTRIAN, state)

p.follow(pwp, loop = True)

state = lgsvl.AgentState()
state.transform.position = lgsvl.Vector(23.9371910095215, -0.000195443630218506, -18.8676452636719)
state.transform.rotation = lgsvl.Vector(-3.88752823710092e-06, 159.779449462891, 4.14250753237866e-05)

npc = sim.add_agent("Sedan", lgsvl.AgentType.NPC, state)

nwp = []

nwp.append(lgsvl.DriveWaypoint(
    position = lgsvl.Vector(23.9371910095215, -0.000195443630218506, -18.8676452636719),
    speed = 1.0,
    angle = lgsvl.Vector(-3.88752823710092e-06, 159.779449462891, 4.14250753237866e-05),
    trigger_distance = 20
))

nwp.append(lgsvl.DriveWaypoint(
    position = lgsvl.Vector(28.6153316497803, -0.000195562839508057, -27.3859901428223),
    speed = 3.0,
    angle = lgsvl.Vector(-4.46926605945919e-06, 126.11262512207, 4.01972865802236e-05),
))

nwp.append(lgsvl.DriveWaypoint(
    position = lgsvl.Vector(39.7509460449219, -0.00019538402557373, -25.8429374694824),
    speed = 3.0,
    angle = lgsvl.Vector(-3.52771462530654e-06, 69.5432662963867, 4.12044173572212e-05),
    trigger_distance = 10
))

nwp.append(lgsvl.DriveWaypoint(
    position = lgsvl.Vector(61.4087600708008, -0.000195329674170353, -17.6602802276611),
    speed = 3.0,
    angle = lgsvl.Vector(-3.78179902327247e-06, 69.1639633178711, 4.094666292076e-05),
))

nwp.append(lgsvl.DriveWaypoint(
    position = lgsvl.Vector(67.7698669433594, -0.000195443630218506, -12.408299446106),
    speed = 3.0,
    angle = lgsvl.Vector(-4.12136387240025e-06, 21.0612297058105, 4.11341534345411e-05),
))

nwp.append(lgsvl.DriveWaypoint(
    position = lgsvl.Vector(65.3287506103516, -0.000195235013961792, -0.184117823839188),
    speed = 3.0,
    angle = lgsvl.Vector(-4.00602039007936e-06, 339.5361328125, 4.08482519560494e-05),
))

npc.follow(nwp, loop = True)

controllables = sim.get_controllables("signal")
signal = sim.get_controllable(lgsvl.Vector(-1.49898147583008, 5.48200035095215, -37.7793426513672), "signal")
control_policy = "green=20;loop"
signal.control(control_policy)

signal = sim.get_controllable(lgsvl.Vector(33.337100982666, 4.76799964904785, -23.2786884307861), "signal")
control_policy = "trigger=40;red=15;yellow=2;green=10;loop"
signal.control(control_policy)

#signal = sim.get_controllable(lgsvl.Vector(75.9334487915039, 5.44899940490723, -9.32880592346191), "signal")
#control_policy = "red=15;yellow=2;green=15;loop"
#signal.control(control_policy)

#apollo.stop_all_modules()
#time.sleep(5.0)
#apollo.launch_all_modules()
#time.sleep(5.0)
#apollo.send_routing_request(LEVEL1d4START[0], LEVEL1d4START[1], LEVEL1d4DEST[0], LEVEL1d4DEST[1])

while True:
    dx = sim.map_to_gps(ego.state.transform).easting - LEVEL1d4DEST[0]
    dy = sim.map_to_gps(ego.state.transform).northing - LEVEL1d4DEST[1]
    print(dx, dy)
    time.sleep(1.0)

After sending routing request in the following point in Shalun map: Car will navigate and stuck near this arrow: It happens even if you will navigate through this point with keyboard, without Apollo. With the same "transform" key exception as I mentioned before. I have no idea how to fix this, could you help me?

UPD: I got this behavior only in case of slow speed in this tunnel

[alexiskovi]

@hadiTab, @martins-mozeiko, I have made a screencast with this bug for better understanding. File test.py, which I have executed in terminal, contains the same code which I gave in the comment above. Simulator release 2020.06

[hadiTab]

@alexiskovi thanks for the details. We will look into this. Did pausing/playing the simulator at the end of your video make it start driving again?

[alexiskovi]

@hadiTab, yes, it helps

[alexiskovi]

@lemketron, rebooting simulator making me able to finish path, but it doesn't solve problem at all, ego is still freezing in tunnel. Don't close this issue for now, please.

[hadiTab]

@alexiskovi how do you run that Python API script? It doesn't have a sim.run?

[alexiskovi]

@hadiTab, simply python3 test.py and it works without sim.run() I deleted line sim.run() because I don't know how to make sim.run() and request ego position in one thread

[lemketron]

I deleted line sim.run() because I don't know how to make sim.run() and request ego position in one thread

Your loop should call sim.run(time_limit = step_time) where step_time is some value in seconds (e.g. 0.2 since that is the step time that you mentioned in your original message above) instead of calling time.sleep(1.0). This will let the simulator run for a bit before you check for updated ego.state information.

[hadiTab]

@alexiskovi what version of the simulator are you using?

[hadiTab]

@alexiskovi I think I have figured out what is going on here! You are running the API-only simulation in interactive mode. This should not be possible, however, from your video it is apparent that you have somehow entered interactive mode. The play/pause button are only available in interactive mode.

The reason you are able to get your PythonAPI script running without ever calling sim.run is because internally all sim.run does is it un-pauses the simulation just like the Play button does (this is exactly why we have disabled interactive mode for Python API). When you press the play button at the start, you are essentially calling sim.run once. However, Python API and simulator communicate synchronously, so the next time a message is to be passed between them the simulation will be paused again and you will need to hit the play button again for it to resume.

The solution to this is to do what @lemketron mentioned in his comment. Also you need to run the Python API simulation in non-interactive mode. To do that follow these steps:

• Click the edit button for the PythonAPI simulation in WebUI
• In the 'General' tab, uncheck the box saying 'Use API to control the simulator'.
• Go to the 'Map & Vehicles' tab and uncheck the 'Run simulation in interactive mode' box.
• Go back to the 'General' tab and this time check the box saying 'Use API to control the simulator'.
• Click submit.

Basically, there is a bug in the webUI that allows you to still get interactive mode if you checked the box before checking the API only box.

[alexiskovi]

@lemketron, @hadiTab, thank you! Give me please one day, I will try and close this issue.

[alexiskovi]

@lemketron, @hadiTab your solution is working! Thanks a lot! But I still have a problem. When I trying to move my car at start point with following code:

LEVEL1d4STARTPOS = lgsvl.Vector(-51.8521118164063, -0.000195413827896118, -43.7105865478516)
LEVEL1d4STARTROT = lgsvl.Vector(-3.2442374049424e-06, 159.470977783203, 4.19762145611458e-05)
ego_state = lgsvl.AgentState()
ego_state.transform.position = LEVEL1d4STARTPOS
ego_state.transform.rotation = LEVEL1d4STARTROT
# Some stuff to run
ego.state = ego_state

Ego camera start to fly randomly, ego respawns, but not sending data to bridge. Should I create other issue or/and send additional information about it?

[hadiTab]

@alexiskovi sorry for the late response. I haven't been able to reproduce your issue using those coordinates. By flying randomly, do you mean the ego vehicle is falling through the ground? In general it's better to spawn the vehicle slightly above ground and allowing it to fall to avoid any possible conflicts with the ground plane.

Even if I make the vehicle fall through the ground, the bridge is still connected and publishing messages for me so without more details I'm not sure what that was about.

[alexiskovi]

@hadiTab, sorry for the late response and bad describing. I made a screencast again for better understanding. Here is what is going on in this video:

1. Ego navigates with Apollo to the destination point
2. When arrived, simulation stopped for 5 seconds to stop some Apollo modules
3. Ego state changed to the start point coordinates, camera started flying in car direction
4. Simulation stopped for 5 seconds to start Apollo modules
5. Simulation started again, camera transform changed to "somewhere in right of ego", but everything else working fine
6. I changed camera state with settings button

My environment:

• Simulator release 2020.06
• Ubuntu 20.04