NPC vehicles become less and less when using Python API

[gyp199009]

Hello, I just found when I used Python API to set the simulation evironment, the NPC vehicles become less and less, it seems that NPC vehicle disappeares when it goes out of screen sight, is it true? If so, then how can I keep the quantity of NPC vehicles. Thanks in advance!

[EricBoiseLGSVL]

@gyp199009 NPC vehicles are culled when out of the EGO sensor and main camera frustum. This is common in game engines to save CPU and GPU cycles. Inside NPCManager we have checks for whether a NPC is visible and despawn if not. In API we do not have this functionality because we expect users to remove NPC's when needed. We will look into this. Any extra information would be helpful. What map are you using, number of NPC's, are they waypoint following, do you expect them to sit at the end of their waypoints?

[osholsonel20]

Piggybacking off of this, is there a quick way to remove the culling system to debug our HD maps. Trying to view the traffic patterns to make sure it's all setup correctly. Preferably just so it's not continuously spawning cars, limit to say 10 and then let them roam around?

[EricBoiseLGSVL]

@osholsonel20 Yes, you can increase the spawn area and always return true for isVisible. This check is whether sensor can "see" the NPC. Also, each map has a MapOrigin object in scene that let's you tweak NPC parameters for that map. Latestly, you can remove the occlusion features to the map. This is a Unity system that culls meshes that are not visible to the camera. Be warned, all these systems are used to keep fps at an acceptable level and I would recommend refactoring code to only use this when in development (I did this exact thing while working on NPC behaviors)

[osholsonel20]

Oh for sure, development only is exactly what this is for. Hopefully the editor view doesn't give me too much issue, after temporarily disabling all of this. Thanks for the quick and detailed response! I commented out DespawnVehicle in Reset() and that seems to be working, but i'll try what you suggested as well.

[gyp199009]

@gyp199009 NPC vehicles are culled when out of the EGO sensor and main camera frustum. This is common in game engines to save CPU and GPU cycles. Inside NPCManager we have checks for whether a NPC is visible and despawn if not. In API we do not have this functionality because we expect users to remove NPC's when needed. We will look into this. Any extra information would be helpful. What map are you using, number of NPC's, are they waypoint following, do you expect them to sit at the end of their waypoints?

I used my own map made from HD map annatation tool in Unity Editor, and I set the maximum NPC number to 50 (which is enough for me) in MapOrigin. In python script, I let the NPCs follow the cloest lane, since the NPCs' route is cyclic ( I edited in Unity Editor), I hope the NPCs can move coutinueslly without being culled when they are out of my view.

[EricBoiseLGSVL]

@gyp199009 We will create a ticket for this and check if we can reproduce. Are you able to post the .py script or a snippet so we can test?

[EricBoiseLGSVL]

@rongguodong This is the issue for the ticket

[rongguodong]

@gyp199009 Can you post your Python script here? It will help us what you are doing, and evaluate if this is a bug or expected feature.

[gyp199009]

@gyp199009 Can you post your Python script here? It will help us what you are doing, and evaluate if this is a bug or expected feature.

Sorry for being late to reply, here is my python script:

#Setup the environment
sim=lgsvl.Simulator(os.environ.get("SIMULATOR_HOST", "127.0.0.1"), 8181)
if sim.current_scene == "GongYanYuan_API":
  sim.reset()
else:
  sim.load("GongYanYuan_API",15245475)

#Setup the RSU
spawns = sim.get_spawn()
rsu1_state = lgsvl.AgentState()
rsu1_state.transform.position = spawns[0].position
rsu1_state.transform.rotation = spawns[0].rotation
forward = lgsvl.utils.transform_to_forward(spawns[0])
data1_collect=sim.add_agent("RSU_V2", lgsvl.AgentType.EGO, rsu1_state)

#Setup the ros_bridge
data1_collect.connect_bridge("192.168.19.137", 9090)
print("Waiting for connection...")
while not data1_collect.bridge_connected:
  time.sleep(1)
#  break
print("Bridge connected:", data1_collect.bridge_connected)

#Add NPCs
npc_names = ["Sedan", "SUV", "Jeep", "Hatchback", "SchoolBus", "BoxTruck","Sedan", "SUV", "Jeep", "Hatchback"]    
npc_state=lgsvl.AgentState()
forward = lgsvl.utils.transform_to_forward(spawns[2])
right = lgsvl.utils.transform_to_right(spawns[2])

for i in range(5):
  npc_state.transform.position = spawns[2].position + i*25*forward
  npc_state.transform.rotation = spawns[2].rotation
  npc =sim.add_agent(random.choice(npc_names), lgsvl.AgentType.NPC,npc_state)
  npc.follow_closest_lane(True, 5)

npc_state.transform.position = spawns[2].position + 115*forward + 3*right
npc_state.transform.rotation = spawns[2].rotation + lgsvl.Vector(0,60,0)
npc =sim.add_agent(random.choice(npc_names), lgsvl.AgentType.NPC,npc_state)
npc.follow_closest_lane(True, 5)

npc_state.transform.position = spawns[2].position + 31*forward + 10*right
npc_state.transform.rotation = spawns[2].rotation + lgsvl.Vector(0,90,0)
npc =sim.add_agent(random.choice(npc_names), lgsvl.AgentType.NPC,npc_state)
npc.follow_closest_lane(True, 5)

npc_state.transform.position = spawns[2].position + 75*forward + 10*right
npc_state.transform.rotation = spawns[2].rotation + lgsvl.Vector(0,90,0)
npc =sim.add_agent(random.choice(npc_names), lgsvl.AgentType.NPC,npc_state)
npc.follow_closest_lane(True, 5)

for i in range(5):
  npc_state.transform.position = spawns[2].position + 120*forward + (i+1)*30*right
  npc_state.transform.rotation = spawns[2].rotation + lgsvl.Vector(0,90,0)
  npc =sim.add_agent(random.choice(npc_names), lgsvl.AgentType.NPC,npc_state)
  npc.follow_closest_lane(True, 5)

npc_state.transform.position = spawns[2].position + 80*forward + 155*right
npc_state.transform.rotation = spawns[2].rotation + lgsvl.Vector(0,180,0)
npc =sim.add_agent(random.choice(npc_names), lgsvl.AgentType.NPC,npc_state)
npc.follow_closest_lane(True, 5)

npc_state.transform.position = spawns[2].position + 50*forward + 155*right
npc_state.transform.rotation = spawns[2].rotation + lgsvl.Vector(0,180,0)
npc =sim.add_agent(random.choice(npc_names), lgsvl.AgentType.NPC,npc_state)
npc.follow_closest_lane(True, 5)

npc_state.transform.position = spawns[2].position - 2*forward + 100*right
npc_state.transform.rotation = spawns[2].rotation + lgsvl.Vector(0,270,0)
npc =sim.add_agent(random.choice(npc_names), lgsvl.AgentType.NPC,npc_state)
npc.follow_closest_lane(True, 5)

npc_state.transform.position = spawns[2].position - 2*forward + 50*right
npc_state.transform.rotation = spawns[2].rotation + lgsvl.Vector(0,270,0)
npc =sim.add_agent(random.choice(npc_names), lgsvl.AgentType.NPC,npc_state)
npc.follow_closest_lane(True, 5)

forward = lgsvl.utils.transform_to_forward(spawns[3])
right = lgsvl.utils.transform_to_right(spawns[3])
for j in range(7):
  npc_state.transform.position = spawns[3].position + j*20*forward
  npc_state.transform.rotation = spawns[3].rotation
  npc =sim.add_agent(random.choice(npc_names), lgsvl.AgentType.NPC,npc_state)
  npc.follow_closest_lane(True, 5)

npc_state.transform.position = spawns[3].position + 150*forward - 6*right
npc_state.transform.rotation = spawns[3].rotation + lgsvl.Vector(0,-60,0)
npc =sim.add_agent(random.choice(npc_names), lgsvl.AgentType.NPC,npc_state)
npc.follow_closest_lane(True, 5)

for j in range(5):
  npc_state.transform.position = spawns[3].position + 152*forward - (j+1)*25*right
  npc_state.transform.rotation = spawns[3].rotation + lgsvl.Vector(0,-90,0)
  npc =sim.add_agent(random.choice(npc_names), lgsvl.AgentType.NPC,npc_state)
  npc.follow_closest_lane(True, 5)

npc_state.transform.position = spawns[3].position + 200*forward - 130*right
npc_state.transform.rotation = spawns[3].rotation
npc =sim.add_agent(random.choice(npc_names), lgsvl.AgentType.NPC,npc_state)
npc.follow_closest_lane(True, 5)

npc_state.transform.position = spawns[3].position + 245*forward - 90*right
npc_state.transform.rotation = spawns[3].rotation + lgsvl.Vector(0,90,0)
npc =sim.add_agent(random.choice(npc_names), lgsvl.AgentType.NPC,npc_state)
npc.follow_closest_lane(True, 5)

npc_state.transform.position = spawns[3].position + 245*forward - 30*right
npc_state.transform.rotation = spawns[3].rotation + lgsvl.Vector(0,90,0)
npc =sim.add_agent(random.choice(npc_names), lgsvl.AgentType.NPC,npc_state)
npc.follow_closest_lane(True, 5)

npc_state.transform.position = spawns[3].position + 200*forward + 10*right
npc_state.transform.rotation = spawns[3].rotation + lgsvl.Vector(0,180,0)
npc =sim.add_agent(random.choice(npc_names), lgsvl.AgentType.NPC,npc_state)
npc.follow_closest_lane(True, 5)

npc_state.transform.position = spawns[3].position + 140*forward + 10*right
npc_state.transform.rotation = spawns[3].rotation + lgsvl.Vector(0,180,0)
npc =sim.add_agent(random.choice(npc_names), lgsvl.AgentType.NPC,npc_state)
npc.follow_closest_lane(True, 5)

npc_state.transform.position = spawns[3].position + 50*forward + 10*right
npc_state.transform.rotation = spawns[3].rotation + lgsvl.Vector(0,180,0)
npc =sim.add_agent(random.choice(npc_names), lgsvl.AgentType.NPC,npc_state)
npc.follow_closest_lane(True, 5)

#Add pedestrians
pedestrian_names = ["Bob", "EntrepreneurFemale", "Howard", "Johny", "Pamela", "Presley", "Robin", "Stephen", "Zoe"]
forward = lgsvl.utils.transform_to_forward(spawns[0])
right = lgsvl.utils.transform_to_right(spawns[0])
pedestrian_state = lgsvl.AgentState()
for i in range(9):
  pedestrian_state.transform.position = spawns[0].position+random.choice([1,-1])*random.uniform(3,30)*forward-random.uniform(3,30)*right
  pedestrian_state.transform.rotation = spawns[0].rotation+lgsvl.Vector(0,random.uniform(0,360),0)
  pedestrian = sim.add_agent(pedestrian_names[i], lgsvl.AgentType.PEDESTRIAN, pedestrian_state)
  pedestrian.walk_randomly(True)

#Set the traffic light signal
controllables = sim.get_controllables()
signal_forwardWest = controllables[4]
signal_forwardEast = controllables[6]
signal_forwardSouth = controllables[3]
signal_forwardNorth = controllables[8]
signal_canteen = controllables[2]

control_policy = "trigger=100;green=8;yellow=3;red=5;loop"
signal_forwardWest.control(control_policy)
control_policy = "trigger=100;red=8;yellow=3;green=5;loop"
signal_forwardEast.control(control_policy)
control_policy = "trigger=100;red=2;green=8;yellow=3;red=5;loop"
signal_forwardNorth.control(control_policy)
control_policy = "trigger=100;green=2;red=8;yellow=3;green=5;loop"
signal_forwardSouth.control(control_policy)

input()
sim.run()

[rongguodong]

@gyp199009 Can you post your py file as attachment? The text was messed up here.

Also, do you only get this issue with your own map (i.e. GongYanYuan_API)? If no, can you post a script using our maps to reproduce this issue? If it only occurs on your own map, can you share the map?

This will help us to reproduce the issue locally.

[martins-mozeiko]

I edited the comment, now it should be fine.

[gyp199009]

@gyp199009 Can you post your py file as attachment? The text was messed up here.

Also, do you only get this issue with your own map (i.e. GongYanYuan_API)? If no, can you post a script using our maps to reproduce this issue? If it only occurs on your own map, can you share the map?

This will help us to reproduce the issue locally.

I finally found out the reason, NPC vehicle disappears when it doesn't move for a period of time (say tens seconds or more), this can be caused by crashing with other agents, waiting for a traffic signal, or lining up in a jam, can I disable this featrue or simply extend the duration of disappearing?

[rongguodong]

You can refer to the four places of Despawn() calling in NPCLaneFollowController.cs. For example, this line despawns the NPC if it stopped for longer than 60s. You can increase the threshold there.

[EricBoiseLGSVL]

@gyp199009 Yes, @rongguodong is right, if you increase the time or comment out you can prevent despawn. We will disable this in API mode, they should not despawn.

[rongguodong]

@gyp199009 We will fix this issue in next public release. In the meanwhile, you can simply change one line yourself as following: Change this line from

if (AutomaticMode)

to

if (AutomaticMode && !SimulatorManager.Instance.IsAPI)