Optimizations following the "jump back issue" problem

rodrigoqueiroz commented 3 years ago

A series of changes related to optimizations and new configuration options that started with the issue #41 (Jumping back ). The main cause is performance impact of multiple vehicles, and required us to tackle different areas.

The main solution is the addition of the fixed planning time in SimConfig (or flexible up to the planning rate max time). We expect the problem to be avoided most of the time, but can't be eliminated if the system is under heavy load (when it happens, it will be logged as error).
Issue #28 After some tests, we gave up from this solution. Threads didn't increase performance of the planners, and decreased overall system performance (affecting the simulation in the main process).
Issue #52 , adding new configuration options for lateral planning and precision to allow most vehicles to use a faster planning.
Merged changes from master after new behavior tree location from #51. The test_scenarios stopped working, and I adapted the tree name according to the new standard.
Small fixes

rodrigoqueiroz commented 3 years ago

New changes:

Fixed issue #56 (for evaluation).
Removed collision testing for vehicles that are not Active in simulation.
Fixed bug in new lateral configuration class LT()

Configuring the lateral planner by tree was not working because the sampling method was Enum and the number codes assigned by the tree were not matching. IntEnum solves the problem. I added a new scenario _test_scenarios/gs_straight_lateraldemo.osm and a Behavior tree to test the new configuration. The btree switches between multiple lateral options over the simulation.

behaviortree st_lateral_demo:
    ?
        ->
            condition c_goal ( reached_goal( threshold=20 ) )
            maneuver m_stop_goal ( MStopConfig(target=2) )
        ->
            condition c_a               (sim_time       ( tmin=1, tmax=4 ))
            maneuver m_drive_centre     (MVelKeepConfig ( vel=MP(8), lat_target=LT(0.0) ))
        ->
            condition c_b               (sim_time       ( tmin=4, tmax=8 ))
            maneuver m_drive_left       (MVelKeepConfig ( vel=MP(8), lat_target=LT(1.0) ))
        ->
            condition c_c               (sim_time       ( tmin=8, tmax=12 ))
            maneuver m_drive_right      (MVelKeepConfig ( vel=MP(8), lat_target=LT(-1.0) ))
        ->
            condition c_d               (sim_time       ( tmin=12, tmax=16 ))
            maneuver m_uniform          (MVelKeepConfig ( vel=MP(8), lat_target=LT(0.0,4,2) ))
        ->
            condition c_e               (sim_time       ( tmin=16, tmax=20 ))
            maneuver m_lin              (MVelKeepConfig ( vel=MP(8), lat_target=LT(0.0,3,1) ))
        ->
            condition c_f               (sim_time       ( tmin=20, tmax=30 ))
            maneuver m_normal           (MVelKeepConfig       ( vel=MP(8), lat_target=LT(0.0,4,3) ))

WAEliasmith commented 3 years ago

We need to ensure all btrees are not referred to with the prefix trees/. For example, in gs_ringroad_stress_loop.osm
Exception: Btree 'trees/drive' was not found in any provided location ['/home/ma/anm_unreal_sim/submodules/geoscenarioserver/btrees']
@WAEliasmith could you please help fixing all test scenarios to make sure all of them work?

Sure!

WAEliasmith commented 3 years ago

both anm scenarios give the error:

File "/home/ae/anm_unreal_sim/submodules/geoscenarioserver/ScenarioSetup.py", line 77, in load_geoscenario_from_file
    ego_vehicle = RV(99, 'Ego', [0.0,0.0,0.0, 0.0,0.0,0.0])
NameError: name 'RV' is not defined

This error is not new, and is not a problem with this PR.

mchlswyr commented 3 years ago

both anm scenarios give the error:
File "/home/ae/anm_unreal_sim/submodules/geoscenarioserver/ScenarioSetup.py", line 77, in load_geoscenario_from_file
    ego_vehicle = RV(99, 'Ego', [0.0,0.0,0.0, 0.0,0.0,0.0])
NameError: name 'RV' is not defined
This error is not new, and is not a problem with this PR.

Yeah, I think RV stood for "remote vehicle". I'm guessing this was replaced with EV, which stands for "external vehicle".

WAEliasmith commented 3 years ago

eval scenarios test: (mostly just to be on the same page, but some errors in these tests were mentioned earlier)

base_eval_scenario_769 has no vehicles but the lights change properly. gn_gs.osm, test1_mixed,test_all has no lanelet map

test_50 has [Errno 2] No such file or directory: 'eval/trajlog/_1.csv' test_gn_gs has [Errno 2] No such file or directory: 'eval/trajlog/_28.csv'

These log missing errors happened to scenarios we care more about as well, so it would be good to get them fixed

rodrigoqueiroz commented 3 years ago

eval scenarios test: (mostly just to be on the same page, but some errors in these tests were mentioned earlier)

base_eval_scenario_769 has no vehicles but the lights change properly. gn_gs.osm, test1_mixed,test_all has no lanelet map

test_50 has [Errno 2] No such file or directory: 'eval/trajlog/_1.csv' test_gn_gs has [Errno 2] No such file or directory: 'eval/trajlog/_28.csv'

These log missing errors happened to scenarios we care more about as well, so it would be good to get them fixed

You can ignore everything related to evaluation. The scenarios from eval work with GSEvaluationServer that connects to the SQL database to retrieve traffic data and populate the dynamic content. The base scenario is just the empty area.

rodrigoqueiroz commented 3 years ago

I ran all of scenarios/test_scenarios and scenarios/pedestrian_scenarios

scenarios/test_scenarios/gs_straight_lanechange.osm stops right after the lane change

v1 gets stuck at the light in scenarios/test_scenarios/gs_intersection_redlight_follow.osm

v1 gets stuck at the light in scenarios/test_scenarios/gs_intersection_redlight.osm

I'm investigating this. Seems to be a more complicated issue introduced by the prediction in the planning.

mchlswyr commented 3 years ago

sv/FrenetTrajectory.py:35: error: Incompatible types in assignment (expression has type "Tuple[float]", variable has type "float")

Due to a comma after the declaration

sv/ManeuverConfig.py:198: error: Incompatible types in assignment (expression has type "Maneuver", variable has type "int") sv/ManeuverConfig.py:204: error: Incompatible types in assignment (expression has type "Maneuver", variable has type "int") sv/ManeuverConfig.py:222: error: Incompatible types in assignment (expression has type "Maneuver", variable has type "int") sv/ManeuverConfig.py:234: error: Incompatible types in assignment (expression has type "Maneuver", variable has type "int") sv/ManeuverConfig.py:241: error: Incompatible types in assignment (expression has type "Maneuver", variable has type "int") sv/ManeuverConfig.py:256: error: Incompatible types in assignment (expression has type "Maneuver", variable has type "int")

Maneuver is derived from Enum, so I'm guessing it is being implicitly converted to an int; I don't think this is causing any problems

sv/ManeuverConfig.py:231: error: Incompatible types in assignment (expression has type "None", variable has type "int") sv/ManeuverConfig.py:239: error: Incompatible types in assignment (expression has type "None", variable has type "int") sv/ManeuverConfig.py:250: error: Incompatible types in assignment (expression has type "None", variable has type "int") sv/ManeuverConfig.py:251: error: Incompatible types in assignment (expression has type "None", variable has type "int") sv/SDVPlannerState.py:28: error: Incompatible types in assignment (expression has type "None", variable has type "Tuple[float, float]") sv/SDVPlannerState.py:29: error: Incompatible types in assignment (expression has type "None", variable has type "Tuple[float, float]")

These are all declared as None, but they are set to the right type when they are used

sv/CostFunctions.py:247: error: Value of type variable "SupportsLessThanT" of "max" cannot be "_T" sv/CostFunctions.py:268: error: Value of type variable "SupportsLessThanT" of "max" cannot be "_T" sv/CostFunctions.py:339: error: Value of type variable "SupportsLessThanT" of "max" cannot be "_T" sv/CostFunctions.py:351: error: Value of type variable "SupportsLessThanT" of "max" cannot be "_T"

I'm not sure what these mean; in the code they are all max(some_list, key=abs)

ScenarioSetup.py:77: error: Name 'RV' is not defined

This should probably be changed to EV (see here)

ScenarioSetup.py:78: error: "SimTraffic" has no attribute "vehicle"; maybe "vehicles" or "add_vehicle"?

This should be sim_traffic.add_vehicle(ego_vehicle)

ScenarioSetup.py:169: error: Incompatible types in assignment (expression has type "Optional[float]", variable has type "float") ScenarioSetup.py:170: error: Incompatible types in assignment (expression has type "Optional[float]", variable has type "float") ScenarioSetup.py:223: error: Incompatible types in assignment (expression has type "Optional[float]", variable has type "float") ScenarioSetup.py:224: error: Incompatible types in assignment (expression has type "Optional[float]", variable has type "float")

These are because there is a case where they can be set to None, even though they are annotated as float

ScenarioSetup.py:269: error: Too many arguments for "my_scenario"

my_scenario() is an empty function; I think it is supposed to be implemented by the user

rodrigoqueiroz / geoscenarioserver

Optimizations following the "jump back issue" problem #53