search

LBNL-UCB-STI / beam

The Framework for Modeling Behavior, Energy, Autonomy, and Mobility in Transportation Systems
https://transportation.lbl.gov/beam
Other
142 stars 57 forks source link

Google API speed DB: a way to reuse accumulated google speed results #3197

Open nikolayilyin opened 3 years ago

nikolayilyin commented 3 years ago

When google speed comparison for simulation is turned on, then during simulation BEAM is doing requests to google API. The requests contain origin and destination X and Y coordinates and the time of the day. The responses contain travel time and distance. Requests, responses, and BEAM simulation travel time and distance are stored in ITERS/X.it/X.googleTravelTimeEstimation.csv file. For each city, during speed calibration, we accumulate a number of these files.

We need a way to reuse these files for rough speed calibration of the same city.

A possible approach is the following:

  1. use https://colab.research.google.com
  2. read all existing google travel time data for the current city (by links to s3 output bucket).
  3. read BEAM PathTraversal events from ITERS/X.it/X.events.csv file for specified simulation.
  4. find matching PathTraversal events for existing google origin and destination
  5. compare matching trips from BEAM simulation vs google travel time data
nikolayilyin commented 3 years ago

an example of how to read all accumulated results using https://colab.research.google.com:

s3urls = """ https://s3.us-east-2.amazonaws.com/beam-outputs/index.html#output/sfbay/sfbay-smart-activitysim-120k-flowcap-0.01__2021-06-02_22-21-30_hff https://s3.us-east-2.amazonaws.com/beam-outputs/index.html#output/sfbay/sfbay-smart-activitysim-120k-flowcap-0.04__2021-06-02_22-21-30_alb https://s3.us-east-2.amazonaws.com/beam-outputs/index.html#output/sfbay/sfbay-smart-activitysim-120k-flowcap-0.03__2021-06-03_10-32-56_ekb https://s3.us-east-2.amazonaws.com/beam-outputs/index.html#output/sfbay/sfbay-smart-activitysim-120k-flowcap-0.02__2021-06-03_10-32-56_jrd https://s3.us-east-2.amazonaws.com/beam-outputs/index.html#output/sfbay/sfbay-smart-activitysim-120k-flowcap-0.04__2021-06-04_10-00-58_ndl https://s3.us-east-2.amazonaws.com/beam-outputs/index.html#output/sfbay/sfbay-smart-activitysim-240k-flowcap-0.04__2021-06-04_10-00-58_cyu https://s3.us-east-2.amazonaws.com/beam-outputs/index.html#output/sfbay/sfbay-smart-activitysim-240k-flowcap-0.05__2021-06-04_10-00-59_euq https://s3.us-east-2.amazonaws.com/beam-outputs/index.html#output/sfbay/sfbay-smart-activitysim-120k-flowcap-0.04-with-overwriteRoadTypeProperties__2021-06-05_17-54-41_dlz https://s3.us-east-2.amazonaws.com/beam-outputs/index.html#output/sfbay/sfbay-smart-activitysim-120k-flowcap-0.04-with-overwriteRoadTypeProperties__2021-06-06_13-46-54_uzv""".split('\n')

google_tt_list = []

for s3url in s3urls: iteration = 5 s3path = get_output_path_from_s3_url(s3url) google_tt = pd.read_csv(s3path + "/ITERS/it.{0}/{0}.googleTravelTimeEstimation.csv".format(iteration)) google_tt_list.append(google_tt[(google_tt['departureTime'] < 24 60 60) & (google_tt['googleTravelTime'] > 0) & (google_tt['simTravelTime'] > 0)])

google_tt = pd.concat(google_tt_list)

nikolayilyin commented 3 years ago

we have almost finished work on Google API DB here - #2904

carloscaldas commented 2 years ago

Subtasks: