Changes on generic_metric and mode_specific_metrics in relation to Mo…

[iantei]

…de_confirm mapping with dynamic labels.

[iantei]

please indicate testing done as well. How do you know that this works?

I have executed manual testing, by modifying the year and month parameter to run for different notebooks. Below:

Mode confirmation for Trips under 10 miles	Number of Trips for each mode
Execution for generic_metrics with params: Year: None, Month: None, dynamic_labels loaded with json object from dynamic_labels	Similarly, execution for generic_metrics notebook with params changed to: Year: 2020, Month: 11, dynamic_labels as empty

[iantei]

Some testing cases:

1. In case of energy_calculations notebook: In both these cases, the Month and Year have been set to "None".

With Default Mapping	With Dynamic Labels
dynamic_labels is passed as { }	dynamic_labels is passed as json object available on Dynamic labels json object example

The difference which is apparent, is the difference in mapping label for Replaced Mode on Y-axis - "Gas Car, Others" vs "Gas Car Shared Ride".

[iantei]

Testing scenario:

Distribution of Mode_confirm attribute without dynamic_labels: Distribution of Mode_confirm attribute with dynamic_labels: Added 6 entries into the database, containing moped as user_input.mode_confirm. The entry for mode_confirm is less than 2%, therefore is merged into Others category in the pie chart representation. It is represented as "Moped" in the table below. Another representation of "Moped" label, showing dynamic_labels: Average miles per transport mode selected: Ran notebook: generic metrics notebook through automation 1. Passing dynamic_labels json object through generate_plots.py: 2. Without passing dynamic_labels json object through generate_plots.py, initializing it as dynamic_labels = { } I will experiment with more entries of Moped data, if required.

[iantei]

almost there. the requested changes are fairly minor. Please remember to resolve the merge conflict.

We need the most improvement in the "Testing done"

Your existing testing:

• does not touch generate_plots.py
• does not actually test the new custom label functionality (I don't see any entries for the one new mode = 'Moped')
• does not compare the same metric with and without dynamic labels to illustrate the difference with and without using custom labels

Code review is not a substitute for testing. It focuses on readability and code structure. We still need to actually test to make sure that the code works (aka "if it is not tested, it is broken" 😄 )

Executed all these testing, mentioned above.

[shankari]

@iantei

Added 6 entries into the database, containing moped as user_input.mode_confirm. The entry for mode_confirm is less than 2%, therefore is merged into Others category in the pie chart representation. It is represented as "Moped" in the table below.

High level feedback: instead of a one-time update saying "I did X", it would be good to record what you did in this "lab notebook" - aka I did X in this way. As you can see from my PR, that helps maintain a record of exactly what was done for reproducibility, and also helps others who want to achieve the same goal.

Ran notebook: generic metrics notebook through automation Passing dynamic_labels json object through generate_plots.py:

High level feedback: Please don't put code into issues in screenshots. Screenshots are not searchable, or copy-pasteable or zoomable... Please replace with markdown code blocks.

I will experiment with more entries of Moped data, if required.

Yes, you should have more entries of Moped data - neither of your tests shows any actual data. I don't understand why you didn't just replace all entries of a mode that is present - e.g. walk -> moped and then check that the graph was the same, only with the moped label instead of walk

Passing dynamic_labels json object through generate_plots.py: Without passing dynamic_labels json object through generate_plots.py, initializing it as dynamic_labels = { }

Both of these are calling the same script in the same way with the same config file. Running tests by editing the code between tests is not principled - you are not testing the code that is going to be committed. Please run the code with different configurations to verify.

[iantei]

**Updated Testing scenario:**

**Testing done:** **Notebook runs through automation:** **With the default settings: STUDY_CONFIG:stage-program** **Results are:** generic_metrics.ipynb: ``` (emission) root@af656e62a649:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/update_mappings.py mapping_dictionaries.ipynb (emission) root@af656e62a649:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_metrics.ipynb /usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="? if r.status_code is not 200: usage: generate_metrics [-h] [-d DATE DATE] plot_notebook program generate_metrics: error: the following arguments are required: program (emission) root@af656e62a649:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_metrics.ipynb default /usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="? if r.status_code is not 200: About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/stage-program.nrel-op.json Successfully downloaded config with version 1 for Staging environment for testing programs only and data collection URL https://openpath-stage.nrel.gov/api/ Dynamic labels are not available. Running at 2023-09-19T17:54:27.240880+00:00 with args Namespace(plot_notebook='generic_metrics.ipynb', program='default', date=None) for range (, ) Running at 2023-09-19T17:54:27.279311+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})] Running at 2023-09-19T17:54:52.317446+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=9), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})] Running at 2023-09-19T17:54:59.014789+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=10), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})] Running at 2023-09-19T17:55:05.722502+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=11), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})] Running at 2023-09-19T17:55:13.210319+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=12), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})] ``` **With the change: STUDY_CONFIG:dev-emulator-study** ``` ashrest2-35384s:em-public-dashboard ashrest2$ git diff docker-compose.dev.yml diff --git a/docker-compose.dev.yml b/docker-compose.dev.yml index fdbdbb8..9cbc4de 100644 --- a/docker-compose.dev.yml +++ b/docker-compose.dev.yml @@ -26,7 +26,7 @@ services: - DB_HOST=db - WEB_SERVER_HOST=0.0.0.0 - CRON_MODE= - - STUDY_CONFIG=stage-program + - STUDY_CONFIG=dev-emulator-study ports: # ipynb in numbers - "47962:8888" ``` **Results are:** generic_metrics.ipynb ``` (emission) root@f9305d3a5550:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/update_mappings.py mapping_dictionaries.ipynb (emission) root@f9305d3a5550:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_metrics.ipynb default /usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="? if r.status_code is not 200: About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/dev-emulator-study.nrel-op.json Successfully downloaded config with version 1 for Development environment (study) and data collection URL default Dynamic labels download was successful. Running at 2023-09-19T18:17:43.139687+00:00 with args Namespace(plot_notebook='generic_metrics.ipynb', program='default', date=None) for range (, ) Running at 2023-09-19T18:17:43.178087+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='study'), Parameter('include_test_users', bool, value=False), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'bikeshare', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'scootershare', 'baseMode': 'E_SCOOTER', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.00894}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'moped', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.05555}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-bike': 'E-bike', 'bike': 'Regular Bike', 'bikeshare': 'Bikeshare', 'scootershare': 'Scooter share', 'drove_alone': 'Gas Car Drove Alone', 'shared_ride': 'Gas Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'moped': 'Moped', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Air', 'not_a_trip': 'Not a trip', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'es': {'walk': 'Caminando', 'e-bike': 'e-bicicleta', 'bike': 'Bicicleta', 'bikeshare': 'Bicicleta compartida', 'scootershare': 'Motoneta compartida', 'drove_alone': 'Coche de Gas, Condujo solo', 'shared_ride': 'Coche de Gas, Condujo con otros', 'e_car_drove_alone': 'e-coche, Condujo solo', 'e_car_shared_ride': 'e-coche, Condujo con ontras', 'moped': 'Ciclomotor', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Autobús', 'train': 'Tren', 'free_shuttle': 'Colectivo gratuito', 'air': 'Avión', 'not_a_trip': 'No es un viaje', 'home': 'Inicio', 'work': 'Trabajo', 'at_work': 'En el trabajo', 'school': 'Escuela', 'transit_transfer': 'Transbordo', 'shopping': 'Compras', 'meal': 'Comida', 'pick_drop_person': 'Recoger/ Entregar Individuo', 'pick_drop_item': 'Recoger/ Entregar Objeto', 'personal_med': 'Personal/ Médico', 'access_recreation': 'Acceder a Recreación', 'exercise': 'Recreación/ Ejercicio', 'entertainment': 'Entretenimiento/ Social', 'religious': 'Religioso', 'other': 'Otros'}}})] Running at 2023-09-19T18:18:02.744057+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=9), Parameter('program', str, value='default'), Parameter('study_type', str, value='study'), Parameter('include_test_users', bool, value=False), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'bikeshare', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'scootershare', 'baseMode': 'E_SCOOTER', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.00894}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'moped', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.05555}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-bike': 'E-bike', 'bike': 'Regular Bike', 'bikeshare': 'Bikeshare', 'scootershare': 'Scooter share', 'drove_alone': 'Gas Car Drove Alone', 'shared_ride': 'Gas Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'moped': 'Moped', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Air', 'not_a_trip': 'Not a trip', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'es': {'walk': 'Caminando', 'e-bike': 'e-bicicleta', 'bike': 'Bicicleta', 'bikeshare': 'Bicicleta compartida', 'scootershare': 'Motoneta compartida', 'drove_alone': 'Coche de Gas, Condujo solo', 'shared_ride': 'Coche de Gas, Condujo con otros', 'e_car_drove_alone': 'e-coche, Condujo solo', 'e_car_shared_ride': 'e-coche, Condujo con ontras', 'moped': 'Ciclomotor', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Autobús', 'train': 'Tren', 'free_shuttle': 'Colectivo gratuito', 'air': 'Avión', 'not_a_trip': 'No es un viaje', 'home': 'Inicio', 'work': 'Trabajo', 'at_work': 'En el trabajo', 'school': 'Escuela', 'transit_transfer': 'Transbordo', 'shopping': 'Compras', 'meal': 'Comida', 'pick_drop_person': 'Recoger/ Entregar Individuo', 'pick_drop_item': 'Recoger/ Entregar Objeto', 'personal_med': 'Personal/ Médico', 'access_recreation': 'Acceder a Recreación', 'exercise': 'Recreación/ Ejercicio', 'entertainment': 'Entretenimiento/ Social', 'religious': 'Religioso', 'other': 'Otros'}}})] Running at 2023-09-19T18:18:09.989461+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=10), Parameter('program', str, value='default'), Parameter('study_type', str, value='study'), Parameter('include_test_users', bool, value=False), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'bikeshare', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'scootershare', 'baseMode': 'E_SCOOTER', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.00894}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'moped', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.05555}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-bike': 'E-bike', 'bike': 'Regular Bike', 'bikeshare': 'Bikeshare', 'scootershare': 'Scooter share', 'drove_alone': 'Gas Car Drove Alone', 'shared_ride': 'Gas Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'moped': 'Moped', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Air', 'not_a_trip': 'Not a trip', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'es': {'walk': 'Caminando', 'e-bike': 'e-bicicleta', 'bike': 'Bicicleta', 'bikeshare': 'Bicicleta compartida', 'scootershare': 'Motoneta compartida', 'drove_alone': 'Coche de Gas, Condujo solo', 'shared_ride': 'Coche de Gas, Condujo con otros', 'e_car_drove_alone': 'e-coche, Condujo solo', 'e_car_shared_ride': 'e-coche, Condujo con ontras', 'moped': 'Ciclomotor', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Autobús', 'train': 'Tren', 'free_shuttle': 'Colectivo gratuito', 'air': 'Avión', 'not_a_trip': 'No es un viaje', 'home': 'Inicio', 'work': 'Trabajo', 'at_work': 'En el trabajo', 'school': 'Escuela', 'transit_transfer': 'Transbordo', 'shopping': 'Compras', 'meal': 'Comida', 'pick_drop_person': 'Recoger/ Entregar Individuo', 'pick_drop_item': 'Recoger/ Entregar Objeto', 'personal_med': 'Personal/ Médico', 'access_recreation': 'Acceder a Recreación', 'exercise': 'Recreación/ Ejercicio', 'entertainment': 'Entretenimiento/ Social', 'religious': 'Religioso', 'other': 'Otros'}}})] ``` **Test Done Manually:** Before: Looked up in the mongo container for the number of entries for "walk" ``` > show dbs > use Stage_database switched to db Stage_database > show collections CarbonUsage Checkinout New_tier PolarBear Stage_Profiles Stage_analysis_timeseries Stage_pipeline_state Stage_timeseries Stage_timeseries_error Stage_usercache Stage_uuids Tier_Sys Usernames > db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"walk"}).count() 4359 > db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"moped"}).count() 0 ``` **Next:** Executed following code inside the mongo container to modify the `"data.user_input.mode_confirm":"walk" to "data.user_input.mode_confirm":"moped"` ``` var result_walk = db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"walk"}, {"_id": 1}) var resultArray = result_walk.toArray(); for (var i = 0; i < resultArray.length; i++) { var doc = resultArray[i]; var docId = doc._id; var updateResult = db.Stage_analysis_timeseries.update( { "_id": docId, "data.user_input.mode_confirm": "walk" }, { "$set": { "data.user_input.mode_confirm": "moped" } } ); if (updateResult.nModified > 0) { print(`Updated document with _id: ${docId}`); } } > db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"moped"}).count() 4359 > db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"walk"}).count() 0 ``` **Current Representation** Notebook: generic_metrics: With empty dynamic_labels: There is no representation of walk - since it's not available, and moped is not mapped correctly, so it's categorized as Other. **Next:** Passing the following parameter into the notebook for dynamic_labels: [dev-emulator-study label_options](https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/label_options/example-study-label-options.json) ``` { "MODE": [ {"value":"walk", "baseMode":"WALKING", "met_equivalent":"WALKING", "kgCo2PerKm": 0}, {"value":"e-bike", "baseMode":"E_BIKE", "met": {"ALL": {"range": [0, -1], "mets": 4.9}}, "kgCo2PerKm": 0.00728}, {"value":"bike", "baseMode":"BICYCLING", "met_equivalent":"BICYCLING", "kgCo2PerKm": 0}, {"value":"bikeshare", "baseMode":"BICYCLING", "met_equivalent":"BICYCLING", "kgCo2PerKm": 0}, {"value":"scootershare", "baseMode":"E_SCOOTER", "met_equivalent":"IN_VEHICLE", "kgCo2PerKm": 0.00894}, {"value":"drove_alone", "baseMode":"CAR", "met_equivalent":"IN_VEHICLE", "kgCo2PerKm": 0.22031}, {"value":"shared_ride", "baseMode":"CAR", "met_equivalent":"IN_VEHICLE", "kgCo2PerKm": 0.11015}, {"value":"e_car_drove_alone", "baseMode":"E_CAR", "met_equivalent":"IN_VEHICLE", "kgCo2PerKm": 0.08216}, {"value":"e_car_shared_ride", "baseMode":"E_CAR", "met_equivalent":"IN_VEHICLE", "kgCo2PerKm": 0.04108}, {"value":"moped", "baseMode":"MOPED", "met_equivalent":"IN_VEHICLE", "kgCo2PerKm": 0.05555}, {"value":"taxi", "baseMode":"TAXI", "met_equivalent":"IN_VEHICLE", "kgCo2PerKm": 0.30741}, {"value":"bus", "baseMode":"BUS", "met_equivalent":"IN_VEHICLE", "kgCo2PerKm": 0.20727}, {"value":"train", "baseMode":"TRAIN", "met_equivalent":"IN_VEHICLE", "kgCo2PerKm": 0.12256}, {"value":"free_shuttle", "baseMode":"BUS", "met_equivalent":"IN_VEHICLE", "kgCo2PerKm": 0.20727}, {"value":"air", "baseMode":"AIR", "met_equivalent":"IN_VEHICLE", "kgCo2PerKm": 0.09975}, {"value":"not_a_trip", "baseMode":"UNKNOWN", "met_equivalent":"UNKNOWN", "kgCo2PerKm": 0}, {"value":"other", "baseMode":"OTHER", "met_equivalent":"UNKNOWN", "kgCo2PerKm": 0} ], "PURPOSE": [ {"value":"home"}, {"value":"work"}, {"value":"at_work"}, {"value":"school"}, {"value":"transit_transfer"}, {"value":"shopping"}, {"value":"meal"}, {"value":"pick_drop_person"}, {"value":"pick_drop_item"}, {"value":"personal_med"}, {"value":"access_recreation"}, {"value":"exercise"}, {"value":"entertainment"}, {"value":"religious"}, {"value":"other"} ], "translations": { "en": { "walk": "Walk", "e-bike": "E-bike", "bike": "Regular Bike", "bikeshare": "Bikeshare", "scootershare": "Scooter share", "drove_alone": "Gas Car Drove Alone", "shared_ride": "Gas Car Shared Ride", "e_car_drove_alone": "E-Car Drove Alone", "e_car_shared_ride": "E-Car Shared Ride", "moped": "Moped", "taxi": "Taxi/Uber/Lyft", "bus": "Bus", "train": "Train", "free_shuttle": "Free Shuttle", "air": "Air", "not_a_trip": "Not a trip", "home": "Home", "work": "To Work", "at_work": "At Work", "school": "School", "transit_transfer": "Transit transfer", "shopping": "Shopping", "meal": "Meal", "pick_drop_person": "Pick-up/ Drop off Person", "pick_drop_item": "Pick-up/ Drop off Item", "personal_med": "Personal/ Medical", "access_recreation": "Access Recreation", "exercise": "Recreation/ Exercise", "entertainment": "Entertainment/ Social", "religious": "Religious", "other": "Other" }, "es": { "walk": "Caminando", "e-bike": "e-bicicleta", "bike": "Bicicleta", "bikeshare": "Bicicleta compartida", "scootershare": "Motoneta compartida", "drove_alone": "Coche de Gas, Condujo solo", "shared_ride": "Coche de Gas, Condujo con otros", "e_car_drove_alone": "e-coche, Condujo solo", "e_car_shared_ride": "e-coche, Condujo con ontras", "moped": "Ciclomotor", "taxi": "Taxi/Uber/Lyft", "bus": "Autobús", "train": "Tren", "free_shuttle": "Colectivo gratuito", "air": "Avión", "not_a_trip": "No es un viaje", "home": "Inicio", "work": "Trabajo", "at_work": "En el trabajo", "school": "Escuela", "transit_transfer": "Transbordo", "shopping": "Compras", "meal": "Comida", "pick_drop_person": "Recoger/ Entregar Individuo", "pick_drop_item": "Recoger/ Entregar Objeto", "personal_med": "Personal/ Médico", "access_recreation": "Acceder a Recreación", "exercise": "Recreación/ Ejercicio", "entertainment": "Entretenimiento/ Social", "religious": "Religioso", "other": "Otros" } } } ``` For generic_metrics notebook - Miles per chosen transport mode (mode_confirm:"moped"). Changed the mode_confirm back to "walk" using the following code (for comparison with the above chart). ``` var result_moped = db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"moped"}, {"_id": 1}) var resultArray = result_moped.toArray(); for (var i = 0; i < resultArray.length; i++) { var doc = resultArray[i]; var docId = doc._id; var updateResult = db.Stage_analysis_timeseries.update( { "_id": docId, "data.user_input.mode_confirm": "moped" }, { "$set": { "data.user_input.mode_confirm": "walk" } } ); if (updateResult.nModified > 0) { print(`Updated document with _id: ${docId}`); } } db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"walk"}).count() 4359 > db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"moped"}).count() 0 ``` For generic_metrics notebook - Miles per chosen transport mode (mode_confirm:"walk") These show that the "moped" label has been properly shown on the pie-chart representation.

[shankari]

For generic_metrics notebook - Miles per chosen transport mode.

For the record, I don't see the old "walk" value for comparison in this figure.

    show dbs

    use Stage_database
    switched to db Stage_database

For the record, this is using the mongo shell. This works and I don't think you should change this now. But you should get familiar with accessing mongo through the timseries interface to be consistent with the rest of the OpenPATH code.

[iantei]

For generic_metrics notebook - Miles per chosen transport mode.

For the record, I don't see the old "walk" value for comparison in this figure.

I have updated the above comment with a figure for the "walk" value for comparative reference to "moped" value for "Miles per chosen transport mode".

[iantei]

Just an overview about the change I introduced in the figure (3): I incorporated the mapping present in example-study-label-options

Excerpt from the above json: Translations_1:

"translations": {
    "en": {
      "walk": "Walk",
      "e-bike": "E-bike",
      "bike": "Regular Bike",
      "bikeshare": "Bikeshare",
      "scootershare": "Scooter share",
      "drove_alone": "Gas Car Drove Alone",
      "shared_ride": "Gas Car Shared Ride",
      "e_car_drove_alone": "E-Car Drove Alone",
      "e_car_shared_ride": "E-Car Shared Ride",
      "moped": "Moped",
      "taxi": "Taxi/Uber/Lyft",
      "bus": "Bus",
      "train": "Train",
      "free_shuttle": "Free Shuttle",
      "air": "Air",
      "not_a_trip": "Not a trip",
      "home": "Home",
      "work": "To Work",
      "at_work": "At Work",
      "school": "School",
      "transit_transfer": "Transit transfer",
      "shopping": "Shopping",
      "meal": "Meal",
      "pick_drop_person": "Pick-up/ Drop off Person",
      "pick_drop_item": "Pick-up/ Drop off Item",
      "personal_med": "Personal/ Medical",
      "access_recreation": "Access Recreation",
      "exercise": "Recreation/ Exercise",
      "entertainment": "Entertainment/ Social",
      "religious": "Religious",
      "other": "Other"
    },

From my observation, there are a few mapping which are missing compared to the mode_labels.csv like pilot_ebike: E-bike mapping is missing. In the stage_database,

> db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"e-bike"}).count()
25
> db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"pilot_ebike"}).count()
3919

Therefore, there is disparity in the representation at the pie chart showcased in the Figure 3.

Furthermore, if we use the following elaborated/edited version of "translations.en" instead of one provided in example-study-label-options

I appended all mapping in mode_labels.csv and one present on example-study-lable-options to create the below translations.en:

Translations_2:

  "translations": {
        "en": {
      "drove_alone": "Gas Car Drove Alone",
      "e_car_drove_alone": "E-Car Drove Alone",
      "work_vehicle": "Work Vehicle", 
      "bus": "Bus",
      "train": "Train",
      "free_shuttle": "Free Shuttle",
      "train,_bus and walk": "Train",
      "train_and pilot e-bike":"Train",
      "taxi": "Taxi/Uber/Lyft",
      "friend_picked me up": "Gas Car Shared Ride",
      "carpool_w/ friend to work": "Gas Car Shared Ride",
      "carpool_to work": "Gas Car Shared Ride",
      "friend/co_worker carpool": "Gas Car Shared Ride",
      "carpool_to lunch":"Gas Car Shared Ride",
      "carpool":"Gas Car Shared Ride",
      "carpool_for lunch": "Gas Car Shared Ride",
      "shared_ride": "Gas Car Shared Ride",
      "e_car_shared_ride": "E-car Shared Ride",
      "bikeshare": "Bikshare",
      "scootershare": "Scooter share",
      "pilot_ebike":"E-bike",
      "e-bike":"E-bike",
      "skateboard":"Skate board",
      "bike": "REgular Bike",
      "the_friend who drives us to work was running errands after the shift before dropping me off. not a trip of mine":"Not a trip",
      "not_a_trip": "Not a trip",
      “no_travel”:”No Travel",
      “same_mode”:”Same Mode",
      "walk": "Walk",
      "e-bike": "E-bike",
      "bike": "Regular Bike",
      "e_car_shared_ride": "E-Car Shared Ride",
      "moped": "Moped",
      "air": "Air",
      "home": "Home",
      "work": "To Work",
      "at_work": "At Work",
      "school": "School",
      "transit_transfer": "Transit transfer",
      "shopping": "Shopping",
      "meal": "Meal",
      "pick_drop_person": "Pick-up/ Drop off Person",
      "pick_drop_item": "Pick-up/ Drop off Item",
      "personal_med": "Personal/ Medical",
      "access_recreation": "Access Recreation",
      "exercise": "Recreation/ Exercise",
      "entertainment": "Entertainment/ Social",
      "religious": "Religious",
      "other": "Other"
    }
}

I observed the following difference without and with the dynamic_labels in generic_metrics notebook for Distribution of mode_Confirm attribute.

Without dynamic config and "walk" changed to "moped"	With dynamic config

I hope this clarifies the percentage difference between (1), (2) with (3).

[shankari]

@iantei the two graphs that you discuss (in great depth) in https://github.com/e-mission/em-public-dashboard/pull/91#issuecomment-1734088434 are basically (1) and (2) from my comment.

I see your comment around

From my observation, there are a few mapping which are missing compared to the mode_labels.csv like pilot_ebike: E-bike mapping is missing.

But if the mappings are missing, then it should default to "Other". That is what happens in (2) - we don't have a mapping for "moped" so it all shows up as "other". But in (3) if we don't have a mapping for pilot_ebike, then it looks like the entries are just dropped. There is no entry for "other" in the pie chart, and these 3919 entries are basically ignored. Which is why the percentages are all completely off.

> db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"e-bike"}).count()
25
> db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"pilot_ebike"}).count()
3919

This is a bug that needs to be fixed.

It would also help if you would clarify which dataset you were working with (e.g CEO full pilot, XXX program) so we would know which labels to expect.

[iantei]

@iantei the two graphs that you discuss (in great depth) in #91 (comment) are basically (1) and (2) from my comment.

The graph which I have posted above is the following :

1. walk->moped, default mapping.
2. walk->moped, with dynamic_config - but with changes on the dynamic_config which I updated to incorporate other attributes. So, these two are cases of (2) and (3) from your comment, with slight changes on (3)'s chart construction since it now includes e-bike with data related with pilot_ebike too.

[shankari]

@iantei do you see my follow-on comment?

But if the mappings are missing, then it should default to "Other". That is what happens in (2) - we don't have a mapping for "moped" so it all shows up as "other". But in (3) if we don't have a mapping for pilot_ebike, then it looks like the entries are just dropped. There is no entry for "other" in the pie chart, and these 3919 entries are basically ignored. Which is why the percentages are all completely off.

with

walk->moped, with dynamic_config - but with changes on the dynamic_config which I updated to incorporate other attributes.

You are saying that your solution will only work if the dynamic config incorporates all the modes that are present in the dataset. If it doesn't, then you will simply drop the dataset entries that are not in the dynamic config. Note that this will also result in graphs where the sum of the counts does not add up to the total, as in (3) above.

That is not acceptable. At the very minimum, the end-user can specify "other" in the app dropdown and enter their own entries. We cannot just drop them. There is a reason that we have the "other" option in the plots.

[iantei]

Introduced a change to incorporate the defaultdic for dic_translations with Other, similar to the dic_re default mapping.

Following steps were followed to test the changes.

1. Notebook generic_metrics run through automation:

With default setting: STUDY_CONFIG: stage-program

Results are:

(emission) root@52e499b537cb:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_metrics.ipynb default
/usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if r.status_code is not 200:
About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/stage-program.nrel-op.json
Successfully downloaded config with version 1 for Staging environment for testing programs only and data collection URL https://openpath-stage.nrel.gov/api/
Dynamic labels are not available.
Running at 2023-09-26T21:26:08.038439+00:00 with args Namespace(plot_notebook='generic_metrics.ipynb', program='default', date=None) for range (<Arrow [2020-09-01T00:00:00+00:00]>, <Arrow [2023-09-01T00:00:00+00:00]>)
Running at 2023-09-26T21:26:08.086231+00:00 with params [Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]
Running at 2023-09-26T21:26:24.788979+00:00 with params [Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]

With the change: STUDY_CONFIG: dev-emulator-study

ashrest2-35384s:em-public-dashboard ashrest2$ git diff docker-compose.dev.yml
diff --git a/docker-compose.dev.yml b/docker-compose.dev.yml
index fdbdbb8..9cbc4de 100644
--- a/docker-compose.dev.yml
+++ b/docker-compose.dev.yml
@@ -26,7 +26,7 @@ services:
       - DB_HOST=db
       - WEB_SERVER_HOST=0.0.0.0
       - CRON_MODE=
-      - STUDY_CONFIG=stage-program
+      - STUDY_CONFIG=dev-emulator-study
     ports:
       # ipynb in numbers
       - "47962:8888"

Results are: generic_metrics.ipynb

(emission) root@04909142f164:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/update_mappings.py mapping_dictionaries.ipynb
(emission) root@04909142f164:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_metrics.ipynb default
/usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if r.status_code is not 200:
About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/dev-emulator-study.nrel-op.json
Successfully downloaded config with version 1 for Development environment (study) and data collection URL default
Dynamic labels download was successful.
Running at 2023-09-26T21:36:28.624919+00:00 with args Namespace(plot_notebook='generic_metrics.ipynb', program='default', date=None) for range (<Arrow [2020-09-01T00:00:00+00:00]>, <Arrow [2023-09-01T00:00:00+00:00]>)
Running at 2023-09-26T21:36:28.677844+00:00 with params [Parameter('program', str, value='default'), Parameter('study_type', str, value='study'), Parameter('include_test_users', bool, value=False), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'bikeshare', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'scootershare', 'baseMode': 'E_SCOOTER', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.00894}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'moped', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.05555}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-bike': 'E-bike', 'bike': 'Regular Bike', 'bikeshare': 'Bikeshare', 'scootershare': 'Scooter share', 'drove_alone': 'Gas Car Drove Alone', 'shared_ride': 'Gas Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'moped': 'Moped', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Air', 'not_a_trip': 'Not a trip', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'es': {'walk': 'Caminando', 'e-bike': 'e-bicicleta', 'bike': 'Bicicleta', 'bikeshare': 'Bicicleta compartida', 'scootershare': 'Motoneta compartida', 'drove_alone': 'Coche de Gas, Condujo solo', 'shared_ride': 'Coche de Gas, Condujo con otros', 'e_car_drove_alone': 'e-coche, Condujo solo', 'e_car_shared_ride': 'e-coche, Condujo con ontras', 'moped': 'Ciclomotor', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Autobús', 'train': 'Tren', 'free_shuttle': 'Colectivo gratuito', 'air': 'Avión', 'not_a_trip': 'No es un viaje', 'home': 'Inicio', 'work': 'Trabajo', 'at_work': 'En el trabajo', 'school': 'Escuela', 'transit_transfer': 'Transbordo', 'shopping': 'Compras', 'meal': 'Comida', 'pick_drop_person': 'Recoger/ Entregar Individuo', 'pick_drop_item': 'Recoger/ Entregar Objeto', 'personal_med': 'Personal/ Médico', 'access_recreation': 'Acceder a Recreación', 'exercise': 'Recreación/ Ejercicio', 'entertainment': 'Entretenimiento/ Social', 'religious': 'Religioso', 'other': 'Otros'}}})]
Running at 2023-09-26T21:36:46.280240+00:00 with params [Parameter('program', str, value='default'), Parameter('study_type', str, value='study'), Parameter('include_test_users', bool, value=False), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'bikeshare', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'scootershare', 'baseMode': 'E_SCOOTER', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.00894}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'moped', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.05555}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-bike': 'E-bike', 'bike': 'Regular Bike', 'bikeshare': 'Bikeshare', 'scootershare': 'Scooter share', 'drove_alone': 'Gas Car Drove Alone', 'shared_ride': 'Gas Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'moped': 'Moped', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Air', 'not_a_trip': 'Not a trip', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'es': {'walk': 'Caminando', 'e-bike': 'e-bicicleta', 'bike': 'Bicicleta', 'bikeshare': 'Bicicleta compartida', 'scootershare': 'Motoneta compartida', 'drove_alone': 'Coche de Gas, Condujo solo', 'shared_ride': 'Coche de Gas, Condujo con otros', 'e_car_drove_alone': 'e-coche, Condujo solo', 'e_car_shared_ride': 'e-coche, Condujo con ontras', 'moped': 'Ciclomotor', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Autobús', 'train': 'Tren', 'free_shuttle': 'Colectivo gratuito', 'air': 'Avión', 'not_a_trip': 'No es un viaje', 'home': 'Inicio', 'work': 'Trabajo', 'at_work': 'En el trabajo', 'school': 'Escuela', 'transit_transfer': 'Transbordo', 'shopping': 'Compras', 'meal': 'Comida', 'pick_drop_person': 'Recoger/ Entregar Individuo', 'pick_drop_item': 'Recoger/ Entregar Objeto', 'personal_med': 'Personal/ Médico', 'access_recreation': 'Acceder a Recreación', 'exercise': 'Recreación/ Ejercicio', 'entertainment': 'Entretenimiento/ Social', 'religious': 'Religioso', 'other': 'Otros'}}})]
red_ride': 'Coche de Gas, Condujo con otros', 'e_car_drove_alone': 'e-coche, Condujo solo', 'e_car_shared_ride': 'e-coche, Condujo con ontras', 'moped': 'Ciclomotor', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Autobús', 'train': 'Tren', 'free_shuttle': 'Colectivo gratuito', 'air': 'Avión', 'not_a_trip': 'No es un viaje', 'home': 'Inicio', 'work': 'Trabajo', 'at_work': 'En el trabajo', 'school': 'Escuela', 'transit_transfer': 'Transbordo', 'shopping': 'Compras', 'meal': 'Comida', 'pick_drop_person': 'Recoger/ Entregar Individuo', 'pick_drop_item': 'Recoger/ Entregar Objeto', 'personal_med': 'Personal/ Médico', 'access_recreation': 'Acceder a Recreación', 'exercise': 'Recreación/ Ejercicio', 'entertainment': 'Entretenimiento/ Social', 'religious': 'Religioso', 'other': 'Otros'}}})]

3. Manual execution of notebook generic_metrics was executed with different scenarios:

Below charts illustrates mapping for Distribution of Mode_confirm attribute:

a. Walk - default mapping	b. Moped - default mapping	c. Moped - dynamic config mapping

Please refer to above comment for reference to see how walk has been converted to moped using mongodb.

[iantei]

@iantei do you see my follow-on comment?

But if the mappings are missing, then it should default to "Other". That is what happens in (2) - we don't have a mapping for "moped" so it all shows up as "other". But in (3) if we don't have a mapping for pilot_ebike, then it looks like the entries are just dropped. There is no entry for "other" in the pie chart, and these 3919 entries are basically ignored. Which is why the percentages are all completely off.

with

walk->moped, with dynamic_config - but with changes on the dynamic_config which I updated to incorporate other attributes.

You are saying that your solution will only work if the dynamic config incorporates all the modes that are present in the dataset. If it doesn't, then you will simply drop the dataset entries that are not in the dynamic config. Note that this will also result in graphs where the sum of the counts does not add up to the total, as in (3) above.

That is not acceptable. At the very minimum, the end-user can specify "other" in the app dropdown and enter their own entries. We cannot just drop them. There is a reason that we have the "other" option in the plots.

I have resolved the issue you've mentioned with the latest commit, where I introduced defaultdict for dic_translations similar to dic_re.

[shankari]

@iantei just FYI, you can and should format your messages to be more concise. For example, you can put the detailed message into a <details></details> block, you can put the images side by side in a table etc. This will avoid a "wall of text" in the issue.

I am also now curious about your testing procedure. You list

1. Notebook generic_metrics run through automation:
2. Manual execution of notebook generic_metrics was executed with different scenarios:

Does this mean that you are running the two steps separately? Concretely, does this mean that in step (2), you are copy-pasting the dynamic config into the notebook?

In that case, that is not a true end-to-end test. You should check the results from the automated run of generic_metrics instead.

[iantei]

Completed end-to-end automated testing.

[iantei]

Adjusted changes for mode_specific_metrics and tested through automated notebook execution.

[iantei]

Re-designed the mapping for dynamic labels. Created individual dictionary for each MODE, PURPOSE AND REPLACE_MODE on the basis of each of the key present in label options.Eg. Create a dictionary for MODE, MODE.value would paired with en.translations's matching key's [MODE.values's value] value.

[iantei]

Testing Summary: For generic_metrics notebook:

Automated notebook testing:

generic_metrics with default mapping:

Execution:

(emission) root@8c468351d76a:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_metrics.ipynb default 
/usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if r.status_code is not 200:
About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/stage-program.nrel-op.json
Successfully downloaded config with version 1 for Staging environment for testing programs only and data collection URL https://openpath-stage.nrel.gov/api/
label_options is unavailable for the dynamic_config.
Running at 2023-10-07T23:02:49.274926+00:00 with args Namespace(plot_notebook='generic_metrics.ipynb', program='default', date=None) for range (<Arrow [2020-09-01T00:00:00+00:00]>, <Arrow [2023-10-01T00:00:00+00:00]>)
Running at 2023-10-07T23:02:49.328732+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]
Running at 2023-10-07T23:03:16.338121+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=9), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]
Running at 2023-10-07T23:03:24.028050+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=10), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]
Running at 2023-10-07T23:03:31.235615+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=11), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]

Results:

Number of Trips for each mode	Miles for each mode	Number of Trips for each purposes

generic_metrics with dynamic labels: STUDY_CONFIG: dev-emulator-program

ashrest2-35384s:em-public-dashboard ashrest2$ git diff docker-compose.dev.yml 
diff --git a/docker-compose.dev.yml b/docker-compose.dev.yml
index fdbdbb8..c2a4342 100644
--- a/docker-compose.dev.yml
+++ b/docker-compose.dev.yml
@@ -26,7 +26,7 @@ services:
       - DB_HOST=db
       - WEB_SERVER_HOST=0.0.0.0
       - CRON_MODE=
-      - STUDY_CONFIG=stage-program
+      - STUDY_CONFIG=dev-emulator-program
     ports:
       # ipynb in numbers
       - "47962:8888"

Execution:

(emission) root@3bf75c967845:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/update_mappings.py mapping_dictionaries.ipynb
(emission) root@3bf75c967845:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_metrics.ipynb default
/usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if r.status_code is not 200:
About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/dev-emulator-program.nrel-op.json
Successfully downloaded config with version 1 for Development environment (program) and data collection URL default
Dynamic labels download was successful.
Running at 2023-10-07T23:17:15.833595+00:00 with args Namespace(plot_notebook='generic_metrics.ipynb', program='default', date=None) for range (<Arrow [2020-09-01T00:00:00+00:00]>, <Arrow [2023-10-01T00:00:00+00:00]>)
Running at 2023-10-07T23:17:15.878257+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=False), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'bikeshare', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'scootershare', 'baseMode': 'E_SCOOTER', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.00894}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'moped', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.05555}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'REPLACED_MODE': [{'value': 'no_travel'}, {'value': 'walk'}, {'value': 'bike'}, {'value': 'bikeshare'}, {'value': 'scootershare'}, {'value': 'drove_alone'}, {'value': 'shared_ride'}, {'value': 'e_car_drove_alone'}, {'value': 'e_car_shared_ride'}, {'value': 'taxi'}, {'value': 'bus'}, {'value': 'train'}, {'value': 'free_shuttle'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-bike': 'E-bike', 'bike': 'Regular Bike', 'bikeshare': 'Bikeshare', 'scootershare': 'Scooter share', 'drove_alone': 'Gas Car Drove Alone', 'shared_ride': 'Gas Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'moped': 'Moped', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Air', 'not_a_trip': 'Not a trip', 'no_travel': 'No travel', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'es': {'walk': 'Caminando', 'e-bike': 'e-bicicleta', 'bike': 'Bicicleta', 'bikeshare': 'Bicicleta compartida', 'scootershare': 'Motoneta compartida', 'drove_alone': 'Coche de Gas, Condujo solo', 'shared_ride': 'Coche de Gas, Condujo con otros', 'e_car_drove_alone': 'e-coche, Condujo solo', 'e_car_shared_ride': 'e-coche, Condujo con ontras', 'moped': 'Ciclomotor', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Autobús', 'train': 'Tren', 'free_shuttle': 'Colectivo gratuito', 'air': 'Avión', 'not_a_trip': 'No es un viaje', 'no_travel': 'No viajar', 'home': 'Inicio', 'work': 'Trabajo', 'at_work': 'En el trabajo', 'school': 'Escuela', 'transit_transfer': 'Transbordo', 'shopping': 'Compras', 'meal': 'Comida', 'pick_drop_person': 'Recoger/ Entregar Individuo', 'pick_drop_item': 'Recoger/ Entregar Objeto', 'personal_med': 'Personal/ Médico', 'access_recreation': 'Acceder a Recreación', 'exercise': 'Recreación/ Ejercicio', 'entertainment': 'Entretenimiento/ Social', 'religious': 'Religioso', 'other': 'Otros'}}})]

Results:	Number of Trips for each mode	Miles for each mode	Number of Trips for each purposes

[iantei]

There are difference in representation of charts above because of a few differences between mode_labels.csv and example-program-label-options.

There is no Air mode of travel in mode_labels.csv compared to example-program-label-options. While there's pilot_ebike mapped to E-Bike in mode_labels.csv which is missing in case of example-program-label-options. Similarly, for mode_purpose, we have To Work, At Work in comparison to the existing mapping present for purpose_labels.csv

[iantei]

Updated the mode_confirm from "Walk"->Moped using the following script via Mongo CLI:

var result_walk = db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"walk"}, {"_id": 1})

var resultArray = result_walk.toArray();
for (var i = 0; i < resultArray.length; i++) {     var doc = resultArray[i];     var docId = doc._id;     var updateResult = db.Stage_analysis_timeseries.update(         { "_id": docId, "data.user_input.mode_confirm": "walk" },         { "$set": { "data.user_input.mode_confirm": "moped" } }     );     if (updateResult.nModified > 0) {         print(`Updated document with _id: ${docId}`);     } }

Executed the generic_metrics notebook:

(emission) root@3bf75c967845:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_metrics.ipynb default
/usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if r.status_code is not 200:
About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/dev-emulator-program.nrel-op.json
Successfully downloaded config with version 1 for Development environment (program) and data collection URL default
Dynamic labels download was successful.
Running at 2023-10-08T00:02:07.465951+00:00 with args Namespace(plot_notebook='generic_metrics.ipynb', program='default', date=None) for range (<Arrow [2020-09-01T00:00:00+00:00]>, <Arrow [2023-10-01T00:00:00+00:00]>)
Running at 2023-10-08T00:02:07.588486+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=False), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'bikeshare', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'scootershare', 'baseMode': 'E_SCOOTER', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.00894}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'moped', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.05555}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'REPLACED_MODE': [{'value': 'no_travel'}, {'value': 'walk'}, {'value': 'bike'}, {'value': 'bikeshare'}, {'value': 'scootershare'}, {'value': 'drove_alone'}, {'value': 'shared_ride'}, {'value': 'e_car_drove_alone'}, {'value': 'e_car_shared_ride'}, {'value': 'taxi'}, {'value': 'bus'}, {'value': 'train'}, {'value': 'free_shuttle'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-bike': 'E-bike', 'bike': 'Regular Bike', 'bikeshare': 'Bikeshare', 'scootershare': 'Scooter share', 'drove_alone': 'Gas Car Drove Alone', 'shared_ride': 'Gas Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'moped': 'Moped', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Air', 'not_a_trip': 'Not a trip', 'no_travel': 'No travel', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'es': {'walk': 'Caminando', 'e-bike': 'e-bicicleta', 'bike': 'Bicicleta', 'bikeshare': 'Bicicleta compartida', 'scootershare': 'Motoneta compartida', 'drove_alone': 'Coche de Gas, Condujo solo', 'shared_ride': 'Coche de Gas, Condujo con otros', 'e_car_drove_alone': 'e-coche, Condujo solo', 'e_car_shared_ride': 'e-coche, Condujo con ontras', 'moped': 'Ciclomotor', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Autobús', 'train': 'Tren', 'free_shuttle': 'Colectivo gratuito', 'air': 'Avión', 'not_a_trip': 'No es un viaje', 'no_travel': 'No viajar', 'home': 'Inicio', 'work': 'Trabajo', 'at_work': 'En el trabajo', 'school': 'Escuela', 'transit_transfer': 'Transbordo', 'shopping': 'Compras', 'meal': 'Comida', 'pick_drop_person': 'Recoger/ Entregar Individuo', 'pick_drop_item': 'Recoger/ Entregar Objeto', 'personal_med': 'Personal/ Médico', 'access_recreation': 'Acceder a Recreación', 'exercise': 'Recreación/ Ejercicio', 'entertainment': 'Entretenimiento/ Social', 'religious': 'Religioso', 'other': 'Otros'}}})]

Result:

Number of Trips for each mode	Miles for each Mode

The results are identical in case the Walk is replaced with Moped

[iantei]

Executed the following automated notebook execution for generic_timeseries:

With STUDY_CONFIG=dev-emulator-program

(emission) root@4d905fc6836e:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/update_mappings.py mapping_dictionaries.ipynb
(emission) root@4d905fc6836e:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_timeseries.ipynb default
/usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if r.status_code is not 200:
About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/dev-emulator-program.nrel-op.json
Successfully downloaded config with version 1 for Development environment (program) and data collection URL default
Dynamic labels download was successful.
Running at 2023-10-10T02:26:26.675808+00:00 with args Namespace(plot_notebook='generic_timeseries.ipynb', program='default', date=None) for range (<Arrow [2020-09-01T00:00:00+00:00]>, <Arrow [2023-10-01T00:00:00+00:00]>)
Running at 2023-10-10T02:26:26.715085+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=False), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'bikeshare', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'scootershare', 'baseMode': 'E_SCOOTER', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.00894}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'moped', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.05555}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'REPLACED_MODE': [{'value': 'no_travel'}, {'value': 'walk'}, {'value': 'bike'}, {'value': 'bikeshare'}, {'value': 'scootershare'}, {'value': 'drove_alone'}, {'value': 'shared_ride'}, {'value': 'e_car_drove_alone'}, {'value': 'e_car_shared_ride'}, {'value': 'taxi'}, {'value': 'bus'}, {'value': 'train'}, {'value': 'free_shuttle'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-bike': 'E-bike', 'bike': 'Regular Bike', 'bikeshare': 'Bikeshare', 'scootershare': 'Scooter share', 'drove_alone': 'Gas Car Drove Alone', 'shared_ride': 'Gas Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'moped': 'Moped', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Air', 'not_a_trip': 'Not a trip', 'no_travel': 'No travel', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'es': {'walk': 'Caminando', 'e-bike': 'e-bicicleta', 'bike': 'Bicicleta', 'bikeshare': 'Bicicleta compartida', 'scootershare': 'Motoneta compartida', 'drove_alone': 'Coche de Gas, Condujo solo', 'shared_ride': 'Coche de Gas, Condujo con otros', 'e_car_drove_alone': 'e-coche, Condujo solo', 'e_car_shared_ride': 'e-coche, Condujo con ontras', 'moped': 'Ciclomotor', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Autobús', 'train': 'Tren', 'free_shuttle': 'Colectivo gratuito', 'air': 'Avión', 'not_a_trip': 'No es un viaje', 'no_travel': 'No viajar', 'home': 'Inicio', 'work': 'Trabajo', 'at_work': 'En el trabajo', 'school': 'Escuela', 'transit_transfer': 'Transbordo', 'shopping': 'Compras', 'meal': 'Comida', 'pick_drop_person': 'Recoger/ Entregar Individuo', 'pick_drop_item': 'Recoger/ Entregar Objeto', 'personal_med': 'Personal/ Médico', 'access_recreation': 'Acceder a Recreación', 'exercise': 'Recreación/ Ejercicio', 'entertainment': 'Entretenimiento/ Social', 'religious': 'Religioso', 'other': 'Otros'}}})]

Results: For Year = 2022 For month = 5	Timeseries of emission	Timeseries of emission per mile	Number of Active Users	Daily Aggregate Mode Share

[iantei]

Made the following changes:

ashrest2-35384s:em-public-dashboard ashrest2$ git diff docker-compose.dev.yml
diff --git a/docker-compose.dev.yml b/docker-compose.dev.yml
index fdbdbb8..c2a4342 100644
--- a/docker-compose.dev.yml
+++ b/docker-compose.dev.yml
@@ -26,7 +26,7 @@ services:
       - DB_HOST=db
       - WEB_SERVER_HOST=0.0.0.0
       - CRON_MODE=
-      - STUDY_CONFIG=stage-program
+      - STUDY_CONFIG=dev-emulator-program
     ports:
       # ipynb in numbers
       - "47962:8888"

The following testing has been conducted with the changes to index.html to hide energy metrics when dynamic labels is available. Loaded the public dashboard in dev mode by passing study_config as follows:

Default	dev-emulator-program	dev-emulator-study
http://localhost:3274/	http://localhost:3274/?study_config=dev-emulator-program	http://localhost:3274/?study_config=dev-emulator-study

[shankari]

@iantei can you please edit the comments and put the screenshots into tables for easier viewing? see the example at https://github.com/e-mission/em-public-dashboard/pull/91#pullrequestreview-1640942981

[iantei]

@iantei can you please edit the comments and put the screenshots into tables for easier viewing? see the example at #91 (review)

Thank you for the recommendation. I have edited the above screenshots into tabular structure for easy viewing.

[iantei]

Could you test this against the custom labels for the Lao deployment? https://github.com/e-mission/nrel-openpath-deploy-configs/blob/main/label_options/usaid-laos-ev-label-options.json

Following scenarios have been tested:

Changes to test with usaid-laos-ev:

ashrest2-35384s:em-public-dashboard ashrest2$ git diff docker-compose.dev.yml
diff --git a/docker-compose.dev.yml b/docker-compose.dev.yml
index fdbdbb8..c12adba 100644
--- a/docker-compose.dev.yml
+++ b/docker-compose.dev.yml
@@ -26,7 +26,7 @@ services:
       - DB_HOST=db
       - WEB_SERVER_HOST=0.0.0.0
       - CRON_MODE=
-      - STUDY_CONFIG=stage-program
+      - STUDY_CONFIG=usaid-laos-ev
     ports:
       # ipynb in numbers
       - "47962:8888"

The screenshot of changes have been taken after loading: http://localhost:3274/?study_config=usaid-laos-ev

Generic_metrics Jupyter notebook automatic execution:

**Generic_metrics** ``` (emission) root@48dce2c1ad0e:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/update_mappings.py mapping_dictionaries.ipynb (emission) root@48dce2c1ad0e:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_metrics.ipynb default /usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="? if r.status_code is not 200: About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/usaid-laos-ev.nrel-op.json Successfully downloaded config with version 1 for USAID-NREL Support for Electric Vehicle Readiness and data collection URL https://USAID-laos-EV-openpath.nrel.gov/api/ Dynamic labels download was successful for nrel-openpath-deploy-configs: usaid-laos-ev Running at 2023-10-24T16:29:33.571079+00:00 with args Namespace(plot_notebook='generic_metrics.ipynb', program='default', date=None) for range (, ) Running at 2023-10-24T16:29:33.612373+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='study'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-auto_rickshaw', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.085416859}, {'value': 'auto_rickshaw', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.231943784}, {'value': 'motorcycle', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.113143309}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-auto_rickshaw': 'E-tuk tuk', 'auto_rickshaw': 'Tuk Tuk', 'motorcycle': 'Motorcycle', 'e-bike': 'E-bike', 'bike': 'Bicycle', 'drove_alone': 'Car Drove Alone', 'shared_ride': 'Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'taxi': 'Taxi/Loca/inDrive', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Airplane', 'not_a_trip': 'Not a trip', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'lo': {'walk': 'ດ້ວຍການຍ່າງ', 'e-auto_rickshaw': 'ລົດ 3 ລໍ້ໄຟຟ້າ ຫລື ຕຸກຕຸກໄຟຟ້າ', 'auto_rickshaw': 'ເດີນທາດ້ວຍ ລົດຕຸກຕຸກ ຫລື ລົດສາມລໍ້', 'motorcycle': 'ລົດຈັກ', 'e-bike': 'ວຍລົດຈັກໄຟຟ້າ', 'bike': 'ລົດຖີບ', 'drove_alone': 'ເດີນທາງ ດ້ວຍລົດໃຫ່ຍ ເຊີ່ງເປັນລົດທີ່ຂັບເອງ', 'shared_ride': 'ເດີນທາງດ້ວຍລົດໃຫ່ຍ ຮ່ວມກັບລົດຄົນອຶ່ນ', 'e_car_drove_alone': 'ດ້ວຍການຂັບລົດໄຟຟ້າໄປເອງ', 'e_car_shared_ride': 'ດ້ວຍການຈ້າງລົດໄຟຟ້າໄປ', 'taxi': 'ແທັກຊີ', 'bus': 'ລົດເມ', 'train': 'ລົດໄຟ', 'free_shuttle': 'ລົດຮັບສົ່ງຟຣີ', 'air': 'ຍົນ', 'not_a_trip': 'ບໍ່ແມ່ນການເດີນທາງ', 'home': 'ບ້ານ', 'work': 'ໄປເຮັດວຽກ', 'at_work': 'ຢູ່ບ່ອນເຮັດວຽກ', 'school': 'ໄປໂຮງຮຽນ', 'transit_transfer': 'ການຖ່າຍໂອນການເດີນທາງ', 'shopping': 'ຊອບປິ້ງ', 'meal': 'ອາຫານ', 'pick_drop_person': 'ໄປຮັບ ຫລື ສົນ ຄົນ', 'pick_drop_item': 'ໄປຮັບ ຫລື ສົ່ງສິນຄ້າ', 'personal_med': 'ໄປຫາໝໍ', 'access_recreation': 'ເຂົ້າເຖິງການພັກຜ່ອນ', 'exercise': 'ພັກຜ່ອນ/ອອກກຳລັງກາຍ', 'entertainment': 'ບັນເທີງ/ສັງຄົມ', 'religious': 'ຈຸດປະສົງທາງສາດສະໜາ', 'other': 'ອື່ນໆ'}}})] ```

Number of Trips	Trips miles of mode	Trip count by purpose

Generic_Timeseries notebook automatic execution:

``` **Generic_timeseries** (emission) root@48dce2c1ad0e:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_timeseries.ipynb default /usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="? if r.status_code is not 200: About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/usaid-laos-ev.nrel-op.json Successfully downloaded config with version 1 for USAID-NREL Support for Electric Vehicle Readiness and data collection URL https://USAID-laos-EV-openpath.nrel.gov/api/ Dynamic labels download was successful for nrel-openpath-deploy-configs: usaid-laos-ev Running at 2023-10-24T17:45:10.258768+00:00 with args Namespace(plot_notebook='generic_timeseries.ipynb', program='default', date=None) for range (, ) Running at 2023-10-24T17:45:10.294902+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='study'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-auto_rickshaw', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.085416859}, {'value': 'auto_rickshaw', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.231943784}, {'value': 'motorcycle', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.113143309}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-auto_rickshaw': 'E-tuk tuk', 'auto_rickshaw': 'Tuk Tuk', 'motorcycle': 'Motorcycle', 'e-bike': 'E-bike', 'bike': 'Bicycle', 'drove_alone': 'Car Drove Alone', 'shared_ride': 'Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'taxi': 'Taxi/Loca/inDrive', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Airplane', 'not_a_trip': 'Not a trip', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'lo': {'walk': 'ດ້ວຍການຍ່າງ', 'e-auto_rickshaw': 'ລົດ 3 ລໍ້ໄຟຟ້າ ຫລື ຕຸກຕຸກໄຟຟ້າ', 'auto_rickshaw': 'ເດີນທາດ້ວຍ ລົດຕຸກຕຸກ ຫລື ລົດສາມລໍ້', 'motorcycle': 'ລົດຈັກ', 'e-bike': 'ວຍລົດຈັກໄຟຟ້າ', 'bike': 'ລົດຖີບ', 'drove_alone': 'ເດີນທາງ ດ້ວຍລົດໃຫ່ຍ ເຊີ່ງເປັນລົດທີ່ຂັບເອງ', 'shared_ride': 'ເດີນທາງດ້ວຍລົດໃຫ່ຍ ຮ່ວມກັບລົດຄົນອຶ່ນ', 'e_car_drove_alone': 'ດ້ວຍການຂັບລົດໄຟຟ້າໄປເອງ', 'e_car_shared_ride': 'ດ້ວຍການຈ້າງລົດໄຟຟ້າໄປ', 'taxi': 'ແທັກຊີ', 'bus': 'ລົດເມ', 'train': 'ລົດໄຟ', 'free_shuttle': 'ລົດຮັບສົ່ງຟຣີ', 'air': 'ຍົນ', 'not_a_trip': 'ບໍ່ແມ່ນການເດີນທາງ', 'home': 'ບ້ານ', 'work': 'ໄປເຮັດວຽກ', 'at_work': 'ຢູ່ບ່ອນເຮັດວຽກ', 'school': 'ໄປໂຮງຮຽນ', 'transit_transfer': 'ການຖ່າຍໂອນການເດີນທາງ', 'shopping': 'ຊອບປິ້ງ', 'meal': 'ອາຫານ', 'pick_drop_person': 'ໄປຮັບ ຫລື ສົນ ຄົນ', 'pick_drop_item': 'ໄປຮັບ ຫລື ສົ່ງສິນຄ້າ', 'personal_med': 'ໄປຫາໝໍ', 'access_recreation': 'ເຂົ້າເຖິງການພັກຜ່ອນ', 'exercise': 'ພັກຜ່ອນ/ອອກກຳລັງກາຍ', 'entertainment': 'ບັນເທີງ/ສັງຄົມ', 'religious': 'ຈຸດປະສົງທາງສາດສະໜາ', 'other': 'ອື່ນໆ'}}})] ```

Timeseries of all mode shares	Timeseries of emissions per kilometer	Timeseries of emissions

Timeseries of energy is unavailable since we have dynamic labels for Laos.

Both Mode_specific_timeseries and energy_calculations failed since this is study specific.

Similarly, notebook execution for sensed notebook failed since we have some NaN values in the dataset used - as justified above.

[iantei]

Since we don't have auto_rickshaw in the dataset I have been using, I changed the values of walk to auto_rickshaw to see its illustration.

Script to execute changes from `walk` to `auto_rickshaw`

``` > db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"walk"}).count() 4359 > db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"auto_rickshaw"}).count() 0 var result_walk = db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"walk"}, {"_id": 1}) var resultArray = result_walk.toArray(); for (var i = 0; i < resultArray.length; i++) { var doc = resultArray[i]; var docId = doc._id; var updateResult = db.Stage_analysis_timeseries.update( { "_id": docId, "data.user_input.mode_confirm": "walk" }, { "$set": { "data.user_input.mode_confirm": "auto_rickshaw" } } ); if (updateResult.nModified > 0) { print(`Updated document with _id: ${docId}`); } } > db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"walk"}).count() 0 > db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"auto_rickshaw"}).count() 4359 ```

Results have been outlined below after execution of generic_metrics and generic_timeseries notebook automatically. The results have been viewed through loading http://localhost:3274/?study_config=usaid-laos-ev

For generic_metrics:

generic_metrics execution process

``` (emission) root@48dce2c1ad0e:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_metrics.ipynb default /usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="? if r.status_code is not 200: About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/usaid-laos-ev.nrel-op.json Successfully downloaded config with version 1 for USAID-NREL Support for Electric Vehicle Readiness and data collection URL https://USAID-laos-EV-openpath.nrel.gov/api/ Dynamic labels download was successful for nrel-openpath-deploy-configs: usaid-laos-ev Running at 2023-10-24T18:33:20.897448+00:00 with args Namespace(plot_notebook='generic_metrics.ipynb', program='default', date=None) for range (, ) Running at 2023-10-24T18:33:21.321147+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='study'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-auto_rickshaw', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.085416859}, {'value': 'auto_rickshaw', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.231943784}, {'value': 'motorcycle', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.113143309}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-auto_rickshaw': 'E-tuk tuk', 'auto_rickshaw': 'Tuk Tuk', 'motorcycle': 'Motorcycle', 'e-bike': 'E-bike', 'bike': 'Bicycle', 'drove_alone': 'Car Drove Alone', 'shared_ride': 'Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'taxi': 'Taxi/Loca/inDrive', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Airplane', 'not_a_trip': 'Not a trip', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'lo': {'walk': 'ດ້ວຍການຍ່າງ', 'e-auto_rickshaw': 'ລົດ 3 ລໍ້ໄຟຟ້າ ຫລື ຕຸກຕຸກໄຟຟ້າ', 'auto_rickshaw': 'ເດີນທາດ້ວຍ ລົດຕຸກຕຸກ ຫລື ລົດສາມລໍ້', 'motorcycle': 'ລົດຈັກ', 'e-bike': 'ວຍລົດຈັກໄຟຟ້າ', 'bike': 'ລົດຖີບ', 'drove_alone': 'ເດີນທາງ ດ້ວຍລົດໃຫ່ຍ ເຊີ່ງເປັນລົດທີ່ຂັບເອງ', 'shared_ride': 'ເດີນທາງດ້ວຍລົດໃຫ່ຍ ຮ່ວມກັບລົດຄົນອຶ່ນ', 'e_car_drove_alone': 'ດ້ວຍການຂັບລົດໄຟຟ້າໄປເອງ', 'e_car_shared_ride': 'ດ້ວຍການຈ້າງລົດໄຟຟ້າໄປ', 'taxi': 'ແທັກຊີ', 'bus': 'ລົດເມ', 'train': 'ລົດໄຟ', 'free_shuttle': 'ລົດຮັບສົ່ງຟຣີ', 'air': 'ຍົນ', 'not_a_trip': 'ບໍ່ແມ່ນການເດີນທາງ', 'home': 'ບ້ານ', 'work': 'ໄປເຮັດວຽກ', 'at_work': 'ຢູ່ບ່ອນເຮັດວຽກ', 'school': 'ໄປໂຮງຮຽນ', 'transit_transfer': 'ການຖ່າຍໂອນການເດີນທາງ', 'shopping': 'ຊອບປິ້ງ', 'meal': 'ອາຫານ', 'pick_drop_person': 'ໄປຮັບ ຫລື ສົນ ຄົນ', 'pick_drop_item': 'ໄປຮັບ ຫລື ສົ່ງສິນຄ້າ', 'personal_med': 'ໄປຫາໝໍ', 'access_recreation': 'ເຂົ້າເຖິງການພັກຜ່ອນ', 'exercise': 'ພັກຜ່ອນ/ອອກກຳລັງກາຍ', 'entertainment': 'ບັນເທີງ/ສັງຄົມ', 'religious': 'ຈຸດປະສົງທາງສາດສະໜາ', 'other': 'ອື່ນໆ'}}})] ```

Results:

Number of Trips	Trip count by purpose	Trip miles by mode

For generic_timeseries:

Execution steps for generic_timeseries notebook automatic

(emission) root@48dce2c1ad0e:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_timeseries.ipynb default /usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="? if r.status_code is not 200: About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/usaid-laos-ev.nrel-op.json Successfully downloaded config with version 1 for USAID-NREL Support for Electric Vehicle Readiness and data collection URL https://USAID-laos-EV-openpath.nrel.gov/api/ Dynamic labels download was successful for nrel-openpath-deploy-configs: usaid-laos-ev Running at 2023-10-24T18:41:35.940356+00:00 with args Namespace(plot_notebook='generic_timeseries.ipynb', program='default', date=None) for range (, ) Running at 2023-10-24T18:41:35.978482+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='study'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-auto_rickshaw', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.085416859}, {'value': 'auto_rickshaw', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.231943784}, {'value': 'motorcycle', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.113143309}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-auto_rickshaw': 'E-tuk tuk', 'auto_rickshaw': 'Tuk Tuk', 'motorcycle': 'Motorcycle', 'e-bike': 'E-bike', 'bike': 'Bicycle', 'drove_alone': 'Car Drove Alone', 'shared_ride': 'Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'taxi': 'Taxi/Loca/inDrive', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Airplane', 'not_a_trip': 'Not a trip', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'lo': {'walk': 'ດ້ວຍການຍ່າງ', 'e-auto_rickshaw': 'ລົດ 3 ລໍ້ໄຟຟ້າ ຫລື ຕຸກຕຸກໄຟຟ້າ', 'auto_rickshaw': 'ເດີນທາດ້ວຍ ລົດຕຸກຕຸກ ຫລື ລົດສາມລໍ້', 'motorcycle': 'ລົດຈັກ', 'e-bike': 'ວຍລົດຈັກໄຟຟ້າ', 'bike': 'ລົດຖີບ', 'drove_alone': 'ເດີນທາງ ດ້ວຍລົດໃຫ່ຍ ເຊີ່ງເປັນລົດທີ່ຂັບເອງ', 'shared_ride': 'ເດີນທາງດ້ວຍລົດໃຫ່ຍ ຮ່ວມກັບລົດຄົນອຶ່ນ', 'e_car_drove_alone': 'ດ້ວຍການຂັບລົດໄຟຟ້າໄປເອງ', 'e_car_shared_ride': 'ດ້ວຍການຈ້າງລົດໄຟຟ້າໄປ', 'taxi': 'ແທັກຊີ', 'bus': 'ລົດເມ', 'train': 'ລົດໄຟ', 'free_shuttle': 'ລົດຮັບສົ່ງຟຣີ', 'air': 'ຍົນ', 'not_a_trip': 'ບໍ່ແມ່ນການເດີນທາງ', 'home': 'ບ້ານ', 'work': 'ໄປເຮັດວຽກ', 'at_work': 'ຢູ່ບ່ອນເຮັດວຽກ', 'school': 'ໄປໂຮງຮຽນ', 'transit_transfer': 'ການຖ່າຍໂອນການເດີນທາງ', 'shopping': 'ຊອບປິ້ງ', 'meal': 'ອາຫານ', 'pick_drop_person': 'ໄປຮັບ ຫລື ສົນ ຄົນ', 'pick_drop_item': 'ໄປຮັບ ຫລື ສົ່ງສິນຄ້າ', 'personal_med': 'ໄປຫາໝໍ', 'access_recreation': 'ເຂົ້າເຖິງການພັກຜ່ອນ', 'exercise': 'ພັກຜ່ອນ/ອອກກຳລັງກາຍ', 'entertainment': 'ບັນເທີງ/ສັງຄົມ', 'religious': 'ຈຸດປະສົງທາງສາດສະໜາ', 'other': 'ອື່ນໆ'}}})]

Results for generic_timeseries:

Timeseries of emissions	Timeseries of emissions per kilometer	Timeseries of all mode shares All	Timeseries of active users

There is no timeseries of energy available.

Similar case are applicable for mode_specific_timeseries and energy_calculations which are only available for PROGRAM.

[iantei]

I have tried to use kg/kms computation for all of the CO2 emission computation, in case of both dynamic config mapping and default mapping. For the implementation, I have incorporated, there's a slight calculation issue where the miles to kms have not been accounted for emissions per trip. A. I could fix it by passing 'distance' over 'distance_miles'. OR B. Introduce if-else conditions in notebook to keep the previous lbs/miles for default mapping while creating kg/kms for the ones with dynamic config. As I recall, the eventual idea is to move altogether into Metric system. With that consideration I think it'd be a good idea to compute every calculation in reference with kg/kms. I will fix the above issues in accordance with it.

[shankari]

As I recall, the eventual idea is to move altogether into Metric system. With that consideration I think it'd be a good idea to compute every calculation in reference with kg/kms. I will fix the above issues in accordance with it.

That is not the eventual idea. The system is in fact completely metric under the hood.

However, in the US, which is the majority of our customer case, we need to use miles because that is the common term of usage. The correct solution is to read the imperialConfig flag from the dynamic config and use it to determine miles versus km.

So the eventual solution will have to support both metric and imperial calculations anyway.

[iantei]

There will be four different cases:

1. With label_options, and use_imperial : false:
   • use Metric systems.
2. With label_options, and use_imperial : true:
   • use Imperial systems.
3. Without label_options, and use_imperial: true:
   • use Imperial systems.
4. Without label_options, and use_imperial: true:
   • use Metric system.

For an example, there is use_imperial present on the config files, but not all config files with use_imperial have label_options like dev-emulator-timeuse.

For the simplicity of things for now, I propose to keep the existing implementation without dynamic config intact - just use the Imperial system. While adding some if-else conditions in the notebook to make required changes for ones with dynamic config.

[shankari]

For the simplicity of things for now, I propose to keep the existing implementation without dynamic config intact - just use the Imperial system. While adding some if-else conditions in the notebook to make required changes for ones with dynamic config.

I am not sure what you mean by this. The existing implementation is wrong for the non-dynamic config case. It uses a CO2/mile value, converts it to kg/mile but then labels it as kg/km.

I appreciate your desire to get this wrapped up, but I will not sign off on incorrect work.

[iantei]

I was not considering to conclude with the obvious known issue of misrepresentation of data from kg/mile as kg/kms.

I was proposing to correct the non-dynamic config case CO2 emission to use lbs/mile (Imperial system) - since most of the config file in the existing config represents use_imperial as True, and support dynamic config case CO2 emission to only use kg/kilometer (Metric case). I was only suggesting not to incorporate use_imperial right now, but I can incorporate it as well. That would require both representation of lbs/mile and kg/km in both dynamic config and non-dynamic config case as well.

[shankari]

I was only suggesting not to incorporate use_imperial right now, but I can incorporate it as well.

It is fine to not incorporate use_imperial right now. I didn't ask you to fix it. I just responded to the comment that "the final goal was metric". It is not. I only asked you to fix the computation.

That would require both representation of lbs/mile and kg/km in both dynamic config and non-dynamic config case as well.

That is not true. You can convert between km and miles just like you did between lbs and kg.

[iantei]

Updated: NOTE: This calculation is incorrect. CO2 emission chart comparison.

With Default Mapping	With dynamic config

I tried to deduce CO2 emission for each mode as kg per km. Studying the value for e-bike, it gave a reflection the value present on the config as kgCO2perkm is significantly higher compared to one we are using with the default mapping. This seem to reflect why the CO2 emission are different in both these cases.

[shankari]

@iantei I don't understand your math.

1. the column with the values 0.00927 and 0.005669 is Mode_confirm_kg_CO2, which already has the distance multiplied in. So I am not sure (i) why you are re-multiplying with the distance, and (ii) multiplying with the distance in meters when the coefficient is per km.
2. if you claim that the values in the dynamic and default configs are different, you can just convert the coefficients based on the units and confirm. I don't see that anywhere.
3. You can look up the previous computations for CO2 computations, and the re-verification of the values as part of the dashboard rewrite if you want to double-check the values.

[shankari]

@iantei I would like a response to https://github.com/e-mission/em-public-dashboard/pull/91#issuecomment-1783696951 before I review further.

I don't fully understand what the computation verification at the top of https://github.com/e-mission/em-public-dashboard/pull/91#issuecomment-1783101803 and would like to see it worked out for at least three rows - please include as text in here using code or $math$ mode

[iantei]

@iantei I don't understand your math.

1. the column with the values 0.00927 and 0.005669 is Mode_confirm_kg_CO2, which already has the distance multiplied in. So I am not sure (i) why you are re-multiplying with the distance, and (ii) multiplying with the distance in meters when the coefficient is per km.

There was some issues in the scaffolding.py computation of CO2 emissions which have been resolved in the above commits. The reference of calculation shown above https://github.com/e-mission/em-public-dashboard/pull/91#issuecomment-1783101803 is not right, which I have fixed, I will showcase the updated computation comparison below in following comments.

[iantei]

3. You can look up the previous computations for CO2 computations, and the re-verification of the values as part of the dashboard rewrite if you want to double-check the values.

Comparison for Default Mapping lb CO2 emission between current and older implementation

| Default Mapping (without any dynamic config changes) | Default Mapping (with dynamic config changes incorporated) | |--------|--------| | | | The left chart is from the older implementation, which does not have changes with the likes of dynamic config, while the right chart is the one which incorporates the recent changes. The `is_debug_mode` was set to `False` in this case. > You can look up the previous computations for CO2 computations, and the re-verification of the values as part of the dashboard rewrite if you want to double-check the values. From the above comparison, it's clear that the default mapping computation for CO2 Emission is intact with the latest changes introducing dynamic config. I re-cloned the `em-public-dashboard` `main` repo, without my code changes. And printed the CO2 emission chart from `energy_calculations` notebook. I appended the following code to display the output below: ``` print("Total lbs CO2 emissions [after dynamic config changes]") print(ebco2['total_lb_CO2_emissions']) ``` I didn't checked-in this code, because we are more interested in comparing the default mapping and dynamic config CO2 emissions.

Both the chart reflect same values. There is supposedly no issue with Default Mapping.

[iantei]

2. if you claim that the values in the dynamic and default configs are different, you can just convert the coefficients based on the units and confirm. I don't see that anywhere.

Detailed computation for CO2 emission in Default Mapping vs Dynamic Config

| Default Mapping | Dynamic Config| |--------|--------| | | | Lets specifically compare one entry in the above table. | Default Mapping | Dynamic Config | |--------|--------| | | | Reference of the kgCO2 per km from `label_options` - [https://github.com/e-mission/nrel-openpath-deploy-configs/blob/main/label_options/example-program-label-options.json](https://github.com/e-mission/nrel-openpath-deploy-configs/blob/main/label_options/example-program-label-options.json) ``` {"value":"e-bike", "baseMode":"E_BIKE", "met": {"ALL": {"range": [0, -1], "mets": 4.9}}, "kgCo2PerKm": 0.00728} ``` In the above case,`mode_confirm` is `e-bike`. Focus on the above screenshot would be on the dotted region of selection. `mode_confirm` = `e-bike` Calculating `kgCO2 per km` in 1. Default Mapping 2. Dynamic Config case: Unit of `distance`: `meter` Unit of `Mode_confirm_kg_CO2`: `kg` ``` 1. For Default Mapping: Computation for the first row: kgCO2 per km = Mode_confirm_kg_CO2 / distance * 1000 = 0.009123 / 1264.664028 * 1000 = 0.007213773617 ~ 0.00721 kg CO2 per km Similar computation for 2nd row: kgCO2 per km = 0.012797 / 1773.961885 * 1000 = 0.007213796 ~ 0.00721 kg CO2 per km kgCO2 per km = 0.009175 / 1271.871607 * 1000 = 0.007214257451 ~ 0.00721 kg CO2 per km ``` ``` 3. For Dynamic Mapping: Computation for the first row: kgCO2 per km = Mode_confirm_kg_CO2 / distance * 1000 = 0.009207 / 1264.664028 * 1000 = 0.007276003584 ~ 0.00728 kg CO2 per km (equivalent to `label_options` for `e-bike`). Computation for the second row: kgCO2 per km = Mode_confirm_kg_CO2/distance * 1000 = 0.012914 / 1773.961885 * 1000 = 0.007279750 ~ 0.00728 kg CO2 per km Computation for the third row: kgCO2 per km = Mode_confirm_kg_CO2/ distance * 1000 = 0.009259 / 1271.871607 * 1000 = 0.00727982286 ~ 0.00728 kg CO2 per km ``` Another approach of identifying kgCO2 per km for Default Mapping: Since `e-bike` is `electric` mode of transport. The following line of code ``` electric_col = (((df[distance]*df['ei_'+col]) + df['ei_trip_'+col])*0.001) * df['CO2_'+col] values_col = [gasoline_col,diesel_col,electric_col] df[col+'_lb_CO2'] = np.select(conditions_col, values_col) ``` can be used to compute the `Mode_confirm_lb_CO2` for `e-bike`. PMT = Personal miles traveled (PMT) is a standard measure of mobility that combines both the number and length of trips. [Personal Miles Traveled](https://www.fhwa.dot.gov/policy/otps/nextgen_stats/chap5.cfm) The computation would be: ``` df[distance] = 0.784092 miles df[ei_Mode_confirm] = 0.022 kWH/PMT df[ei_trip_Mode_confirm] = 0 kWH/trip df[CO2_Mode_confirm] = 1166 lb_CO2/MWH 1 lb = 0.453592 kg 1 mile = 1.60934 km Mode_confirm_lb_CO2 = ( (0.784092 * 0.022) + 0) * 0.001 * 1166 = 0.02011352798 lb_CO2/trip Deduction of the unit: ( miles * kWH/ (trip * miles) + kWH/trip ) * lb_CO2/kWH [converted from MWH to kWH with that 0.001 conversion rate]. = (kWH/ trip) * lb_CO2/kWH = lb_CO2/trip From the first row: distance_miles represent the trip. Mode_confirm_lb_CO2 per mile = Mode_confirm_lb_CO2/ distance_miles = 0.02011352798 / 0.784092 = 0.02565199999 Mode_confirm_kg_CO2 per km = 0.00722865918 kg/km = ~ 0.00723 kg/km ``` The `kg_CO2 per km` for `e-bike` in `Default Mapping` is different to `Dynamic Config` part. This can be inferred to other mode of commute as well, thereby there is a difference in the final output of the chart's Total CO2 emission for each Replaced Mode. | Total lb CO2 Emission (Default Mapping) | Total kg CO2 Emission (Dynamic Config) | |--------|--------| | | | | | |

The value of kgCO2 per km for Default Mapping (which is done with the reference from values in auxillary_files/energy_intensity.csv) and Dynamic Config (which is done with the reference from values in label_options) are different.

[iantei]

@iantei I would like a response to #91 (comment) before I review further.

I don't fully understand what the computation verification at the top of https://github.com/e-mission/em-public-dashboard/pull/91#issuecomment-1783101803 and would like to see it worked out for at least three rows - please include as text in here using code or math mode

I have added comments regarding these in above comments.

[shankari]

without reviewing the computations carefully, which I have not yet done, the challenge with the computational approach above is that it doesn't show that the values are actually different. It shows that reconstituting the coefficient by dividing the result by the distance is the same for the dynamic label case. But it does not show that this is the same as the coefficient in the default case (or how different it is) because in the default case, the coefficients are not specified in kg/km.

So to show this, you should convert the coefficient in the dynamic labels from kg/km to lbs/mile and then compute the coefficient in lbs/mile for the dynamic using the kWH/mile * lbs/kWH for the grid.

All of this is static, none of it is trip-specific so we can remove the potential cause of the computation being wrong for the default side.

If the values for both the dynamic and default computations were the same, we would not have to do this, since they would provide checks for each other. HOWEVER, the values are now different, so if we want to claim that it is because the coefficents are different, we need to show that the coefficients are the same "from first principles"

[iantei]

So to show this, you should convert the coefficient in the dynamic labels from kg/km to lbs/mile and then compute the coefficient in lbs/mile for the dynamic using the kWH/mile * lbs/kWH for the grid.

Calculation For e-bike: https://github.com/e-mission/nrel-openpath-deploy-configs/blob/main/label_options/example-program-label-options.json

{"value":"e-bike", "baseMode":"E_BIKE", "met": {"ALL": {"range": [0, -1], "mets": 4.9}}, "kgCo2PerKm": 0.00728}

Conversion of the coefficient in the dynamic labels from kgCO2/km to lb_CO2/mile: 1 kg = 2.20462 lb 1 km = 0.621371 mile "kgCo2PerKm": 0.00728 -> 0.00728 * 2.20462 / 0.621371 = 0.02582938953 $\frac{lb}{mile}$

Next, I understand you mean to compute the coefficient in lbs/mile for default mapping using the computation values available at energy_intensity.csv.

mode	fuel	(kWH)/trip	EI(kWH/PMT)	energy_intensity_factor	energy_intensity_units
E-bike	electric	0	0.022	0.022	kWH / PMT

CO2_factor	CO2_factor_units
1166	lb_CO2/MWH

Referring to this definition of PMT from National Household Travel Survey, PMT = Person Miles of Travel. PMT: The number of miles traveled by each person on a trip. The purpose is to account for all MILES traveled by all people.

Considering, we're just talking about a single person in this trip. The PMT unit can be considered as Mile.

Energy Intensity CO2_factor = 0.022 kWH/PMT 1166 lb_CO2/MWH = 0.022 $\frac{kWH}{mile}$ 0.001 1166 $\frac{lb}{kWH}$ [kWH gets cancelled from numerator and denominator.] = 0.025652 lb_CO2/mile

Conclusion From the above computation, it is clear that the values for CO2 emission in terms of lb_CO2/mile is different for default mapping (accounted from [energy_intensity.csv])(https://github.com/e-mission/em-public-dashboard/blob/main/viz_scripts/auxiliary_files/energy_intensity.csv) vs dynamic config (accounted from https://github.com/e-mission/nrel-openpath-deploy-configs/blob/main/label_options/usaid-laos-ev-label-options.json ).	With Default Mapping	With Dynamic Config
0.025652 lb_CO2/mile	0.02582938953 lb_CO2/mile

[iantei]

Executed the following testing for regression testing with default mapping STUDY_CONFIG=stage-program

Default Mapping: Automated notebook execution for `energy_calculations` and `generic_timeseries` notebook.

Energy_calculations.ipynb ``` (emission) root@651667db60c6:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py energy_calculations.ipynb default /usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="? if r.status_code is not 200: About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/stage-program.nrel-op.json Successfully downloaded config with version 1 for Staging environment for testing programs only and data collection URL https://openpath-stage.nrel.gov/api/ label_options is unavailable for the dynamic_config in stage-program Running at 2023-11-07T19:54:02.870477+00:00 with args Namespace(plot_notebook='energy_calculations.ipynb', program='default', date=None) for range (, ) Running at 2023-11-07T19:54:02.973217+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('mode_of_interest', str, value='e-bike'), Parameter('include_test_users', bool, value=True), Parameter('is_debug_mode', bool, value=False), Parameter('dynamic_labels', dict, value={})] ``` Generic_timeseries.ipynb: ``` (emission) root@651667db60c6:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_timeseries.ipynb default /usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="? if r.status_code is not 200: About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/stage-program.nrel-op.json Successfully downloaded config with version 1 for Staging environment for testing programs only and data collection URL https://openpath-stage.nrel.gov/api/ label_options is unavailable for the dynamic_config in stage-program Running at 2023-11-07T20:35:56.112032+00:00 with args Namespace(plot_notebook='generic_timeseries.ipynb', program='default', date=None) for range (, ) Running at 2023-11-07T20:35:56.152280+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})] Running at 2023-11-07T20:36:15.883235+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=9), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})] ``` Results: | Timeseries of emissions All Data | Timeseries of emissions per mile All Data | |--------|--------| | | |

Dynamic Labels: Automated notebook execution for `energy_calculations` and `generic_timeseries` notebook

Execution: ``` Executed the following testing for regression testing with default mapping STUDY_CONFIG=dev-emulator-program Loaded the web page using `http://localhost:3274/?study_config=dev-emulator-program` ``` Results: | Timeseries of emissions | Timeseries of emissions per | |--------|--------| | | |

[iantei]

The math still seems to be wrong.

I have given justification for the math I am using for the computation of CO2 emission - which is lb_CO2/ trip or kg_CO2/trip. Please let me know if that adds up rightfully.

And you have too much copy-pasted code.

Yes, I did realize that. I have re-factored both generic_timeseries and energy_calculations notebooks to avoid code duplication while keeping the functionality intact.

[shankari]

@iantei you need to fix your markdown. I have edited https://github.com/e-mission/em-public-dashboard/pull/91#issuecomment-1736338402 and https://github.com/e-mission/em-public-dashboard/pull/91#issuecomment-1800435273, for example, to be correct.

[iantei]

Accounting to above changes the generic_timeseries notebook has been tested for regression testing in both default mapping and dynamic config cases.

Notebook: `Generic Timeseries` with STUDY_CONFIG=`dev-emulator-program` | Dynamic Config

Steps executed: ``` (emission) root@dac7d8c525e4:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/update_mappings.py mapping_dictionaries.ipynb (emission) root@dac7d8c525e4:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_timeseries.ipynb default /usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="? if r.status_code is not 200: About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/dev-emulator-program.nrel-op.json Successfully downloaded config with version 1 for Development environment (program) and data collection URL default Dynamic labels download was successful for nrel-openpath-deploy-configs: dev-emulator-program Running at 2023-11-13T21:21:36.391448+00:00 with args Namespace(plot_notebook='generic_timeseries.ipynb', program='default', date=None) for range (, ) Running at 2023-11-13T21:21:36.428724+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=False), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'bikeshare', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'scootershare', 'baseMode': 'E_SCOOTER', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.00894}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'moped', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.05555}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'REPLACED_MODE': [{'value': 'no_travel'}, {'value': 'walk'}, {'value': 'bike'}, {'value': 'bikeshare'}, {'value': 'scootershare'}, {'value': 'drove_alone'}, {'value': 'shared_ride'}, {'value': 'e_car_drove_alone'}, {'value': 'e_car_shared_ride'}, {'value': 'taxi'}, {'value': 'bus'}, {'value': 'train'}, {'value': 'free_shuttle'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-bike': 'E-bike', 'bike': 'Regular Bike', 'bikeshare': 'Bikeshare', 'scootershare': 'Scooter share', 'drove_alone': 'Gas Car Drove Alone', 'shared_ride': 'Gas Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'moped': 'Moped', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Air', 'not_a_trip': 'Not a trip', 'no_travel': 'No travel', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'es': {'walk': 'Caminando', 'e-bike': 'e-bicicleta', 'bike': 'Bicicleta', 'bikeshare': 'Bicicleta compartida', 'scootershare': 'Motoneta compartida', 'drove_alone': 'Coche de Gas, Condujo solo', 'shared_ride': 'Coche de Gas, Condujo con otros', 'e_car_drove_alone': 'e-coche, Condujo solo', 'e_car_shared_ride': 'e-coche, Condujo con ontras', 'moped': 'Ciclomotor', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Autobús', 'train': 'Tren', 'free_shuttle': 'Colectivo gratuito', 'air': 'Avión', 'not_a_trip': 'No es un viaje', 'no_travel': 'No viajar', 'home': 'Inicio', 'work': 'Trabajo', 'at_work': 'En el trabajo', 'school': 'Escuela', 'transit_transfer': 'Transbordo', 'shopping': 'Compras', 'meal': 'Comida', 'pick_drop_person': 'Recoger/ Entregar Individuo', 'pick_drop_item': 'Recoger/ Entregar Objeto', 'personal_med': 'Personal/ Médico', 'access_recreation': 'Acceder a Recreación', 'exercise': 'Recreación/ Ejercicio', 'entertainment': 'Entretenimiento/ Social', 'religious': 'Religioso', 'other': 'Otros'}}})] Running at 2023-11-13T21:22:00.189112+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=9), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=False), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'bikeshare', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'scootershare', 'baseMode': 'E_SCOOTER', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.00894}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'moped', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.05555}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'REPLACED_MODE': [{'value': 'no_travel'}, {'value': 'walk'}, {'value': 'bike'}, {'value': 'bikeshare'}, {'value': 'scootershare'}, {'value': 'drove_alone'}, {'value': 'shared_ride'}, {'value': 'e_car_drove_alone'}, {'value': 'e_car_shared_ride'}, {'value': 'taxi'}, {'value': 'bus'}, {'value': 'train'}, {'value': 'free_shuttle'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-bike': 'E-bike', 'bike': 'Regular Bike', 'bikeshare': 'Bikeshare', 'scootershare': 'Scooter share', 'drove_alone': 'Gas Car Drove Alone', 'shared_ride': 'Gas Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'moped': 'Moped', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Air', 'not_a_trip': 'Not a trip', 'no_travel': 'No travel', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'es': {'walk': 'Caminando', 'e-bike': 'e-bicicleta', 'bike': 'Bicicleta', 'bikeshare': 'Bicicleta compartida', 'scootershare': 'Motoneta compartida', 'drove_alone': 'Coche de Gas, Condujo solo', 'shared_ride': 'Coche de Gas, Condujo con otros', 'e_car_drove_alone': 'e-coche, Condujo solo', 'e_car_shared_ride': 'e-coche, Condujo con ontras', 'moped': 'Ciclomotor', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Autobús', 'train': 'Tren', 'free_shuttle': 'Colectivo gratuito', 'air': 'Avión', 'not_a_trip': 'No es un viaje', 'no_travel': 'No viajar', 'home': 'Inicio', 'work': 'Trabajo', 'at_work': 'En el trabajo', 'school': 'Escuela', 'transit_transfer': 'Transbordo', 'shopping': 'Compras', 'meal': 'Comida', 'pick_drop_person': 'Recoger/ Entregar Individuo', 'pick_drop_item': 'Recoger/ Entregar Objeto', 'personal_med': 'Personal/ Médico', 'access_recreation': 'Acceder a Recreación', 'exercise': 'Recreación/ Ejercicio', 'entertainment': 'Entretenimiento/ Social', 'religious': 'Religioso', 'other': 'Otros'}}})] Running at 2023-11-13T21:22:05.560116+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=10), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=False), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'bikeshare', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'scootershare', 'baseMode': 'E_SCOOTER', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.00894}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'moped', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.05555}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'REPLACED_MODE': [{'value': 'no_travel'}, {'value': 'walk'}, {'value': 'bike'}, {'value': 'bikeshare'}, {'value': 'scootershare'}, {'value': 'drove_alone'}, {'value': 'shared_ride'}, {'value': 'e_car_drove_alone'}, {'value': 'e_car_shared_ride'}, {'value': 'taxi'}, {'value': 'bus'}, {'value': 'train'}, {'value': 'free_shuttle'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-bike': 'E-bike', 'bike': 'Regular Bike', 'bikeshare': 'Bikeshare', 'scootershare': 'Scooter share', 'drove_alone': 'Gas Car Drove Alone', 'shared_ride': 'Gas Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'moped': 'Moped', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Air', 'not_a_trip': 'Not a trip', 'no_travel': 'No travel', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'es': {'walk': 'Caminando', 'e-bike': 'e-bicicleta', 'bike': 'Bicicleta', 'bikeshare': 'Bicicleta compartida', 'scootershare': 'Motoneta compartida', 'drove_alone': 'Coche de Gas, Condujo solo', 'shared_ride': 'Coche de Gas, Condujo con otros', 'e_car_drove_alone': 'e-coche, Condujo solo', 'e_car_shared_ride': 'e-coche, Condujo con ontras', 'moped': 'Ciclomotor', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Autobús', 'train': 'Tren', 'free_shuttle': 'Colectivo gratuito', 'air': 'Avión', 'not_a_trip': 'No es un viaje', 'no_travel': 'No viajar', 'home': 'Inicio', 'work': 'Trabajo', 'at_work': 'En el trabajo', 'school': 'Escuela', 'transit_transfer': 'Transbordo', 'shopping': 'Compras', 'meal': 'Comida', 'pick_drop_person': 'Recoger/ Entregar Individuo', 'pick_drop_item': 'Recoger/ Entregar Objeto', 'personal_med': 'Personal/ Médico', 'access_recreation': 'Acceder a Recreación', 'exercise': 'Recreación/ Ejercicio', 'entertainment': 'Entretenimiento/ Social', 'religious': 'Religioso', 'other': 'Otros'}}})] ```

Notebook: `Generic Timeseries` with STUDY_CONFIG=`stage-program` | Default Mapping

Execution steps: ``` (emission) root@52bab48d1dd6:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_timeseries.ipynb default /usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="? if r.status_code is not 200: About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/stage-program.nrel-op.json Successfully downloaded config with version 1 for Staging environment for testing programs only and data collection URL https://openpath-stage.nrel.gov/api/ label_options is unavailable for the dynamic_config in stage-program Running at 2023-11-13T21:47:40.574469+00:00 with args Namespace(plot_notebook='generic_timeseries.ipynb', program='default', date=None) for range (, ) Running at 2023-11-13T21:47:40.607876+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})] Running at 2023-11-13T21:48:05.833069+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=9), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})] Running at 2023-11-13T21:48:11.220470+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=10), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})] Running at 2023-11-13T21:48:16.596462+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=11), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})] Running at 2023-11-13T21:48:21.795305+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=12), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})] Running at 2023-11-13T21:48:28.244178+00:00 with params [Parameter('year', int, value=2021), Parameter('month', int, value=1), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})] Running at 2023-11-13T21:48:34.418982+00:00 with params [Parameter('year', int, value=2021), Parameter('month', int, value=2), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})] ```

Results comparison between Default Mapping and Dynamic Config in Generic Timeseries:

Timeseries of Emissions | Default Mapping | Dynamic Config | |--------|--------| |  |  | Timeseries of Emissions per mile/kilometer | Default Mapping | Dynamic Config | |--------|--------| |  |  | Timeseries of active users | Default Mapping | Dynamic Config | |--------|--------| |  |  | Timeseries of all mode shares | Default Mapping | Dynamic Config | |--------|--------| |  |  |

[shankari]

@iantei thank you for putting the test results into the PR as "Details". Can you please put them side by side in a table (you can still put them in an expandable section)? It is easiest to compare values when they are side by side so that our brains pick up on visual cues properly. Please also include testing for the generic metrics in this final round to ensure that there are no regressions.

[shankari]

@iantei looking through my prior comments to make sure that they are all resolved, I ran into this comment.

Similarly, notebook execution for sensed notebook failed since we have some NaN values in the dataset used - as justified above.

As justified where? The sensed notebooks should not be config specific since we don't use user labels in them.

[shankari]

I am provisionally marking this as approved. Congratulations, it only took 2 months 😄

Before I merge, I want to see:

• I still want to see all the outputs (e.g. generic metrics) one last time to make sure that there are no regressions
• I want to see an explanation of the sensed notebooks in the Lao case

[iantei]

Comparison of generic metrics notebook outputs:

Executed the notebook "Generic Metrics" for both the STUDY_CONFIG for comparison between default mapping and dynamic config. Steps executed: A. For Default Mapping (STUDY_CONFIG=stage-program) ```shell (emission) root@d7768840b5c9:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_metrics.ipynb default /usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="? if r.status_code is not 200: About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/stage-program.nrel-op.json Successfully downloaded config with version 1 for Staging environment for testing programs only and data collection URL https://openpath-stage.nrel.gov/api/ label_options is unavailable for the dynamic_config in stage-program Running at 2023-11-20T19:11:35.554525+00:00 with args Namespace(plot_notebook='generic_metrics.ipynb', program='default', date=None) for range (, ) Running at 2023-11-20T19:11:35.592872+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})] Running at 2023-11-20T19:11:51.849538+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=9), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})] Running at 2023-11-20T19:11:58.706112+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=10), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})] Running at 2023-11-20T19:12:05.421774+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=11), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})] Running at 2023-11-20T19:12:12.831446+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=12), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})] ``` B. For Dynamic Config (STUDY_CONFIG=dev-emulator-program): ```shell (emission) root@4f0d3eb39c2e:/usr/src/app/saved-notebooks# ls Dockerfile auxiliary_files docker generic_metrics_sensed.ipynb mode_specific_metrics.ipynb run_from_host variation_across_individuals.ipynb Program_trip_characteristics.ipynb bin energy_calculations.ipynb generic_timeseries.ipynb mode_specific_timeseries.ipynb scaffolding.py __pycache__ conf generic_metrics.ipynb mapping_dictionaries.ipynb plots.py tests (emission) root@4f0d3eb39c2e:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/update_mappings.py mapping_dictionaries.ipynb (emission) root@4f0d3eb39c2e:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_metrics.ipynb default /usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="? if r.status_code is not 200: About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/dev-emulator-program.nrel-op.json Successfully downloaded config with version 1 for Development environment (program) and data collection URL default Dynamic labels download was successful for nrel-openpath-deploy-configs: dev-emulator-program Running at 2023-11-20T17:05:28.739809+00:00 with args Namespace(plot_notebook='generic_metrics.ipynb', program='default', date=None) for range (, ) Running at 2023-11-20T17:05:28.779264+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=False), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'bikeshare', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'scootershare', 'baseMode': 'E_SCOOTER', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.00894}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'moped', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.05555}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'REPLACED_MODE': [{'value': 'no_travel'}, {'value': 'walk'}, {'value': 'bike'}, {'value': 'bikeshare'}, {'value': 'scootershare'}, {'value': 'drove_alone'}, {'value': 'shared_ride'}, {'value': 'e_car_drove_alone'}, {'value': 'e_car_shared_ride'}, {'value': 'taxi'}, {'value': 'bus'}, {'value': 'train'}, {'value': 'free_shuttle'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-bike': 'E-bike', 'bike': 'Regular Bike', 'bikeshare': 'Bikeshare', 'scootershare': 'Scooter share', 'drove_alone': 'Gas Car Drove Alone', 'shared_ride': 'Gas Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'moped': 'Moped', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Air', 'not_a_trip': 'Not a trip', 'no_travel': 'No travel', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'es': {'walk': 'Caminando', 'e-bike': 'e-bicicleta', 'bike': 'Bicicleta', 'bikeshare': 'Bicicleta compartida', 'scootershare': 'Motoneta compartida', 'drove_alone': 'Coche de Gas, Condujo solo', 'shared_ride': 'Coche de Gas, Condujo con otros', 'e_car_drove_alone': 'e-coche, Condujo solo', 'e_car_shared_ride': 'e-coche, Condujo con ontras', 'moped': 'Ciclomotor', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Autobús', 'train': 'Tren', 'free_shuttle': 'Colectivo gratuito', 'air': 'Avión', 'not_a_trip': 'No es un viaje', 'no_travel': 'No viajar', 'home': 'Inicio', 'work': 'Trabajo', 'at_work': 'En el trabajo', 'school': 'Escuela', 'transit_transfer': 'Transbordo', 'shopping': 'Compras', 'meal': 'Comida', 'pick_drop_person': 'Recoger/ Entregar Individuo', 'pick_drop_item': 'Recoger/ Entregar Objeto', 'personal_med': 'Personal/ Médico', 'access_recreation': 'Acceder a Recreación', 'exercise': 'Recreación/ Ejercicio', 'entertainment': 'Entretenimiento/ Social', 'religious': 'Religioso', 'other': 'Otros'}}})] ``` Results: Overall Comparison: | Default Mapping | Dynamic Config | |--------|--------| |  |  | Individual Comparisons: Number of Trips | Default Mapping | Dynamic Config | |--------|--------| |  |  | Number of commute Trips | Default Mapping | Dynamic Config | |--------|--------| |  |  | Trip count by purpose | Default Mapping | Dynamic Config | |--------|--------| |  |  | Trip count under 10 miles | Default Mapping | Dynamic Config | |--------|--------| |  |  | Trip miles by mode | Default Mapping | Dynamic Config | |--------|--------| |  |  | Average Trip Length | Default Mapping | Dynamic Config | |--------|--------| |  |  | Trip Frequency | Default Mapping | Dynamic Config | |--------|--------| |  |  | Trip Frequency (Weekday) | Default Mapping | Dynamic Config | |--------|--------| |  |  | Note: There is no mapping for "pilot_ebike" to "E-bike". Therefore, these mapping has been transformed into "Others" in "Number of Commute Trips" and "Trip count by Purpose" charts above.

Overall the charts looks identical.

UPDATE: There is some disparity in the charts depicted above, which has been explained below: https://github.com/e-mission/em-public-dashboard/pull/91#issuecomment-1826437021