Update metrics to the ones used in the thesis

[shankari]

While fixing https://github.com/MobilityNet/mobilitynet.github.io/issues/11 I inadvertently reset from head, which reverted all my local changes.

I believe I had implemented better versions of multiple metrics in different notebooks. Need to see if I can recover them.

[shankari]

Looking at the thesis logs, I finished the evaluation at around this time

commit 079cceae70d868aae4070f1aba7eab811f8a0b0a
Author: K. Shankari <shankari@eecs.berkeley.edu>
Date:   Wed Oct 2 14:08:09 2019 -0700

    Peformance evaluation is complete!

    let's push and start polishing

commit e153794ee0c2606e8282d4fc5097f1a9683f4cde
Author: K. Shankari <shankari@eecs.berkeley.edu>
Date:   Tue Aug 20 16:19:12 2019 -0700

    Final version before putting in the results

[shankari]

The checkpoint files for ipython notebooks might be a good option. Although they are apparently only updated on manual save. There are several files with timestamps of "Sep XX" there, so maybe I can recover things after all?

[shankari]

I also checked my backups, but because I had issues with Time Machine, I had no backups between March 2019 and May 2020. So it is checkpoints or bust.

[shankari]

Checked in the checkpointed files; now we just have to go through them one at a time and see which version to keep

[shankari]

There were also some files which were not checked in, and so were not modified in the reset. Given that they are not modified, and I ran the same metrics calculation for the baseline and the analysed master, these can provide an additional clue to the final code that I used.

-rw-r--r--  1 shankari  649427843   5.4M Sep 25  2019 Evaluate_power_vs_classification_analysis_master.ipynb
-rw-r--r--  1 shankari  649427843   5.8M Sep 25  2019 Evaluate_power_vs_motion_activity_analysed_master.ipynb
-rw-r--r--  1 shankari  649427843   4.6M Sep 24  2019 Evaluate_power_vs_trip_start_end_analysed_master.ipynb
-rw-r--r--  1 shankari  649427843    25M Aug 23  2019 exploration_trajectory_timestamp_many_alternatives.ipynb
-rw-r--r--  1 shankari  649427843    34M Aug 26  2019 exploration_trajectory_timestamp_predict_time_not_distance.ipynb
-rw-r--r--  1 shankari  649427843   5.6M Sep 18  2019 trajectory_evaluation_analysis_master.ipynb
-rw-r--r--  1 shankari  649427843    24M Sep 18  2019 trajectory_evaluation_spatio_temporal_analysis_master.ipynb

[shankari]

For the analysis_master files (which were not reset), the timestamps for the raw notebook and the checkpoint match consistently. This is a pretty positive indication that we were in fact checkpointing regularly. And since the checkpoints are from late Sept and we checked in the figures on Oct 2, at worst, we have lost the changes between Sep 25 and Oct 2.

So the planned resolution (@singhish) is:

• compare the current version and the checkpointed version (new_versions directory in new_spec_edits)
• pick the checkpointed version every time
• run the notebook and compare the results to the figures in the thesis

If they don't match, file an issue with the details for me to investigate.

-rw-r--r--   1 shankari  649427843   1973110 Sep 25  2019 Evaluate_power_vs_classification-checkpoint.ipynb
-rw-r--r--   1 shankari  649427843   5682909 Sep 25  2019 Evaluate_power_vs_classification_analysis_master-checkpoint.ipynb
-rw-r--r--   1 shankari  649427843   1441980 Sep 25  2019 Evaluate_power_vs_motion_activity-checkpoint.ipynb
-rw-r--r--   1 shankari  649427843   6046166 Sep 25  2019 Evaluate_power_vs_motion_activity_analysed_master-checkpoint.ipynb
-rw-r--r--   1 shankari  649427843  22075597 Sep 24  2019 Evaluate_power_vs_trip_start_end-checkpoint.ipynb
-rw-r--r--   1 shankari  649427843   4782774 Sep 24  2019 Evaluate_power_vs_trip_start_end_analysed_master-checkpoint.ipynb

-rw-r--r--   1 shankari  649427843    758540 Sep 24  2019 trajectory_evaluation-checkpoint.ipynb
-rw-r--r--   1 shankari  649427843   5881785 Sep 18  2019 trajectory_evaluation_analysis_master-checkpoint.ipynb
-rw-r--r--   1 shankari  649427843  28320089 Sep 18  2019 trajectory_evaluation_spatio_temporal-checkpoint.ipynb
-rw-r--r--   1 shankari  649427843  25572796 Sep 18  2019 trajectory_evaluation_spatio_temporal_analysis_master-checkpoint.ipynb

[shankari]

As a concrete example of Evaluate_power_vs_trip_start_end.ipynb

• git log shows that it was last checked in on Mon Sep 16 18:18:56 2019
• the checkpoint is from Sep 24 2019
• the analysed_master is from Sep 24 2019

So we pick the checkpoint implementation all through