Simplify the process of adding a new metric

[maxspahn]

Currently, if a new metric needs to be added, the name of the metric must be added at various places in the code. This process can be made simpler.

https://github.com/maxspahn/localPlannerBench/blob/d2e5f47acd8ddcb2bdca362f04c3258991510d31/plannerbenchmark/exec/postProcessor#L30

https://github.com/maxspahn/localPlannerBench/blob/d2e5f47acd8ddcb2bdca362f04c3258991510d31/plannerbenchmark/exec/postProcessor#L180

[niketagrawal]

Developer notes on the current implementation:

• To create a new metric, inherit from the metric class and implement the computemetric() method in metrics.py

• In order to use the metric for benchmarking purposes for a specific type of robot, it must be instantiated for that type of robot in the postprocessor script.

• self._metrics['time2Goal'] = TimeToReachGoalMetric(
          "time2Goal",
          ["q0", "q1", "goal_0_0", "goal_1_0", "t"],
          {"m": 2, "des_distance": self._experiment.primeGoal().epsilon()},
      )

  • The fist argument is a string to represent the type of metric. It doesn't need to be passed as an argument here, it can be an instance attribute of the metric class.
  • The 2nd argument is a list of column indices in res.csv that needs to be selected for postprocessing. The columns chosen for each type of robot and metric combination will differ which requires a unique instantiation for each metric and robot combination, but this issue can be solved by renaming the columns in the res.csv to represent the parameters type and not the specific variable names.
  • The 3rd argument comes from experiemnt.py; this can be standardized too. in the above manner.

• Requirements for creating a new metric

  • Knowledge of the computation logic for the metric; this is needed to implement the computemetric method()
  • Knowledge of how the new metric will be instantiated for a particular type of robot. This is needed to specify which column indices from res.sv are required for metric computation.

• The arguments of the class instantiation of the metrics in the postprocessor script can be standardized so that they can be embedded in the metric class itself.

• self.experiment.robot_type() can be used to construct 2nd and 3rd argument of the class instantiation

• A new metric may not apply to all the robot types; it would depend on the what is being computed in the metric.

[niketagrawal]

Expected functionality (tentative): A metric class should be instantiated for a particular type of robot based on the arguments passed in the postprocessor command. Currently, all metric class are being instantiated for all types of robots in advance.

[maxspahn]

This could be done with the registry pattern. Similar to #25 for planners.