search

LMZimmer / Auto-PyTorch_refactor

Apache License 2.0
0 stars 2 forks source link

Reimplementation of autopytorchmetric #40

Closed ravinkohli closed 3 years ago

ravinkohli commented 3 years ago

Changes made:

  1. autoPyTorchMetric is now a wrapper on sklearn metrics
  2. Added _ProbMetric, _ThresholdMetric and _PredictMetric based on autosklearn
  3. Made other utility functions in autosklearn metrics compatible with get_metrics and get_supported_metrics
  4. Changed compute metric function in BaseTrainerComponent and added task_type in prepare which is needed by compute_metric
  5. Added unittests for the same

Reasons to switch from PyTorch lightning to sklearn:

  1. PyTorch lightning in their recent update removed class metrics used in the old implementation.
  2. sklearn functionals perform faster than pytorch lightning functionals even for tensors (tested locally on my laptop)
  3. Elegant implementation in autosklearn that handles various prediction and target types

Also, Resolves #21

LMZimmer commented 3 years ago

We may have to make some small changes with the dataset PR but for now this can be merged as is 👍