search

autogluon / tabrepo

Apache License 2.0
27 stars 7 forks source link

Rename `zeroshot_pred_proba` and `zeroshot_gt` #3

Closed Innixma closed 6 months ago

Innixma commented 1 year ago

At some point we should revisit our naming for zeroshot_pred_proba and zeroshot_gt.

Probably deserves its own PR dedicated to only the renaming. I think we should address a few things:

  1. [Resolved, will keep existing name] This is not specific to Tabular. Therefore we should not have classes named such as TabularModelPredictions. They could instead be ModelPredictions.
  2. Currently we use things like zpp. We can consider changing to pp or predictions.
  3. zeroshot_pred_proba -> pred_proba or model_pp or model_predictions or bench_pp, etc.
  4. zeroshot_gt -> gt, ground_truth, bench_gt, task_gt, etc.
geoalgo commented 1 year ago

Entirely agree about the renaming and having a PR solely dedicated to this is the best approach.

Regarding your points,

  1. Why is it not? Tabular is defined as having results a list of configurations in the AutoML community (for instance https://ml.informatik.uni-freiburg.de/wp-content/uploads/papers/20-NIPS_WML-NB301.pdf). This seems to be exactly what we have right? The alternate naming is "surrogate" which is used when instead a random set of configuration is evaluated and a surrogate model such as XGBoost is used to serve prediction at any possible configuration.
  2. +1, in particular predictions is much clearer.
  3. +1, model_predictions is better I believe since we do have probabilities only for classification, for regression we just have a single prediction number.
  4. +1, ground_truth is the clearest I believe (we could use target which is used the most in scikit-learn and seems to also be used in AG)
Innixma commented 1 year ago

@geoalgo re 1. I see what you mean. I thought that the naming was meaning that the predictions came from tabular models, but I see your point and thus we probably don't need to change the name.