dotnet / machinelearning-modelbuilder

Simple UI tool to build custom machine learning models.
Creative Commons Attribution 4.0 International
264 stars 56 forks source link

Text classification Predict result is not calibrated. #2430

Open zewditu opened 1 year ago

zewditu commented 1 year ago

System Information (please complete the following information):

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior A clear and concise description of what you expected to happen.

Screenshots, Code, Sample Projects image

Additional context Add any other context about the problem here.

michaelgsharp commented 1 year ago

@zewditu this seems like an AutoML issue. We probably don't want the calibration in by default do we @luisquintanilla in the framework? That way the can calibrate it if they want it? Or do they always want it calibrated?

luisquintanilla commented 1 year ago

@zewditu this seems like an AutoML issue. We probably don't want the calibration in by default do we @luisquintanilla in the framework? That way the can calibrate it if they want it? Or do they always want it calibrated?

Correct. I think this is something we might want to solve in tooling by appending a calibrator to the pipeline like other algorithms that don't do it by default unless there's a good reason for us to do it in the framework.

@zewditu here is a sample which appends a callibrator to a pipeline: https://gist.github.com/luisquintanilla/77664adc3f0e920648954244893a6bf8

luisquintanilla commented 1 year ago

Moving this issue to the Model Builder repo.