search

a-hanf / mlr3automl

Automated machine learning in mlr3
GNU Lesser General Public License v3.0
25 stars 4 forks source link

Integration with DALEX #29

Closed johnfrombluff closed 2 years ago

johnfrombluff commented 2 years ago

Dear Alex,

Thanks for your hard work on this amazing package! I watched your presentation on UserR session 3A and followed your examples on my own data. However I'm having trouble analysing the result using DALEX and other related packages (ArenaR, Triplot, ModelStudio, rSAFE). I wonder if you could provide example code in you vignette that shows how to:

  1. create a DALEX explainer
  2. display information about the final model and its parameters
  3. display information about the model selection and tuning history

Thanks again for your amazing work,

John

a-hanf commented 2 years ago

Hi John,

thanks for the question. I have never used DALEX before, but at first glance it looks like they have an interface for mlr3 in the DALEXtra package. It would be super valuable to add some XAI functionality to mlr3automl, so this is definitely worth looking into. Can't give you an ETA at the moment, but it's next on my TODOs :)

Thanks! Alex

a-hanf commented 2 years ago

Hi John, I finally had some time to look into DALEX, thanks again for the suggestion :) Regarding your questions:

  1. I implemented an explain() method for DALEX and iml. For a trained mlr3automl model, you can get the DALEX explainer via model$explain() (or model$explain("iml") if you want to use the iml package). Examples are in the vignette
  2. You can use model$tuned_params() for this
  3. You can use model$learner$archive to find the full tuning history

Let me know if this works for you.

Thanks! Alex

johnfrombluff commented 2 years ago

Thanks so much Alex, you're a star!