search

PAIR-code / what-if-tool

Source code/webpage/demos for the What-If Tool
https://pair-code.github.io/what-if-tool
Apache License 2.0
921 stars 170 forks source link

Can What-if-tool handle multimodal structural data which has continuous, categorical, text and image columns? #197

Open wjlgatech opened 2 years ago

wjlgatech commented 2 years ago

Can What-if-tool handle multimodal structural data which has continuous, categorical, text and image columns?

Could you point me to some examples on how to set up wit for such TF multimodal model?

Thanks!

jameswex commented 2 years ago

WIT should be able to handle all those input types in a single datapoint just fine. Your code might look like some combination of https://colab.sandbox.google.com/github/PAIR-code/what-if-tool/blob/master/WIT_Smile_Detector.ipynb (images), https://colab.sandbox.google.com/github/pair-code/what-if-tool/blob/master/WIT_Toxicity_Text_Model_Comparison.ipynb (text), and https://colab.sandbox.google.com/github/pair-code/what-if-tool/blob/master/WIT_COMPAS_with_SHAP.ipynb (continuous and categorical)

Another option would be to look into using LIT, which is more full-featured and has better multi-modal support, and is being actively developed (https://pair-code.github.io/lit/)

wjlgatech commented 2 years ago

@jameswex Amazing! Thanks for your specific, point to point feedback. I will check out those resource and update to the WIT community what it ends up.