dianna-ai / dianna

Deep Insight And Neural Network Analysis
https://dianna.readthedocs.io
Apache License 2.0
44 stars 13 forks source link

Run text model inputs one-by-one through model to avoid shape mismatch errors #773

Closed loostrum closed 1 month ago

loostrum commented 1 month ago

There was a difference between the movie review model runner used in the RISE text tutorial, vs LIME text, the tests and the dashboard. The LIME/dashboard/tests version first tokenizes all inputs, then runs the model once. The RISE version runs sentences through the model one-by-one. The latter is slower, but avoids the issues we've been seeing with special chars. The faster version could work if one makes sure the inputs all have the same shape, but this can be complicated so I suggest we go with the slower, but correct, version.

Fixes #531 Fixes #771 Fixes #751

review-notebook-app[bot] commented 1 month ago

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB