Add T5 data (encoder-decoder model)

catherinesyeh / attention-viz

Visualizing query-key interactions in language + vision transformers

http://attentionviz.com/

MIT License

122 stars 15 forks source link

Add T5 data (encoder-decoder model) #39

Open catherinesyeh opened 1 year ago

catherinesyeh commented 1 year ago

Another possibility if we want to include another type of transformer in the visualization tool

Bachstelze commented 1 year ago

How can we add more models? We could analyze instructions models with flan-t5.

catherinesyeh commented 1 year ago

Hi, we have not tried flan-t5 yet, but if you can get the data in a similar format as the provided gpt or bert data, you can substitute that in when running the application. Streamlining this process is an area for future work, as currently, adding new models may require modifications to both the frontend and backend.