zeno-ml / zeno-build

Build, evaluate, understand, and fix LLM-based apps
MIT License
484 stars 33 forks source link

Add [google-cloud, gpt-3.5-turbo-0301, gpt-4-0314] to analysis_gpt_mt #139

Closed zwhe99 closed 1 year ago

zwhe99 commented 1 year ago

Description

This PR adds system outputs of [google-cloud, gpt-3.5-turbo-0301, gpt-4-0314] to MT analysis example.

References

zwhe99 commented 1 year ago

@neubig https://github.com/zwhe99/gpt-MT also has outputs of DeepL. However, DeepL does not support en<->ha and en<->is. I'm not sure how to handle a situation where a system only supports partial language pairs

neubig commented 1 year ago

Thanks a bunch @zwhe99! I'll take a look and play around with this now.

To answer your question, it's already possible to create slices for each language pair to investigate accuracy be language pair separately, and we could create a slice "DeepL Supported Language Pairs" for the average over all language pairs that DeepL supports as well.

zwhe99 commented 1 year ago

Thank you! I hope this helps the community!