search

evidentlyai / evidently

Evidently is ​​an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.
https://www.evidentlyai.com/evidently-oss
Apache License 2.0
5.07k stars 566 forks source link

Request for feature - Transformer models/ Clustering/ Pandas Data-frame descriptions #44

Open pratikchhapolika opened 3 years ago

pratikchhapolika commented 3 years ago

Do we have the flexibility to create dashboard for NLP related tasks from Hugging face libraries https://huggingface.co/.

Example1: Comparing two classification models using NLP models like BERT and Roberta. Compare metrics and show them in dashboards. Also show their loss in the dashboard.

Example2: Comparing two Clustering models using on text data. Compare metrics such as Silhouette score and show them in dashboards. Also plot the clustering result for 3Dand 2Dby using Tsne

emeli-dral commented 3 years ago

Thanks for opening the issue @pratikchhapolika!

Adding features for text-based data is in our mid-term roadmap. However, we want to expand the functionality with tabular data first.

Adding the possibility to add custom plots/metrics to existing reports or create custom reports from pre-built plots is in the near-term roadmap. But we will again focus on tabular data first.

danilyef commented 1 year ago

Do you have any updates regarding the topic? Maybe "how to" examples with Hugging Face?