search

ohcnetwork / ayushma

Empowering Nurses with Multilingual ICU Protocols. Leveraging the rapid advancements in AI technology, created multilingual interfaces that assist nurses in rapidly upgrading their knowledge about ICU protocols.
https://ayushma-api.ohc.network
MIT License
8 stars 9 forks source link

Add more metrics for evaluating test runs #230

Open shivankacker opened 1 year ago

shivankacker commented 1 year ago

The following metrics for comparing responses would be best: 1) GLOVE + Cosine Similarity 2) BertScore: compute token similarity using contextual embeddings 3) Glove + Word2vec + BiLSTM : word embedding is first made with Glove and Word2Vec, two BLSTM networks are used separately for sentence embedding, these are passed through a classifier 4) Word mover Distance 5) Pretrained sentence encoders: Such as Google Sentence encoder 6) Siamese Manhattan LSTM

cc. @bodhish

bodhish commented 1 year ago

We will just implement 1/2 from the list; lets do some research on this before going ahead;