search

dice-group / dice-embeddings

Hardware-agnostic Framework for Large-scale Knowledge Graph Embeddings
MIT License
44 stars 12 forks source link

Create examples/eval_multip_hop_query_answering.py to ease the evaluation #213

Closed Demirrr closed 8 months ago

Demirrr commented 8 months ago

Feature: Ease evaluation of multi-hop queries

Creating a script to ease the evaluation of pre-trained models in multi-hop query ansering tasks

Example of Usage

  1. Train a model: dicee --dataset_dir "KGs/UMLS" --model Keci --path_to_store_single_run Keci_UMLS
  2. Eval a model : python eval_multip_hop_query_answering.py --dataset_dir "KGs/UMLS" --pretrained_dir "Keci_UMLS"

eval_multip_hop_query_answering.py should implement argparse to parse dataset_dir and pretrained_dir and other inputs if there are more.

Optional: Multi-hop Query Answering on a given query and answer pairs

To allow a user to use their own queries and answers, we can also introduce --query_and_answers python eval_multip_hop_query_answering.py --query_and_answers "query_and_answers.json

Demirrr commented 8 months ago

done