search

eeml2022-nlp / iglu-nlp

Solving NLP task of NIPS 2022 IGLU challenge
MIT License
2 stars 0 forks source link

What to ask: rank a list of clarifying questions #2

Open ilyalasy opened 1 year ago

ilyalasy commented 1 year ago

Build a model for ranking clarifying questions given an instruction. See What to ask.

Questions to rank are in question_bank.csv, see dataset.

Models to try:

  1. BM25 Ranker (no training needed) - used in official baseline
  2. Semantic search (no training needed) - sentence-transformers
  3. Polyencoder ranker - parlai
  4. Dense Passage Retrieval - facebookresearch
  5. ColBERT - stanford-futuredata
Darth-Carrotpie commented 1 year ago

Note: Some of the clarifying questions seem to be in broken English or contain spelling errors. Probably should keep that in mind when testing.

ilyalasy commented 1 year ago

FYI: I preprocessed data and splitted it into train/dev, so we should use these csvs for our models. Notebook used for preprocessing is here.