search

fani-lab / RePair

Extensible and Configurable Toolkit for Query Refinement Gold Standard Generation Using Transformers
5 stars 5 forks source link

Overall Tasks on Repair #19

Open yogeswarl opened 1 year ago

yogeswarl commented 1 year ago

No context

  1. Transformer Fine-Tuning
    • [x] #24
    • [ ] Bert (or other one)
  2. Choice of pairing
    • [ ] query.docs
    • [ ] query.doc
    • [x] docs.query
    • [ ] doc.query
  3. Query Set
    • [x] msmarco.passage
    • [ ] msmarco.document
    • [x] Aol-title
    • [x] Aol-tile-url
    • [ ] Aol-text
    • [ ] Yahoo Q & A

Information Retrieval

  1. Sparse Retrieval
    • [x] BM25
    • [ ] qld
  2. Dense Retrieval
    • [x] #20
    • [ ] Hybrid Retrieval

Evaluation

Stats

Supervised Query Refinement

Context

Paper Writeup

hosseinfani commented 1 year ago

@yogeswarl any update?

yogeswarl commented 1 year ago

Sorry @hosseinfani, Forgot to update this issue page. I have completed the MRR and boxes computation of title ,title-url . I am keeping them in a separate folder from the current publicly available data. I have finished computing TCT Colbert for MSMarco originals. I still have to do the 25 predicted queries. The only problem is the tct index is 31GB with encoders of 441MB. The search is relatively slow when compared to BM25. But I am hoping to give you the results of MSMARCO's dense by this weekend. If this works. I will start with Hybrid retrieval as well. Once these are done, I will move on to AOL context version and run the whole results in the pipeline!

hosseinfani commented 1 year ago

@yogeswarl These are the remaining items. Please read and come up with timeline. I need you to drop by during office every week and report on your progress.

sparse experiment:

colbert experiment:

Benchmarks

On paper:

yogeswarl commented 8 months ago

Hello @DelaramRajaei and @ZahraTaherikhonakdar, This is the main tasks list of completed works on Repair. Can you please add the query sets and variants to this task to keep track of all the works done on repair.

Thanks