Weekly digest - 4th Week'Feb 21

primeqa / primeqa

The prime repository for state-of-the-art Multilingual Question Answering research and development.

https://primeqa.github.io/primeqa

Apache License 2.0

727 stars 57 forks source link

Closed jdpsen closed 2 years ago

jdpsen commented 2 years ago

jaydesen created the following on Feb 28:

jdpsen commented 2 years ago

On Feb 28, jaydesen commented: Table+text QA (Vishwajeet)

jdpsen commented 2 years ago

On Feb 28, jaydesen commented: QG (saneem)

using the parallel data from HybridQA-OTTQA for training "decontextualizer" module to produce decontextualized questions after T5 based QG.
experimentation in progress
discussed possible steps for Table+text based QG

jdpsen commented 2 years ago

On Feb 28, jaydesen commented:

Multi-lingual Retriver

[Martin, Yulong] best numbers obtained on XOR-Retrieve - gone to NAACL paper
brainstorming on what we should try next options: (1) Benchmarking the number on Task3 of XOR-tydi i.e. full multilingual QA with retriever + reader (2) should we submit our system in this NAACL workshop ? https://mia-workshop.github.io/shared_task.html
[Bhavani] best numbers in Mr Tydi
- ColBERT xlmr-nq model +5 on MRR and +6 on RECALL compared to the best baseline system.
- Fine-tuning on MrTyDI additional +11 for MRR and +0.4 for Recall

jdpsen commented 2 years ago

On Feb 28, jaydesen commented: One-QA framework (Anthony)

stub packages created for disjoint set of functionalities/models to be inducted to One-QA framework for V1 (ACL deadline)
MRC pipeline implementation in progress

jdpsen commented 2 years ago

On Feb 28, jaydesen commented: Generation Based QA

[Mihela, Arafat] experiments in progress for using retriever+reranker for better answer generation. Retriever+generator does not yield good score.
[Sarah] "claim annotation" task in progress to explore a new metric for evaluating answer generation based on " # of validated claim"
[Scott] discussed about "Answer Diversity" metric inspired by Strategy QA

jdpsen commented 2 years ago

On Feb 28, jaydesen commented:

ListQA

[IBM internal annotators] Pilot annotation task (of 20 questions) assigned to internal annotators to ensure high quality test sets for list question answering using GooAQ, CCQA, and NQ
[Sarah] Data cleanup for GooAQ, CCQA, NQ to produce an accurate set of questions/models for listQA