search

pragyalms / experiments

0 stars 0 forks source link

Automatic Question Generation #8

Open abhinavdayal opened 3 years ago

abhinavdayal commented 3 years ago

Given any text or transcript, create objective questions automatically for quick assessment of whether a person has gone through and and remembers the basic understandings.

While we can start with text, automatic questions can also be from images, hotspots, video, audio etc. in various formats and resources.

Some resources:

  1. Quillionz
  2. https://paperswithcode.com/task/question-generation
  3. https://devopedia.org/question-generation
  4. https://www.eurekaselect.com/176644/article
  5. https://towardsdatascience.com/questgen-an-open-source-nlp-library-for-question-generation-algorithms-1e18067fcdc6 (Video https://www.youtube.com/watch?v=hoCi_bJHyb8)
  6. https://aclanthology.org/2020.lrec-1.217/
  7. https://patents.google.com/patent/US20160117314
JayasriAngara commented 3 years ago

Basic understanding:

  1. Question Generation involves identification of sentences, keywords and distractors
  2. Different types of objective questions: ---MCQs based on wh-type questions ---Fill in the blank questions where blanks are given along with possible alternatives (closed questions) ---Fill in the blanks without alternatives

Questions to be discussed.

  1. How identification of distractors is possible just by sticking to the content available in the given text?
  2. Do we include questions with single clause answers as objective questions?
  3. I am assuming Medium scope (where answer involves more than 1 clause) and General scope (scope of the answer is entire article) questions are not part of the current scope of study. Please clarify.

Observations on Quillionz tool

  1. There is word limit of 300 to 3000 to input the content. However, in real time to prepare questions on Annotations, the limit is less than this.
  2. The content readiness meter shows 60%. Is there any awareness given to the end user on how to increase this level, what aspects need to be looked into to get good score on content readiness?
  3. Questions are not very meaningful.
  4. Two blanks are given where only one answer needs to be typed. It is given based on number of words in the answer. It may cause confusion to students as they are habituated to see two blanks for two different answers. a. Ex: Source code. It contains two separate words, however both the words belong to single answer
  5. Quality of Distractors given for the MCQs need to be improved.
JayasriAngara commented 3 years ago

https://docs.google.com/document/d/1y2NyCLfdPSYso8Z2t4rKD_y1UOta2g8NrcLwF8gvgjw/edit?usp=sharing link to the google doc where observations from Literature survey are noted down.