Add the QA model for Covid19

[nudles]

https://www.kaggle.com/matteomuffo/a-fine-grained-covid-19-question-answering-engine

[delphieritas]

Closes #5

[nudles]

Update the QA model to follow the general QA task format #22 .

[delphieritas]

The classification for the trending QA datasets:

Examples for each kind of QA datasets:

extractive (EX), abstractive (AB), multiple-choice (MC), and yes/no (YN). ‘idk’ denotes ‘I don’t know’ or unanswerable questions.

[nudles]

can we use a uniform (input and output) data format for all these tasks?

[delphieritas]

can we use a uniform (input and output) data format for all these tasks?

Can. There is an existing paper coping with this.

And here is its demo: https://unifiedqa.apps.allenai.org/

Will see and try how we implement this.

[nudles]

Then pls post the format for the input and output here.

On Wed, Jun 10, 2020 at 11:06 AM delphieritas notifications@github.com wrote:

can we use a uniform (input and output) data format for all these tasks?

Can. There is an existing paper coping with this. Will see and try how we implement this.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/nusdbsystem/singa-easy/issues/4#issuecomment-641692066, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA47DR6WSAGHBGQN2IJOHETRV32DDANCNFSM4MZI5RPA .

[delphieritas]

Then pls post the format for the input and output here. … On Wed, Jun 10, 2020 at 11:06 AM delphieritas @.***> wrote: can we use a uniform (input and output) data format for all these tasks? Can. There is an existing paper coping with this. Will see and try how we implement this. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA47DR6WSAGHBGQN2IJOHETRV32DDANCNFSM4MZI5RPA .

The input and output of the existing paper is as following:

Input for extraction:
At what speed did the turbine operate? \n (Nikola_Tesla) On his 50th birthday in 1906, Tesla demonstrated his 200 horsepower (150 kilowatts) 16,000 rpm bladeless turbine. During 1910–1911 at the Waterside Power Station in New York, several of his bladeless turbine engines were tested at 100–5,000 hp.

Output: 100–5,000 hp

#Input for I DON'T KNOW extraction
What does being a Conservative do to the seriousness of a protest? \n (Civil_disobedience) Howard Zinn writes, "There may be many times when protesters choose to go to jail, as a way of continuing their protest, as a way of reminding their countrymen of injustice. But that is different than the notion that they must go to jail as part of a rule connected with civil disobedience. The key point is that the spirit of protest should be maintained all the way, whether it is done by remaining in jail, or by evading it. To accept jail penitently as an accession to 'the rules' is to switch suddenly to a spirit of subservience, to demean the seriousness of the protest...In particular, the neo-conservative insistence on a guilty plea should be eliminated."

[No Answer]

#Input for abstraction:
What does Paul McCartney think about his music? \n LAS VEGAS, Nevada (CNN) -- Former Beatles Paul McCartney and Ringo Starr clowned around and marveled at their band's amazing impact in an interview Tuesday on CNN's "Larry King Live."  Larry King, left, poses with (l-r) Paul McCartney, Yoko Ono Lennon, Olivia Harrison and Ringo Starr Tuesday in Las Vegas.  "We were just kids from Liverpool," McCartney said. "And, yes, it is quite amazing, because as time goes on, it kind of becomes more and more of a phenomenon."  McCartney said the early Beatles knew they were a good band and were pretty sure of themselves, but Starr said, "We thought we'd be really big in Liverpool."  "I think the most exciting thing is that, you know, we expect people our age to know the music. But actually, a lot of kids know the music," Starr said. "And if anything is left, we have left really good music, and that's the important part, not the moptops or whatever."  The pair appeared relaxed in sneakers and almost matching black suits and joked frequently -- often at each other's expense.  "They were nothing," Starr said of his former bandmates. "And then I joined and then they got this record deal and look what happened."  "No, we were good," McCartney retorted. "You wanted to join us. You begged to join us."  "I didn't beg," Starr said. Watch Paul and Ringo talk about the Beatles' magic »  McCartney and Starr were in Las Vegas with Yoko Ono Lennon and Olivia Harrison, the widows of John Lennon and George Harrison, to celebrate the anniversary of Cirque du Soleil's "Love," which uses the Beatles' music.  Harrison said her husband was friends with Cirque du Soleil founder Guy Laliberte and came up with the idea before he died of cancer in 2001.  "George was around just long enough to transmit that to all of us," Harrison said.  Ono said she wasn't sure at first what her husband would have thought about the project. John Lennon was shot near the couple's New York apartment in 1980.  "Now I really know that John would be very happy with this," she said.  Neither woman has remarried and they both said it was still sometimes difficult to deal with their losses.  "We feel so strongly about our husbands that sometimes it's hard for us, isn't it?" Ono said.  "It's hard, you know," Harrison said. "I mean their presence is very powerful and very strong. But the incredible thing about them is that they -- everything they left the world and left us is uplifting and joyful."  The Beatles' music received a bit of a facelift for the show and has been remixed in 5.1 surround sound. (An album, "Love," came out last year.)  "Paul and I went to listen to the music in 5.1 and we go 'Whoa, listen to that,' " Starr said. "You know you can hear everything now. Things that we buried a lot. It's all very clear, so it's really great to hear it."  "Most historic stuff goes down with age, you know?" McCartney added. "Winston Churchill's old papers go brown and crinkly, while our music gets brighter and shinier."  "Next year, it will be 10.1," Starr joked.  McCartney's latest solo album, "Memory's Almost Full," is No. 3 on the album charts and Starr is scheduled to release a greatest hits album in August. E-mail to a friend

very good

#Input for multiple-choice question:
The author tells us that to succeed in a project you are in charge of, you should   _  . \n  (A) make everyone work for you (B) get everyone willing to help you (C) let people know you have the final say (D) keep sending out orders to them \n If you're in charge of  a project, the key to success is getting everyone to want to help you. As a director, I point, I suggest, I gently push the actors in the direction  I want them to go. In the 1986 movie Nothing in Common, Jackie Gleason's character, Max Basner, gets fired (or unemployed) from his job as a clothing salesman. The scene, shot   on a boat, shows Max's despair  about being out of work. I was looking for some gesture that would allow Max to show his feelings. Jackie had far more experience  at everything than I did, and at first I was frightened (very much afraid). What could I possibly tell "The Great One" about acting? Out of fear I decided to direct by suggestion, and I sat down with Gleason to talk about the scene. "So Max is sad, right?" I said. Gleason nodded. "And he's probably still carrying his pens with his name on them --- the ones he used to hand out to his customers, right?" Gleason nodded. " So what would you want to do with the pens after you were fired?" He was silent for a moment. "Why don't I throw them overboard (or all over the boat)?" I stood up and turned toward the crew. " Hey, everybody, Jackie has a wonderful idea. Let's shoot it." After filming the scene, Gleason called me over and said with a smile, "Garry, what kind of wonderful idea am I going to have tomorrow?" You and your team can discover the answers to problems together. When there are no prizes or gold stars for who gets the solution (or the way to work out a problem) first, you'll all benefit (or get something of use or value) when everything turns out right.

get everyone willing to help you

#Input for Yes/No question:
is the isle of man a part of great britain? \n (Isle of Man) In 1266, the island became part of Scotland under the Treaty of Perth, after being ruled by Norway. After a period of alternating rule by the kings of Scotland and England, the island came under the feudal lordship of the English Crown in 1399. The lordship revested into the British Crown in 1765, but the island never became part of the 18th-century Kingdom of Great Britain or its successors the United Kingdom of Great Britain and Ireland and the present-day United Kingdom: it retained its status as an internally self-governing Crown dependency.

no

[nudles]

We need the format defined like https://singa-auto.readthedocs.io/en/latest/src/user/tasks.html?highlight=TASK

[delphieritas]

QUESTION_ANSWERING

Query Format ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. note::

- The pretrained model should be fine-tuned with a dataset first to adapt to particular question domains when necessary. 
- Otherwise, following the question, input should contain relevant information (context paragraph or candidate answers, or both), whether or not addresses the question. 

Query is in JSON format. While the relevant information is provided in query, the question always comes first, followed by additional information. We use “\n” separators between different parts of the input.

.. code-block:: text

{
 'questions': ['At what speed did the turbine operate? \n (Nikola_Tesla) On his 50th birthday in 1906, .... several of his bladeless turbine engines were tested at 100–5,000 hp.',
              'What does Paul McCartney think about his music? \n LAS VEGAS, Nevada (CNN) -- Former Beatles Paul McCartney and Ringo Starr clowned around and marveled at their band's amazing impact in an interview Tuesday on CNN's "Larry King Live."   ... McCartney said the early Beatles knew they were a good band and were pretty sure of themselves, but Starr said, "We thought we'd be really big in Liverpool."  ',
              'The author tells us that to succeed in a project you are in charge of, you should   _  . \n  (A) make everyone work for you (B) get everyone willing to help you (C) let people know you have the final say (D) keep sending out orders to them \n If you're in charge of  a project, the key to success is getting everyone to want to help you. ...  You and your team can discover the answers to problems together. ',
              'is the isle of man a part of great britain? \n (Isle of Man) In 1266, the island became part of Scotland under the Treaty of Perth, after being ruled by Norway.'
                          ]

'target_answers':['16,000 rpm',
                  'very good',
                  'get everyone willing to help you',
                  'no'
                ]
}

Prediction Format ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The output is in JSON format.

     {'answers':['16,000 rpm',
                 'very good',
                 'get everyone willing to help you',
                 'no'
                 ]}

Hi @nudles . The above content has been revised.

[delphieritas]

The layout format is a little different from the target page. Once moving the comment to the new web page, it should be fine and will adjust accordingly. And we also need to add entry for https://singa-auto.readthedocs.io/en/latest/src/user/datasets.html#dataset-type-image-files

We need the format defined like https://singa-auto.readthedocs.io/en/latest/src/user/tasks.html?highlight=TASK

[delphieritas]

I have made a Pull Request for Question answering task documentation & update dev.py admin.py #80

Pls see the this detailed commit: https://github.com/nusdbsystem/singa-auto/pull/80/commits/c816eea74c9bd5cb2ef772b6cac845d01ab58de0