Add BMWI Scraper - Githubissues

deepset-ai / COVID-QA

API & Webapp to answer questions about COVID-19. Using NLP (Question Answering) and trusted data sources.

Apache License 2.0

344 stars 121 forks source link

Add BMWI Scraper #78

Closed DataWorm closed 4 years ago

bogdankostic commented 4 years ago

Thank you for contributing! The extraction of questions looks good. However, there are some issues with the extraction of answers:

Some answers just contain an empty string (e.g. the answer to the question 'Welche arbeitsrechtlichen Fragen können sich im Zusammenhang mit dem Coronavirus für Arbeitnehmer und Arbeitgeber ergeben?')
I would strip the answer strings to remove whitespace at the beginning and the end of the answers

DataWorm commented 4 years ago

The answer string already got stripped, the stripped version just wasn't stored! :-D The answers are structured in at least three different formats but I think with this latest version of the scraper it works quite well.

bogdankostic commented 4 years ago

Looking good now! :)