fix: do not combine speeches for questions - Githubissues

parleh-mate / singapore-parliament-speeches

Making parliament speeches from Singapore's Parliament Hansard structured and accessible.

Creative Commons Zero v1.0 Universal

2 stars 0 forks source link

fix: do not combine speeches for questions #11

Closed jeremychia closed 5 months ago

jeremychia commented 5 months ago

context:

in the parsing logic, if successive rows have the same member speaking, the rows are combined.
example: on 2024-05-07, patrick tay asked two questions consecutively. in the parsing logic, the two rows would be combined.
as a result, the accurate count of primary questions is not possible.

this PR:

ensures that if the rows are like a question (starts with "asked" or "to ask"), they will not be combined.
tries to relocate back some missing names - for example, in the format "The question stood in the name of", the name would be missing. tries to get back the name.
cleans \t out of the text.
cleans out html tags from text content (previously having tags in there. only content is kept).
adds date to the raw table of speeches