Data Processing - Swap Backend to Beautiful Soup

liamreilly01 / Sweng-Group-20

1 stars 0 forks source link

Data Processing - Swap Backend to Beautiful Soup #5

Closed MylanaBulat closed 1 year ago

MylanaBulat commented 1 year ago

use beautiful soup to extract data from the website,

you make a new Github branch if you want or continue using the same one.
put updates in the chat as you make progress

MylanaBulat commented 1 year ago

"The soup getter has been swapped back to xml as beautiful soup allows for replacing tags with text, making the fades easy to deal with. Currently can print out the titles of acts in a given year"

mccabed7 commented 1 year ago

Some xml files have the description in uncommon places. Currently writing a script to search for the descriptions and return them in method get_description().

mccabed7 commented 1 year ago

Completed as at 16:52 17/02/23. Removed method get_description() and now code is called directly from fetch_acts(). Script currently scrapes the names + descriptions of the acts, fixes Irish fada errors, and prints them to the console.