Break the Bio file into manageable chunks for both schools - Githubissues

hydrosquall / yale_athletics_data

data about student-athletes at harvard and yale from 1940 - 2016

MIT License

4 stars 2 forks source link

Break the Bio file into manageable chunks for both schools #13

Closed hydrosquall closed 7 years ago

hydrosquall commented 7 years ago

Takes the {schoolname}_bios.csv files and break them into the following structure

Page URL
Headers (The table at the top of the bio page that lists major, hometown, etc)
Synopsis (The fulltext below, which currently eludes being meaningfully chunked)

For example: This Bio has the following

url: /sports/m-footbl/2015-16/bios/stallings_jackson_wksl
headers: major, residential college, etc
synopsis: all the text in the white rectangle area