Kiro47 / MTU-Transfer-Course-Gatherer

I got tired of looking for classes to transfer in by hand, so here we are.
Mozilla Public License 2.0
4 stars 2 forks source link

Scrape MTU classes #63

Open Blu3spirits opened 4 years ago

Blu3spirits commented 4 years ago

I've got a PoC done that dumps out all the class info from the Magic:tm: url.

BASE_URL = [Removed because it's huge]
req = requests.get(BASE_URL)
data = req.text

soup = BeautifulSoup(data, 'html5lib')
body = soup.find_all('tr')
for row in body:
    cols = row.find_all('td')
    columns = [x.text.strip() for x in cols]
    col_length = len(columns)
    if col_length >= 15:
        print(columns)

Quick explanation, a given class will only ever have 15+ columns, all of the other items are meaningless whitespace.