Open gahjelle opened 6 years ago
The current get_data.py is both a crawler and tries to wrangle the data into meaningful CSV.
get_data.py
We should split it in two parts. One for just simple crawling (downloading data), and another script that parses the data and creates the structure/meaning.
The current
get_data.pyis both a crawler and tries to wrangle the data into meaningful CSV.We should split it in two parts. One for just simple crawling (downloading data), and another script that parses the data and creates the structure/meaning.