clean data - Githubissues

KaylaCrush / maple-lobbying

1 stars 1 forks source link

clean data #8

Open KaylaCrush opened 1 year ago

KaylaCrush commented 1 year ago

Let's get this filthy lobbying data cleaned up a little! In no particular order:

[ ] check the data for other weirdness not included below
[ ] Replace all double spaces with a single space (maybe make it work for triple spaces and tabs and stuff too)
[ ] add proper data types

KaylaCrush commented 1 year ago

Now that I am able to pull large datasets I should start looking for other weirdness too

KaylaCrush commented 1 year ago

ALSO, data types! Right now pretty much everything is str (er I mean varchar) but I should convert the numeric stuff into ints or floats and so on. Lots of this to be done in the headers table: date ranges into dates, maybe make countries a category - I think that's a thing in SQL. Tied in with KaylaCrush/maple-lobbying#7