Turn data ingest into OOP

[jwestw]

Improve the organisation of the code

[jwestw]

(Editing this ticket to make the requirements clearer)

Consider whether the zip file needs to be written out before extracting. It might be that it can be unzipped in memory. Or maybe request a single file from the zip.

• Create 2 data importer classes, one that has a Pandas Dataframe as the underlying object the other that has a Geo Dataframe as the object.

Class attributes:

• these are the variables or state
• CRS is fixed, so will be in __init__

Methods:

• main data object would be a geopandas df, so all methods that read or write would do so from the gpd_df
• check file exists (for csv, feather, zip etc). If filepath is passed as a parameter this might be a static method, decorator @static, otherwise if path is stored in class, then it's a standard method)
• csv to pandas dataframe
• json to pandas df (possibly other formats as well)
• read shp file --> geopandas df
• pandas to geopandas df
• writes to csv (for persistence)
• writes to feather (for persistence)
• retrieve data from zip file

[jwestw]

I have referred to this issue in #120 .

It might be a good idea to solve this ticket before that.

Not only would the any_to_pd function be better as a class, but also it also needs to:

• have an method/function to handle Excel files
• the Excel handler needs to accept parameters for extracting particular sheets in a workbook
• possibly needs to be able to be able to extract particular rows/columns in any particular sheet too, depending on the structure of the population estimates we are downloading.

[jwestw]

Make use of ABC's abstractmethod

See the Core Programming section in the BPI. There is a section on Classes and using the abstractmethod decorator.