11 fetch preprocess gdp

[Juan-Mateos]

Closes #7 Closes #12 Closes #11

[georgerichardson]

I got the following error log when I cloned the repo and ran make install from the pip install xlrd command

ERROR: Could not find a version that satisfies the requirement xldr (from industrial-taxonomy[dev]) (from versions: none)
ERROR: No matching distribution found for xldr

[Juan-Mateos]

Thanks for the review! I have:

• Added openpyxl to requirements
• Added instructions to README
• Put hardcoded variables (excel sheets / year ranges) at the top of the script
• Made URL names more informative, split into multiline, removed commas
• Inserted whitespace before Attributes, Arguments and Returns
• Replaced regex with .replace()

[Juan-Mateos]

And then I added the getters.

[Juan-Mateos]

I did the following:

• [x] Ran flake8 and fixed problems
• [x] Simplified doc-string for get_run and changed argument name
• [x] Changed top-level descriptions to doc-string
• [x] Added Returns to getters
• [x] Added run parameter to getters
• [x] Renamed itl1_region as nuts1_name
• [x] Fixed doc-string about columns being returned from gva_pc_lad
• [x] Made imports consistent
• [x] Changed get function so it returns a response object instead of content
• [x] Made process_gdp_table more informative
• [x] Chained rename in process_gdp_table
• [x] Removed unused kwargs argument from process_nomis
• [x] Re-ran all flows without datastore=local
• [x] Tested all getters

I didn't:

• Separate aps from ashe processing in the process_nomis util
• Create separate getters for nomis variables (they are used together so this isn't necessary here)

Note that:

• I had to keep the pandas import outside of the flows. It was required for the typing hints at the beginning of the flows. When I tried to import pandas as pd when declaring the flow I got the following error: Cannot pickle dump artifact named "pd". I suppose I could:
  • Remove the pd.DataFrame typing hints
  • Declare the typing hints somewhere else in the script (in the last step?)

[Juan-Mateos]

I did the following:

• [x] Fix new flake8 problems
• [x] Include the typing import 'hack' in the gdp and nomis flows
• [x] Change itl1_region names in all getters.
• [x] Added None to industrial_taxonomy/__init__.py.
• [x] Re-ran all the flows
• [x] Tested the getters