search

WPRDC / wprdc-etl

MIT License
8 stars 3 forks source link

added use of encoding in RemoteFileConnnector, way to style headers on load #47

Closed saylorsd closed 8 years ago

saylorsd commented 8 years ago

implemented encoding in RemoteFileConnector

added method to change header style when loading into CKAN. This is primarily used to match heading styles to those of legacy datasets.

bsmithgall commented 8 years ago

Why not use the built-in Marshmallow dump_to on a field? That accomplishes the same thing without creating a duplicate copy of the data.

saylorsd commented 8 years ago

@bsmithgall Take a look when you have a second. In pipeline.load_line I have it return the data from schema.dump, so it both loads the data for validation etc. and and then dumps it so that it matches the format needed for the loader.

bsmithgall commented 8 years ago

:+1: LGTM