dataverbinders / statline-bq

Library to fetch CBS open datasets into parquet and optionally load into Google Cloud Storage and BigQuery
MIT License
0 stars 0 forks source link

Fix data properties #70

Closed galamit86 closed 3 years ago

galamit86 commented 3 years ago

This PR:

dkapitan commented 3 years ago

@galamit86 You may want to consider including a more generic clean_names function where we could add more logic as we go along.

See https://gist.github.com/dkapitan/89ff20eeed38e6d9757fef9e09e23c3d

I have written than in the past to get rid of spaces. We could at a line to replace periods

galamit86 commented 3 years ago

@dkapitan Added generic function based on yours, with 2 differences:

  1. Added option to add other characters to replace (in this instance adding ".")
  2. Return s instead of s.lower() - I tend to stick to the original names as far as possible. Additionally, since this is is not applied on all tables, but only on DataProperties, I think it's better not to lowercase only some tables.
dkapitan commented 3 years ago

@galamit86 Looks good to me!