Adding race/ethnicity data and fixing final parquet schema error

makepath / census-parquet

Python tools for creating Parquet files from 2020 Census Data

MIT License

16 stars 4 forks source link

Adding race/ethnicity data and fixing final parquet schema error #10

Closed dylanrstewart closed 2 years ago

dylanrstewart commented 2 years ago

Fixes #6 #7

Proposed Changes

add function to process_blocks to merge geos and pops

dylanrstewart commented 2 years ago

there is still an error with checking the divisions of the written parquet files due to the lack of metadata being written. I am thinking this is mainly a dask versioning issue. Will investigate.

dylanrstewart commented 2 years ago

enforcing the metadata file by calling ddf.to_parquet(...,write_metadata_file=True) has fixed the divisions checks. passing all tests with dask version 2021.08.1

dylanrstewart commented 2 years ago

modified division checks to work with dask 2022.6.1 and dask-geopandas v0.2.0