search

bmeg / iceberg-schema-tools

Create and maintain central iceberg schema.
2 stars 0 forks source link

Feature/swap jq for glom #31

Closed matthewpeterkort closed 11 months ago

matthewpeterkort commented 11 months ago

Refs https://github.com/ACED-IDP/aced_etl_pod/issues/17

pyjq does not support python >3.10

Glom seems to be far better maintained with support for python 3.12: https://github.com/mahmoud/glom

This PR swaps out pyjq with glom. pytests pass, graph and simplified schemas match the old schemas that were generated with pyjq dependencies.

Also includes some flake8 fixes to main

matthewpeterkort commented 11 months ago

Flake8, cleanup Merge ready