search

mapbox / cloudfriend

Helper functions for assembling CloudFormation templates in JavaScript
ISC License
69 stars 9 forks source link

Cloudfriend-generated Glue table missing `TableType` property #131

Closed mappingvermont closed 2 years ago

mappingvermont commented 2 years ago

👋 hi @mapbox/orchestration!

I'm trying to query a cloudfriend-generated DWH table from pyspark, and getting the following error:

Unable to fetch table accounts. TableType cannot be null for table

I can confirm there's no tabletype defined using the AWS Glue CLI:

aws glue get-table --database-name snowflake_sfdc_production --name accounts --region us-east-1 | jq ".Table | keys"

@nickcordella pointed me to the code where this particular table is created. It's using the GlueTable shortcut, which does appear to set TableType to EXTERNAL_TABLE here.

I'm not sure where this is getting dropped, but would love some help in solving it. Many thanks!

mappingvermont commented 2 years ago

Looks like this property is overwritten by AWS Glue updates later on - not a cloudfriend issue 🎉