Add json as allowable file type to copy_s3

move-coop / parsons

A python library of connectors for the progressive community.

https://www.parsonsproject.org/

Other

258 stars 128 forks source link

Add json as allowable file type to copy_s3 #844

Closed KasiaHinkson closed 11 months ago

KasiaHinkson commented 1 year ago

Add json as allowable file type to copy_s3. Since it's converting to a Parsons table, should be no downstream effects.

KasiaHinkson commented 1 year ago

My test case failed because it's line delimited JSON, which Redshift can't handle because Redshift is a big baby, so I'll test this as soon as I can find a better test case

KasiaHinkson commented 12 months ago

Ok, this branch has now been tested VERY thoroughly, our Hustle sync is actively running on this branch, so I'm confident it's ready for review