search

SatcherInstitute / prototype

MIT License
4 stars 4 forks source link

Household income to BQ #44

Closed jenniebrown closed 3 years ago

jenniebrown commented 3 years ago

Adds code for uploading household income data to BQ from GCS. Also increases pub/sub subscription ack timeouts to prevent infinite request spam.

mayaspivak commented 3 years ago

https://github.com/SatcherInstitute/prototype/pull/45/files is my draft attempt to generalize the append_dataframe_to_bq util function.

I tested it with my inferred table schema and an explicit schema table and it seems to work.

Would this work for your data source?

jenniebrown commented 3 years ago

https://github.com/SatcherInstitute/prototype/pull/45/files is my draft attempt to generalize the append_dataframe_to_bq util function.

I tested it with my inferred table schema and an explicit schema table and it seems to work.

Would this work for your data source?

Yeah, I believe that would work. Do you mind if I incorporate it into my change?

mayaspivak commented 3 years ago

https://github.com/SatcherInstitute/prototype/pull/45/files is my draft attempt to generalize the append_dataframe_to_bq util function. I tested it with my inferred table schema and an explicit schema table and it seems to work. Would this work for your data source?

Yeah, I believe that would work. Do you mind if I incorporate it into my change?

Not at all. Please do!