meztez / bigrquerystorage

R Client for BigQuery Storage API
Apache License 2.0
19 stars 3 forks source link

downloading FLOATs from bigquery as Integers #76

Closed michaelgap360 closed 1 week ago

michaelgap360 commented 1 week ago

bqs_table_download now seems to be downloading all FLOATS in big query tables as integers,

have tried downloading the same table using bq_table_download and it worked fine

looks like bqs_table_download is returning a tibble by default now, is this converting the columns to integers?

meztez commented 1 week ago

Do you have an example I can use to reproduce the problem? What version are you using?

michaelgap360 commented 1 week ago

I've updated to the latest version (run ran the normal install.packages("bigrquerystorage") and it looked like it updated all fine)

as an example


upload <- data.table(number =  rnorm(10, mean = 0, sd = 1))
upload$text <- "some_text"

bq_table_upload('TABLE_NAME',
                upload,
                write_disposition='WRITE_TRUNCATE',
                create_disposition='CREATE_IF_NEEDED')

download <- bq_table_download('TABLE_NAME')

this gives the correct data image

but

download <- bqs_table_download('TABLE_NAME')

gives it as an integer image

michaelgap360 commented 1 week ago

table in BQ image

meztez commented 1 week ago

I can replicate it. Set bigint = "numeric" in bqs_table_download. I'll put a fix in. And yes it's tibble now, but the issue is the condition evaluated to decide if a certain logic has to be applied in the post processing.

michaelgap360 commented 1 week ago

brill thanks

meztez commented 1 week ago

@michaelgap360 Should be fixed on main branch.