cloudyr / googleCloudStorageR

Google Cloud Storage API to R
https://code.markedmondson.me/googleCloudStorageR
Other
104 stars 29 forks source link

Get Full size of object, in bytes #156

Closed muschellij2 closed 2 years ago

muschellij2 commented 2 years ago

I would like the size (in bytes) to check against a listing to make sure all bytes are downloaded. The issue is that googleCloudStorageR automatically formats this: https://github.com/cloudyr/googleCloudStorageR/blob/ae8670ed60f7bd70f1b9399565a473f90a3134ff/R/objects.R#L94 And you can't set an option for format_object_size. I think if you use something like:

format_object_size(getOption("googleCloudStorageR.format_units", default = "auto"))

Then you can set an option("googleCloudStorageR.format_units" = "b") then this should work I think

head(googleCloudStorageR::gcs_list_objects()$size)
#> [1] "218.8 Mb"  "137.9 Mb"  "1.1 Kb"    "45.6 Kb"   "74 bytes"  "159 bytes"
head(googleCloudStorageR::gcs_list_objects(detail = "full")$size)
#> [1] "218.8 Mb"  "137.9 Mb"  "1.1 Kb"    "45.6 Kb"   "74 bytes"  "159 bytes"

Created on 2022-03-07 by the reprex package (v2.0.1)

MarkEdmondson1234 commented 2 years ago

I'd rather set fewer rather than more options, how about adding a column to the data.frame output called "size_bytes" ?

muschellij2 commented 2 years ago

Yeah I think that'd work too. Just figured i on the one hand you have checks on the column names or something like that it'd be an issue. On the other hand, you may have data type checks which may be different (size is character, size_bytes is likely numeric).

MarkEdmondson1234 commented 2 years ago

Lets try it see what breaks ;) it may be a chance to also inspect what the API returns, it may have some added fields from when it was first done that aren't surfaced (perhaps versioning?)