rstudio / pins-r

Pin, discover, and share resources
https://pins.rstudio.com
Other
312 stars 63 forks source link

Differences in pin_versions output based on board type #756

Closed dareneiri closed 1 year ago

dareneiri commented 1 year ago

When I write to pins::board_folder(), I get a different output from pin_versions() than if I use pins::board_connect() for Posit Connect.

  1. Is this a matter of waiting for the connectapi package to get updated?
  2. Is there a plan to use the version string from board_folder for connect boards? I definitely appreciate the more meaningful output for the version.
  3. I'm confused with the size output when using board_connect -- the value is different than what is displayed in connect.

Using board_folder

> board <- pins::board_folder("~/tmp/pins")
> board %>%  pin_write(mtcars, name = "mtcars-WriteItemsToConnect")
Guessing `type = 'rds'`
Creating new version '20230706T164326Z-64844'
Writing to pin 'mtcars-WriteItemsToConnect'
> board %>%  pin_versions("mtcars-WriteItemsToConnect")
# A tibble: 1 × 3
  version                created             hash 
  <chr>                  <dttm>              <chr>
1 20230706T164326Z-64844 2023-07-06 16:43:26 64844

Using board_connect

> board <- pins::board_connect(
+   auth = "manual",
+   server = CONNECT_URL,
+   key = Sys.getenv("API_KEY")
+ )
Connecting to Posit Connect 2023.03.0 at <https://URL>
> 
> # pin an object
> board %>%
+   pin_write(mtcars, name = "mtcars-WriteItemsToConnect")
Guessing `type = 'rds'`
! The hash of pin "mtcars-WriteItemsToConnect" has not changed.
• Your pin will not be stored.
> board %>%  pin_versions("mtcars-WriteItemsToConnect")
# A tibble: 1 × 4
  version created             active  size
  <chr>   <dttm>              <lgl>  <dbl>
1 45726   2023-07-06 16:36:00 TRUE   22252

Output above shows size of 22252, but below the size is 1218

image

juliasilge commented 1 year ago

The version ID on Posit Connect for pins is the ID of the "bundle"; if you look in the Connect UI for one of your pins as shown in that support article, you'll see the same bundle IDs that you'll see when calling pin_versions(). A folder on your computer obviously doesn't have that kind of ID infrastructure, so we have to make up a version using the timestamp and hash. We do need to use the bundle ID to get out versions correctly from the Connect API so that's not something that we can change currently. You do have access to both the timestamp and pin contents hash in the pin metadata, if you need those for your use case.

For the size, sizes of different things are being measured. Notice the difference between file_size and size:

library(pins)
board <- board_connect()
#> Connecting to Posit Connect 2023.05.0 at <https://colorado.posit.co/rsc>
board |> pin_meta("julia.silge/nice-numbers")
#> List of 12
#>  $ file       : chr "nice-numbers.rds"
#>  $ file_size  : 'fs_bytes' int 62
#>  $ pin_hash   : chr "c282ef5efc7f30fb"
#>  $ type       : chr "rds"
#>  $ title      : chr "nice-numbers: a pinned integer vector"
#>  $ description: NULL
#>  $ tags       : NULL
#>  $ created    : POSIXct[1:1], format: "2023-04-27 15:25:25"
#>  $ api_version: num 1
#>  $ user       : list()
#>  $ name       : chr "julia.silge/nice-numbers"
#>  $ local      :List of 4
#>   ..$ dir       : 'fs_path' chr "~/Library/Caches/pins/connect-31a399613ac85efda77ffe6ff20c105a/1cb6f19d-c26e-45d4-95e6-40084824b999/73310"
#>   ..$ url       : chr "https://colorado.posit.co/rsc/content/1cb6f19d-c26e-45d4-95e6-40084824b999/_rev73310/"
#>   ..$ version   : chr "73310"
#>   ..$ content_id: chr "1cb6f19d-c26e-45d4-95e6-40084824b999"
board |> pin_versions("julia.silge/nice-numbers")
#> # A tibble: 3 × 4
#>   version created             active  size
#>   <chr>   <dttm>              <lgl>  <dbl>
#> 1 73310   2023-04-27 15:25:00 TRUE   19315
#> 2 73309   2023-04-27 15:25:00 FALSE  19313
#> 3 70040   2023-02-28 16:56:00 FALSE  19223

Created on 2023-07-06 with reprex v2.0.2

In the pins metadata (the YAML you shared) the file_size is the size of the pins contents only, as you saved it. If you pinned mtcars as a CSV, it would be this:

path <- fs::file_temp(ext = "csv")
readr::write_csv(mtcars, path)
fs::file_size(path)
#> 1.25K

Created on 2023-07-06 with reprex v2.0.2

When you call pin_versions() for Connect, the size is the size of the bundle. You can read more about what is returned here.

We could consider changing the column name there to bundle_size, although that would technically be a breaking change.

dareneiri commented 1 year ago

Thank you so much @juliasilge for your thorough response! The size and file_size difference makes more sense now.

It also helps to know that for Connect, the version refers to the bundle. The bundle number was not displaying in the Connect UI because I was a viewer of the pin that I was looking at specifically.

And as a viewer you do not see the bundle version:

image

But as a collaborate you do see the bundle version.

This won't be much of an issue as we're migrating from an older version of pins to the latest version (1.2.0) which now includes the created column when using pin_versions(), in addition to the version column. So when comparing the output of pin_version() to the Connect UI, it will be easier.

This can be closed.

github-actions[bot] commented 1 year ago

This issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.