rstudio / pins-python

https://rstudio.github.io/pins-python/
MIT License
49 stars 12 forks source link

Support new board manifest `_pins.yaml` #174

Open juliasilge opened 1 year ago

juliasilge commented 1 year ago

In rstudio/pins-r#661 we added support for a manifest file. In pins for Python, we now need to:

The first one is high priority as we plan to not release pins for R until this gets implemented. The second is low priority and we can wait on it until whatever is a good time.

juliasilge commented 1 year ago
    @juliasilge it appears adding a `pins.txt` file as a manifest breaks `pin_search` on python pins. The reason is that in the python version `pin_list()` returns the filename, and `pin_search` attempts to retrieve metadata for every pin as part of search.

Here's the relevant R pins pins_search code fetching metadata (which works fine with the manifest file).

wdyt is a good move here? I can modify pins-python to ignore pins.txt files, but I'm wondering what the right way sequence is for releasing..

edit: how I tested

Using an .env file with credentials filled in from .env.dev.

In python:

from dotenv import load_dotenv
load_dotenv()
from pins import board_s3
from pins.tests.helpers import BoardBuilder

bb = BoardBuilder("s3")
board = bb.create_tmp_board("pins/tests/pins-compat")

# list path to temporary pins bucket, for copying into R
board.board

Switching to R:

board <- board_s3(<bucket_from_above>, prefix="<prefix_from_above>/")

# note that this also prints a warning
board %>% board_manifest()

Back in python:

board.pin_search("abc")

Originally posted by @machow in https://github.com/rstudio/pins-r/issues/661#issuecomment-1307576924

machow commented 1 year ago

v0.7.3 shouldn't identify _pins.yaml as a pin. Still need to add an analogue to R pin's write_board_manifest()