rstudio / pins-python

https://rstudio.github.io/pins-python/
MIT License
46 stars 10 forks source link

fix: RsConnectFs now strips protocol #193

Closed machow closed 6 months ago

machow commented 1 year ago

This PR enables pin's fsspec implementation (RsConnectFs) to work with the new fsspec GenericFileSystem, by ensuring the protocol gets stripped from any paths. For example, "rsc://my_user" now works.

This bug did not affect any pins behaviors, but fixing it allows us to query Posit Connect from duckdb...

import pins
import duckdb

# note that my connect credentials are in a .env file
from dotenv import load_dotenv
load_dotenv()

# connect to board, register fs to duckdb ----
board = pins.board_connect("https://colorado.posit.co/rsc")
duckdb.register_filesystem(board.fs)

# look up bundle id ----
board.pin_meta("michael.chow/mtcars3")

# query with duckdb ----
query = duckdb.execute(
    "SELECT * FROM read_csv_auto('rsc://michael.chow/mtcars3/65421/mtcars3.csv')"
)

query.df()
machow commented 6 months ago

This was an interesting exploration in querying duckdb on rsconnect with pins, but I'm not sure there's much interest in this functionality. Let's close for now (and we can always take useful parts out of it later; splitting the protocol correctly is a useful fsspec activity!)