support for open_fs's protocols is fine, and shouldn't be dropped but a manual open_chunky and open_sga would allow for async operations, or synchronous operations with callbacks.
A simple alias would allow us to avoid rewriting open_fs(f"chunky://{path}") / open_fs(f"sga://{path}"). which isn't problematic, but isn't type-safe under mypy.
Async/Callbacks would help provide some response to users opening large SGA/Chunky assets, but would require editing the pipelines to support the 'async'/'callback' operations.
It should be noted that pyfilesystem specifically does not support async, and as of my writing this, they don't plan to.
support for
open_fs's protocols is fine, and shouldn't be dropped but a manualopen_chunkyandopen_sgawould allow for async operations, or synchronous operations with callbacks.A simple alias would allow us to avoid rewriting
open_fs(f"chunky://{path}")/open_fs(f"sga://{path}"). which isn't problematic, but isn't type-safe under mypy.Async/Callbacks would help provide some response to users opening large SGA/Chunky assets, but would require editing the pipelines to support the 'async'/'callback' operations.
It should be noted that pyfilesystem specifically does not support async, and as of my writing this, they don't plan to.