You mentioned that you could add some STAC specific settings later on in a different PR. No need to do that now but could you make an issue that outlines some of your ideas so that we don't lose track of them.
The optional-components/stac-data-proxy allows the platform to share file contents referenced by STAC collections and items externally. Since this content can have variable media-types, sizes, stream strategies, the optimized methods to expose this data efficiently must be explored, such that Nginx configurations provide the best user experience possible.
A few initial ideas:
[ ] STAC employs collections to regroup items and assets. Specific subdirectories will therefore most probably be used to categorize those elements.
[ ] Cowbird could be used to sync access between those file-directories from nested secure-data-proxy/stac/<collection-id> under Magpie, and the corresponding stac/stac/<collection-id> to access the STAC API through Magpie/Twitcher.
[ ] A similar concept to Magpie's browse/read used for THREDDS could be defined for STAC-API (browse) vs stac-data-proxy (read).
[ ] Common assets in STAC that could require Nginx optimizations for file-stream transfer
[ ] model references (in exploration) (pytorch .ckpt, ONNX, etc.)
[ ] others?
[ ] stac-data-proxy could allow file-upload (only download for now via Magpie read permission)
Given that the file structure to be used under the directory aliased by Nginx could vary depending on necessary optimizations, providing a way to upload STAC assets at the same time as pushing STAC Collection/Item references to STAC-API could require some more advanced logic.
Originally posted by @mishaschwartz in https://github.com/bird-house/birdhouse-deploy/pull/403#pullrequestreview-1757787855
The
optional-components/stac-data-proxyallows the platform to share file contents referenced by STAC collections and items externally. Since this content can have variable media-types, sizes, stream strategies, the optimized methods to expose this data efficiently must be explored, such that Nginx configurations provide the best user experience possible.A few initial ideas:
[ ] STAC employs collections to regroup items and assets. Specific subdirectories will therefore most probably be used to categorize those elements.
secure-data-proxy/stac/<collection-id>under Magpie, and the correspondingstac/stac/<collection-id>to access the STAC API through Magpie/Twitcher.browse/readused for THREDDS could be defined for STAC-API (browse) vsstac-data-proxy(read).[ ] Common assets in STAC that could require Nginx optimizations for file-stream transfer
[ ]
stac-data-proxycould allow file-upload (only download for now via Magpiereadpermission)Given that the file structure to be used under the directory aliased by Nginx could vary depending on necessary optimizations, providing a way to upload STAC assets at the same time as pushing STAC Collection/Item references to STAC-API could require some more advanced logic.