biobricks-ai / biobricks

BioBricks makes loading data from biological datasets and databases easy. Python and R interfaces, data version control, and an API for pulling datasets that have been converted to easy-to-use formats.
https://docs.biobricks.ai
MIT License
5 stars 2 forks source link

Biobricks Fails on Windows for Users Without Symlink Capabilities #18

Closed tomlue closed 4 months ago

tomlue commented 4 months ago

Issue Summary: Biobricks encounters functionality issues on Windows systems where users lack the permissions to create symbolic links. This limitation can lead to inconsistent user experiences, particularly in environments where symlink permissions vary.

Proposed Solution: To ensure consistent behavior across all user environments, including those with restricted symlink capabilities, we propose implementing a fallback mechanism. This mechanism would involve hard copying data from the cache directory to the brick directory when symlink creation is not possible.

Potential Drawback: While this approach guarantees data availability, it comes with the significant downside of increased disk space usage. Since the data would be duplicated rather than referenced, users will experience double or more the expected disk space consumption, especially in scenarios where data is shared across multiple bricks.

Rationale: The primary goal is to maintain data integrity and availability, ensuring that all users, regardless of their system's symlink capabilities, have reliable access to the necessary data. However, we must also consider the trade-off regarding disk space efficiency and seek input on prioritizing these aspects.