Add experimental GCS support in Afero. Experimental because the CI infra of afero does not test with real GCS buckets.
Limitations:
No Chmod support - The GCS ACL could probably be mapped to *nix style permissions but that would add another level of complexity and is ignored in this version.
No Chtimes support - Could be simulated with attributes (gcs a/m-times are set implicitly) but that's is left for another version.
NOTE: Not thread safe - Also assumes all file operations are done through the same instance of the GcsFs. File operations between different GcsFs instances are not guaranteed to be consistent.
Performance implications
Sequential reads are performant
Sequential writes are performant.
Seek + Read or ReadAt is performant after the initial seek. (produces a warning)
Alternating reads/writes to the same file handler are highly inefficient. To get consistent FS behavior using an API that separates readers and writers we close any open readers before an write as well close open writers before a read (ensure the data is committed).
Seek + Write such as WriteAt, Truncate, Seek+Write will work as expected but with significant overhead. Doing a seek + write will in effect download the old file/object, overlay it with the new writes and save it back. This is done in a streaming fashion so large files will not clog the memory but will trigger a full download and upload of the file/object.
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
dependabot[bot]
commented
2 years ago
Bumps github.com/spf13/afero from 1.4.1 to 1.9.0.
Release notes
Sourced from github.com/spf13/afero's releases.
Commits
b0a534aUpdate README.mdc92ae36Make IOFS.ReadDir check for fs.ReadDirFile52b6417Fix staticcheck lint errors939bf3dFix test failures on Windows9439436all: Run gofmt -s -w015be45Move CI tests to GitHub Action100c9a6Merge pull request #346 from jeffwidman/bump-minimum-go-versionc45d7c7Bump minimumgoversion to1.168d9b795go mod tidya2bf4c8Merge pull request #329 from yearnfar/masterDependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)