blob: figure out a design/strategy for provider-specific string restrictions

google / go-cloud

The Go Cloud Development Kit (Go CDK): A library and tools for open cloud development in Go.

https://gocloud.dev/

Apache License 2.0

9.55k stars 811 forks source link

blob: figure out a design/strategy for provider-specific string restrictions #905

Closed vangent closed 5 years ago

vangent commented 5 years ago

@vsaroopchand has discovered two things while working on Azure support:

Azure only allows C# identifiers in Metadata keys.
Azure has subtle behavior regarding "\" in blob names. Something like: -- It is not illegal in the name. -- It is illegal as a delimiter. -- It is interpreted as a "/" when doing virtual directories.

We don't have a good story for this right now. There's some discussion here.

For now, I'm going to disable some conformance tests so that the Azure contribution is unblocked.

vangent commented 5 years ago

Consider strongly typing names instead of using string? E.g., blob.Name. This might provide hooks for validation, etc.
Consider having provider implementations accept translation functions (e.g., Azure constructor could take a function for metadata keys that converts arbitrary keys to C# identifiers). This can only partially work since you can't recover the original name, so writing and then reading back would give a changed result.

jba commented 5 years ago

This can only partially work since you can't recover the original name...

You can if you designate some set of names as being off limits. Then you can recognize the mangled names and unmangle them. For example, you could say that user-provided names beginning and ending with double-underscores are not allowed. Then when someone gives you a-b you can losslessly translate it __aX2Db__ (where X.. denotes "character with hex code ..").

vangent commented 5 years ago

/cc @zombiezen