`Pointer<Utf8>` and `Pointer<Utf16>` should be allocatable

dart-lang / native

Dart packages related to FFI and native assets bundling.

BSD 3-Clause "New" or "Revised" License

114 stars 40 forks source link

`Pointer<Utf8>` and `Pointer<Utf16>` should be allocatable #932

Open robertbastian opened 10 months ago

robertbastian commented 10 months ago

If I have a string which I know fits in exactly n UTF-8 code units, I should be able to do allocator<Utf8>(n). This is currently not possible and requires doing allocator<Uint8>(n).cast<Utf8>().

dcharkes commented 10 months ago

I believe this only makes sense for non-zero-terminated strings, where the length in bytes is known in advance.

https://github.com/dart-lang/native/issues/933

So maybe the ZeroTerminatedUtf8 should extend Opaque while NonZeroTerminatedUtf8 should be extends Struct with a single Uint8 field. (Names TBD.)

robertbastian commented 10 months ago

ZeroTerminatedUtf8 could extend Utf8, as it's a special kind of Utf8 pointer, and Utf8 would continue to extend NativeType. The extension methods would be:

String Pointer<ZeroTerminatedUtf8>.toDartString() (this is unsafe if the zero is missing!)
String Pointer<Utf8>.toDartString(length)
(Pointer<Utf8>, int) String.toUtf8(Allocator, bool zeroTerminate)

dcharkes commented 10 months ago

I'm afraid letting the two types extend each other doesn't work as an API right now. Native types need to extend Opaque (un-sized, not allocatable), or Struct (sized and allocatable).

I think we can opt to rename the Utf8 to ZeroTerminatedUtf8 if we're certain the non-zero-terminated one is more common. Introducing Utf8 as the non-zero-terminated should be okay with a major version bump. The new API has different types than the old API, so users bumping the package version should get compile-time errors.

Are there any other types of strings we should consider supporting before we make a breaking change?