dart-lang / sdk

The Dart SDK, including the VM, JS and Wasm compilers, analysis, core libraries, and more.
https://dart.dev
BSD 3-Clause "New" or "Revised" License
10.11k stars 1.56k forks source link

API to create deeply immutable typed data, from one or multiple chunks. #50068

Open mkustermann opened 2 years ago

mkustermann commented 2 years ago

The VM has support for deeply immutable typed data. Those can be shared by-pointer across isolates. Though right now one can only create them using an embedder API.

Using UnmodifiableUint8ListView(bytes) is not deeply immutable, because bytes may be mutable and someone may write to them.

So I propose to make an API that can make an UnmodifiableUint8ListView from any Uint8List and it may make a copy of it if the underlying bytes are mutable.

Since data is often available as chunks (e.g. loading from network results in List<Uint8List>), the API should support making UnmodifiableUint8ListView from such a set of chunks.

A possible API would be

class UnmodifiableUint8ListView {
    factory UnmodifiableUint8ListView.fromChunks(List<Uint8List> chunks);
}

/cc @lrhn opinions about the API?

lrhn commented 2 years ago

This API sounds unrelated to being unmodifiable.

We should have a modifiable ByteBuffer and an unmodifiable ByteBuffer, and then (seemingly) use the same typed-data views on top.

Building a buffer from chunks would apply to both kinds of buffers.

So, we can have both Uint8List.fromChunks(Iterable<Uint8List> chunks) and Uint8List.unmodifiableFromChunks(Iterable<Uint8List> chunks)/UnmodifiableUint8ListView.fromChunks.

Or even ByteBuffer.fromChunks(Iterable<...something...> chunks).

(The UnmodifiableThingListView types are mostly bad design. They should just be UnmodifiableThingList.)

mkustermann commented 2 years ago

Building a buffer from chunks would apply to both kinds of buffers.

So, we can have both Uint8List.fromChunks(Iterable<Uint8List> chunks) and Uint8List.unmodifiableFromChunks(Iterable<Uint8List> chunks)/UnmodifiableUint8ListView.fromChunks. Or even ByteBuffer.fromChunks(Iterable<...something...> chunks).

LGTM. @lrhn would this be something you'd like to do?

(Side note: Maybe fromList is better for consistency than fromChunks, since we have TransferableTypedData.fromList already?)

(The UnmodifiableThingListView types are mostly bad design. They should just be UnmodifiableThingList.)

Something for dart 3.0? (could make typedef now and migrate code)

lrhn commented 2 years ago

LGTM. @lrhn would this be something you'd like to do?

Design the API, sure.

Can you elaborate a little more on how the native unmodifiable type data works? Do you simply have an unmodifiable ByteBuffer which throws on modification attempts?

We can update the fromList to take a flag saying whether the result should be immutable or not:

external factory Uint8List.fromList(List<int> bytes, {bool unmodifiable = false});

The immutable list will have the same API and static type as the mutable list.

With that as a way to directly create an unmodifiable typed-data lis, I'd assume the .buffer of that would inherit the immutability, so immutableList.buffer.asUint16List() would give you an unmodifiable 16-bit list. We can choose to only use UnmodifiableUint16ListView as views (what the unmodifiable ByteBuffer.asUint16ListView returns) so that it throws early on attempts to modify, or use the normal views and have the underlying buffer throw on modification attempts.

The second part, concatenating existing bytes, could be a separate constructor:

external factory Uint8List.fromLists(Iterable<List<int>> bytes, {bool unmodifiable = false});

Those two constructor changes should be the only needed API for this to work. Then you can do everything you do today with the unmodiifable lists, like create views, as long as you don't try to change the contents.

We can consider adding a bool get isMutable to the lists, or to the buffer. It's not necessary, we don't do that for normal unmodifiable lists. (But it's technically possible because all typed-data types are sealed.)

WDYT?