Kotlin / kotlinx-io

Kotlin multiplatform I/O library
Apache License 2.0
1.31k stars 59 forks source link

Add `ByteString` conversions for JS binary types like `ArrayBuffer`, `Int8Array`, `Blob`, etc. #268

Open joffrey-bion opened 9 months ago

joffrey-bion commented 9 months ago

In the same vibe as https://github.com/Kotlin/kotlinx-io/issues/266, we could add conversions from JS binary types like ArrayBuffer, Blob, Int8Array...

Dealing with binary data in JS is a bit annoying. For example, the ArrayBuffer type is not directly usable and needs to be wrapped in a view. Providing such conversions out of the box in the kotlinx-io-bytestring artifact would be useful for a true multiplatform experience.

Examples:

/**
 * Creates a new [ArrayBuffer] containing the data copied from this [ByteString].
 */
fun ByteString.toArrayBuffer(): ArrayBuffer = toInt8Array().buffer

/**
 * Creates a new [Int8Array] containing the data copied from this [ByteString].
 */
fun ByteString.toInt8Array() = Int8Array(toArrayOfBytes())

private fun ByteString.toArrayOfBytes() = Array(size) { this[it] }

/**
 * Creates a new [ByteString] containing the data copied from this [ArrayBuffer].
 */
fun ArrayBuffer.toByteString(): ByteString = ByteString.wrap(toByteArray())

/**
 * Creates a new [ByteArray] containing the data copied from this [ArrayBuffer].
 */
fun ArrayBuffer.toByteArray(): ByteArray = Int8Array(this).toByteArray()

/**
 * Creates a new [ByteArray] containing the data copied from this [Int8Array].
 */
fun Int8Array.toByteArray() = ByteArray(length) { this[it] } // maybe there is more efficient here
fzhinkin commented 2 months ago

@joffrey-bion Are there any scenarios where these function would be useful right now? Since copying is inevitable, does it make sense to support reading/writing ArrayBuffers from/to Sources/Sinks instead?

joffrey-bion commented 2 months ago

Yes, those issues are not hypothetical cases. I needed them in my stomp library (Krossbow), to bridge web socket implementations in different platforms. I need to convert to and from ByteString to different platform-specific types. In this case, it's JS ArrayBuffer.

Since copying is inevitable, does it make sense to support reading/writing ArrayBuffers from/to Sources/Sinks instead?

The goal is to turn it into a ByteString in my case. Wouldn't going through a Source/Sink incur another copy?

Also this would force a dependency on the core library instead of just staying in the "simple" kotlinx-io-bytestring. But I think that's not a very big deal (because if we're "processing stuff", then maybe the core is needed).

lppedd commented 2 months ago

I just happened to see this issue.

Conversions of a Buffer or ByteString from and to Int8Array and Uint8Array would be a nice addition to the JS interop (only if the data is wrapped tho, and not copied unless it's strictly necessary). We use those two data types when handling TCP sockets.

Well, we mostly use Node's Buffer, but it's just a Uint8Array with some utility functions to operate on the underlying data, just like kotlinx-io's Buffer.