API design - initial sketch

[mpizenberg]

EDIT: this has switched into a general API discussion, not only Uint8ClampdArray. Original message below

---

I randomly chose to start API design thinking with the Uint8ClampedArray typed array [1]. I've come up with this API (4e00e490, see documentation file [2] ). You can load it in elm preview docs [3]. Let me know what you think before any implementation.

[1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Uint8ClampedArray [2] documentation.txt [3] http://package.elm-lang.org/help/docs-preview

[jscriptcoder]

Great initiative!!... Let me contribute with an idea. Since all the TypedArrays share the same interface, maybe instead of wrapping all the views, it would be interesting to think about a having just two types, such as:

type Dtype
    = Int8
    | Uint8
    | Uint8Clamped
    | Int16
    | Uint16
    | Int32
    | Uint32
    | Float32
    | Float64
    | Array

type JsTypedArray
    = JsTypedArray

And instantiating the right view depending on the DType. For example:

initialize Int -> DType -> JsTypedArray
initialize length dtype =
    Native.JsTypedArray.initialize (max 0 int) dtype

Any thoughts about this?

[mpizenberg]

Thanks @jscriptcoder , you're right, we probably don't need seperate modules! I think we can group the different functions of such API into 3 categories:

1. Initializer: needs to know the datatype of the returned typed array. For example initialize : Int -> SpecializedTypedArray
2. Polymorph: independent of input type, just keep same output type. For example extract : Int -> Int -> SomeTypedArray -> SomeTypedArray
3. Elm typed: Depend on some Elm type (Int or Float for example). For example map : (Int -> a -> a) -> SomeTypedArrayOfA -> SomeTypedArrayOfA

It seems to me that any polymorph function would have the exact same code, thus could all be implemented in the same Native.JsTypedArray package.

Regarding the Initializer and Elm typed categories, we need a way to differentiate the function types and make type errors impossible. Maybe phantom types could help. What do you think of somethink like this:

type JsTypedArray a b
    = JsTypeArray

type Uint8 = Uint8
type Float64 = Float64
...

initUint8 : Int -> JsTypedArray Int Uint8
initFloat64 : Int -> JsTypedArray Float Float64
...

map : (Int -> a -> a) -> JsTypedArray a b -> JsTypedArray a b

The only issue I see with this approach is that it forces to have one function per initializer, so if we have 4 initializers per type, it implies almost 40 functions. I'm not sure it's such a bad thing. Anyway it's either this, or 4 functions with case expressions for each, which is not necessarily better, and does not provide type safety.

Any thought?

[mpizenberg]

If you'd like to have a more concrete view of what my previous comment with phantom types would look like (codewise), I've just pushed a commit (6656310) on branch polymorph showcasing it. Thoughts?

Corresponding documentation: documentation.txt

[jscriptcoder]

I see your point... but I don't understand the reason for the phantom type. Not sure what's the equivalent of Elm Int and Float in Javascript, but can we not assume that Elm Int --> Int8, Int16, Int32, and Float --> Float16, Float32, Float64... or is it too much assuming?. Still, we would need as many as TypedArray types.

initUint8 : Int -> JsTypedArray Int
initInt32 : Int -> JsTypedArray Int
initFloat64 : Int -> JsTypedArray Float
--- etc...

Or just two initializers if we pass Dtype

initInt : Int -> Dtype -> JsTypedArray Int
initFloat : Int -> Dtype -> JsTypedArray Int
--- etc...

Actually, I don't like this second option, you can always do: let int8arr = initInt 12 Float32, which is not very type safe as you can see.

[mpizenberg]

Actually even in your examples initFloat64 : Int -> JsTypedArray Float, the Float is called a phantom type. It is a type made up out of the blue. Just here to be hold by the type signature of the function in order to enforce how you can use the type. Like so, you will not be able to map a function working on Float on an array of type JsTypedArray Int.

So I guess your question is "why the other phantom type?". The other phantom type, which hold the underlying array data type has no compilation time use indeed. Its only purpose is information. As a non expert JS user, I had many wonderings when reading the typed arrays documentations. Let me take the map documentation of the Uint8Array as an example:

Uint8Array.prototype.map() Creates a new array with the results of calling a provided function on every element in this array. See also Array.prototype.map().

When I saw that, my first question was "wait, can the returned array be of a different type and hold negative integers? or floats?". Turns out nope. It returns an Uint8Array. At this moment I though we can do better in elm. One option, the simplest one, is to just have a better documentation, stating that the returned array is of the same data type than the original. But elm has a rich type system so we can do even better, we can say it in the function type. That's what brought me to adding a second phantom type, allowing type signatures like:

map : (b -> b) -> JsTypedArray a b -> JsTypedArray a b

The a in the previous type signature clearly states: "if you had Uint8, you are still going to return Uint8". While the b makes sure at compile time that you are mapping a function working on compatible types.

Also I find it useful, to have reminders in my programs functions of the kind of typed arrays I'm manipulating. Imagine you are new to a project, and you want to modify part of a code, similar to this:

doSomeStuff : JsTypedArray Int -> JsTypedArray Int
doSomeStuff typedArray =
    typedArray
        |> JsTypedArray.map ( doing )

doing : Int -> Int
    ...

At this point you don't have a clue about the fact that typedArray might be holding unsigned integers only. More so if you are new to elm and JS. When you learn elm, unsigned ints are not even types, how could this be. If you modify doing without knowing that info, the code may not behave how you thought it would.

What do you think?

[jscriptcoder]

I see... We need to get people involved in this discussion. I'm not an expert in Elm. So I'm not the best to help to design and API to cover TypedArrays. I can definitely help in the Javascript part, but I guess the idea is to write as little JavaScript as possible and MDN is a great source of information about this data structures.

How can we bring the attention to get this moving forward?. They're gonna release JsArray in the next version. I don't see why not to also include support for TypedArrays.

Sorry I answer so late. Super busy lately.

[mpizenberg]

No problem, I'm not in a hurry. I think I can ask other elm people I've already interacted with to give their ideas if they have some time. Don't hesitate to do the same if you know some who might be interested.

On Dec 2, 2017 17:41, "Francisco Ramos" notifications@github.com wrote:

I see... We need to get people involved in this discussion. I'm not an expert in Elm. So I'm not the best to help to design and API to cover TypedArrays. I can definitely help in the Javascript part, but I guess the idea is to write as little JavaScript as possible and MDN is a great source of information about this data structures.

How can we bring the attention to get this moving forward?. They're gonna release JsArray in the next version. I don't see why not to also include support for TypedArrays.

Sorry I answer so late. Super busy lately.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mpizenberg/elm-js-typed-array/issues/1#issuecomment-348680670, or mute the thread https://github.com/notifications/unsubscribe-auth/ACxXCZooUcOKXGn5HIWiAMtX-rEEtVzBks5s8Rs0gaJpZM4Qs40a .

[mpizenberg]

So I've slowly continued the implementation (branch polymorph) to check that all is coherent. At this stage there are four modules in the package:

• JsArrayBuffer -> for creation of array buffers (underlying data structure of typed arrays).
• JsUint8Array and JsFloat64Array -> for initialization of typed arrays
• JsTypedArray -> all functions to manipulate typed arrays.

The full API is available in this documentation.txt to upload on the documentation preview website. I've not implemented yet other typed arrays (only Uint8 and float64) since I don't want to overload this first draft with too many redundant information.

Functions are also tested using fuzzy testing. Tests may not check all cases (especially different typed arrays) but cover the majority I think.

If/when you think this draft is sound enough, I'd like to merge it in master and post something on the elm discourse. It might help getting some feedback.

Thoughts @jscriptcoder ?

[mpizenberg]

Now that I've implemented a first version following the structure above, I'm closing this issue in favor of this new one #5 detailing current API.