gzuidhof / zarr.js

Javascript implementation of Zarr
https://guido.io/zarr.js
Apache License 2.0
133 stars 23 forks source link

feat: add (optional) float16 support #129

Closed manzt closed 2 years ago

manzt commented 2 years ago

Fixes #127

This PR implements optional support for float16 datatypes by inspecting globalThis.Float16Array. Rather than adding https://github.com/petamoriken/float16 as a dependency, users must opt into this behavior by adding Float16Array as a global in their environment.

import { openArray } from 'zarr';

const z = await openArray({ store: "http://example.com/f2data.zarr" });
await z.getRaw(null) // throws: '<f2' is not supported natively in zarr.js. In order to access this dataset you must make Float16Array available as a global. See https://github.com/gzuidhof/zarr.js/issues/127

import { Float16Array } from '@petamoriken/float16';
globalThis.Float16Array = Float16Array;
await z.getRaw(null)  // { data: Float16Array, ... }
manzt commented 2 years ago

cc: @keller-mark

manzt commented 2 years ago

From the typescript perspective, there is a more “correct” way of communicating this behavior, which I will add to this PR https://github.com/DefinitelyTyped/DefinitelyTyped/blob/fb66211010980641f01bcfe8ffbc0cc3d71989ca/types/ndarray/index.d.ts#L43

manzt commented 2 years ago

Ok, I've changed the TYPED_ARRAY_MAPPING to inspect globalThis on import only, and also removed the global declaration from zarr. This way it is up to the end user to add the global declaration and be explicit about the type.

with globalThis.Float16Array defined in user code (TypedArray union includes Float16Array).

image

without globalThis.Float16Array defined in user code (TypedArray union does not include Float16Array).

image

ref: https://github.com/DefinitelyTyped/DefinitelyTyped/pull/56163#discussion_r732279580

I'll merge and add some additional docs.