search

silx-kit / h5web

React components for data visualization and exploration
https://h5web.panosc.eu/
MIT License
183 stars 18 forks source link

Fetch float16 and float128 with h5grove as binary using `dtype=safe` #1561

Closed axelboc closed 7 months ago

loichuder commented 7 months ago

Sorry, took a while to get back to this and I would need some context to refresh my mind.

Why using dtype=safe makes it possible to fetch float16 and float128 as binary ?

axelboc commented 7 months ago

Sorry, took a while to get back to this and I would need some context to refresh my mind.

Why using dtype=safe makes it possible to fetch float16 and float128 as binary ?

No worries!

There's no Float16Array and Float128Array in JS, so if we fetched float16/128 datasets with format=bin without dtype=safe, we'd get binary we don't understand (at least without using custom libraries like https://github.com/petamoriken/float16 or https://github.com/munrocket/double.js).

When fetching float16/128 datasets with format=bin and dtype=safe, h5grove converts them to float32/64 respectively. With this knowledge, we can adjust our provider logic to say that when fetching float16, the response buffer should be passed into a Float32Array and when fetching float128, into a Float64Array. (Before, we were saying that float16/128 datasets had no corresponding typed array, which led to fetching them as JSON).