francisrstokes / construct-js

🛠️A library for creating byte level data structures.
MIT License
1.37k stars 29 forks source link

Question: Does it work with floats? #37

Open theodesp opened 12 months ago

theodesp commented 12 months ago

Thank you for the library. I was wondering to know if this library works with floats or is a way to serialize single or double precision numbers and restore them for later ( with some loss of precision).

fudgepop01 commented 10 months ago

unless I'm mistaken, this project does not appear to be maintained much these days. However, it's fortunately fairly easy to make whatever custom construct class you wish thanks to how construct-js works - and I happened to have had to tackle that issue in the past myself using typescript and I can share it here 😄

the particularly unfortunate thing is, every number in javascript is internally a 64 bit number, and as such, only values of (2 ^ 53 - 1) will accurately serialize to double precision floating point numbers. It is also LUDICRIOUSLY difficult to serialize to half-precision IEE754 floats in javascript alone. The only webapp I've seen that can do this uses a gargantuan library that I don't fully understand. 🧙

(see: https://evanw.github.io/float-toy/)

As for implementation, this is what I did for 32-bit floats using a package called ieee754:

(https://www.npmjs.com/package/ieee754)

import { write as floatWrite } from 'ieee754';

/////////////////////////////////////////////////////
// basically a trimmed down version of the field class in construct-js' source code
/////////////////////////////////////////////////////

import { Endian } from "construct-js";

const assert = (condition, message) => {
  if (!condition) {
      throw new Error(message);
  }
};

export default class FloatField {
  public width;
  public min;
  public max;
  public toBytesFn;
  public value;
  public endian;

  constructor(width, min: number, max: number, toBytesFn: (vals: number[], isLE: boolean) => Uint8Array, value: number, endian: Endian) {
      this.width = width;
      this.min = min;
      this.max = max;
      this.toBytesFn = toBytesFn;
      this.assertInvariants(value);
      this.value = value;
      this.endian = endian;
  }
  assertInvariants(value) {
      assert(value >= this.min && value <= this.max, `value must be an integer between ${this.min} and ${this.max}`);
  }
  computeBufferSize() { return this.width; }
  toUint8Array() {
      return this.toBytesFn([this.value], this.endian === Endian.Little);
  }
  set(value) {
      this.assertInvariants(value);
      this.value = value;
  }
  get() { return this.value; }
}

/////////////////////////////////////////////////////
// the implementation of the F32Type
/////////////////////////////////////////////////////

const IEEE754_FLOAT32_MAX = (2 - (2 ** -23)) * (2 ** 127);
const IEEE754_FLOAT32_MIN = IEEE754_FLOAT32_MAX * -1;

const f32Tou8s = (vals: number[], isLittleEndian: boolean) => {
  const stride = 4;
  const buff = new Uint8Array(vals.length * stride);
  for (let [i, val] of vals.entries()) {
    floatWrite(buff, val, i * stride, isLittleEndian, 23, 4);
  }
  return buff;
}

export class F32Type extends BaseField {
  constructor(value: number, endian: Endian = Endian.Little) {
    super(4, IEEE754_FLOAT32_MIN, IEEE754_FLOAT32_MAX, f32Tou8s, value, endian);
  }
}

export const F32 = (value: number, endian: Endian = Endian.Little) => new F32Type(value, endian);

Hope this helps!