Add support for exotic floating point types

[scott-griffiths]

There was a previous effort to add float16 support (there's a branch somewhere). It would be interesting to also add other new types. For example:

float16 (the IEEE spec) bfloat16 float8 formats other float16 formats

The ordinary IEEE float16 could be done in the same was as the 32 and 64 bit variants. So

a = BitArray(float=0.2, length=16) b = a.float

The IEEE float16 has 5 exponent bits, bfloat has 8, and the 8-bit floats vary also.

Maybe a special case for bfloat - we don't even need to give the length: a = BitArray(bfloat=0.2)

For fp8 we can specify the sign:exponent:significand format:

a = BitArray(float152=0.2) b = a.float143

So one reasonable way to do this would be:

1) allow the float property to accept 16 bits. This is essentially what the previous abandoned work was doing, and would correspond to the IEEE float16 format. 2) Add a bfloat property. This would always be 16 bits long. 3) Add properties for float152, float143 and float134. These correspond to the formats suggested here: https://arxiv.org/pdf/2206.02915.pdf. These should remain as experimental features as the format hasn't been standardised yet (not sure if bfloat has either, but it is used a lot).

Initialising via the auto keyword would look like this?:

a = Bits("float:16=0.2, bfloat=0.3, float152=0.4")

[scott-griffiths]

The above description for float8 missed out the exponent bias, which can take on a whole range of values. So really there are dozens of related float8 formats.

Easiest thing to do would be to have a module-level bitstring.fp8_bias variable?

[scott-griffiths]

The struct module supports the e type since Python 3.6, which is the IEEE 754 16-bit float type! So that one should be easy now.

[scott-griffiths]

IEEE float-16 done. It was even easier than I thought.

[scott-griffiths]

bfloat doesn't look too bad either. Link to implementation below - it really does seem to be just zeroing out half of the float32 representation. So

    # Getting example (bf = s.bfloat)
    tmp = s + '0x0000'
    bf = tmp.float32

    # Setting example (s.bfloat = 1000.2)
    tmp.f32 = 1000.2
    s = tmp[0:16]

The above works for big-endian only.

https://github.com/tensorflow/tensorflow/blob/1e44b15ff02fe3bf4764189eb1b796602d669c3e/tensorflow/core/framework/bfloat16.cc

[scott-griffiths]

Some useful concrete examples here https://github.com/openxla/stablehlo/blob/main/rfcs/20230321-fp8_fnuz.md

[scott-griffiths]

bfloat done in 4.0. Plan to take a look at the fp8 format(s) for 4.2.

[scott-griffiths]

Also useful reference here https://jax.readthedocs.io/en/latest/_modules/ml_dtypes/_finfo.html

[scott-griffiths]

Added float8_152 and float8_143 types. Will be available in v4.1.