Proposal: Posit support

[ghost]

Posits, also known as Type III unums, are a proposed hardware numeric format with many documented advantages:

• higher precision around 1 and higher dynamic range compared to same-length floating point types
• much simpler operations and mechanics; implementing the entire specification is possible and advantageous
• certain classes permit a sigmoid operation which is almost computationally free
• implementing hardware can be smaller and more efficient than equivalent FP hardware, and
• hardware shares much circuitry with the integer ALU

Posits are not currently implemented in any low-level programming language or processor architecture, precisely because they are not implemented in any processor architecture or low-level programming language. No one will do it unless someone does it, and I believe we in Zig are in a better position than most to turn the screws.

Why do I say this?

• Zig is on a trajectory to take over the world, even infiltrating ecosystems that would otherwise have no interest in the language itself. We probably won’t ever reach C or C++’s ubiquity, but in the far future, I believe that hardware designers will be more likely to take us seriously than any other recent language.
• Hardware is utterly useless unless there is software to take advantage of it; however, software can run even if all of the hardware is not there. We as a software project are in a unique position to provide posit support alone, even without hardware — and it may even be efficient, at the very least much more efficient than soft float; see the final point above.

This is, of course, a useless feature. That means that no one will do it. But then, someone will do it, and it will suddenly be very useful. That will only happen, however, if someone does something useless first. What we have here is a unique opportunity, not only to improve the state of software, but the state of hardware.

Language Changes

Introduce a new keyword family: pXXuY, indicating a posit with n = XX and es = Y. (Both of these are necessary, as the structure of posits means that fine-tuning the parameters for different use cases is practical.) So for instance, the latest draft standard for a 32-bit posit would be written as p32u2, and 64-bit as p64u3. These are usable with all operations and builtins that take floating point values currently (modulo some renaming of functions that mention floats specifically).

[daurnimator]

Related #3148

As a more general issue: how should we add new numeric types going forward? e.g. Unum. With zig not supporting operator overloading, such types would have to be provided by the core for ergonomic use.

[matu3ba]

My main concern is 1. missing proper statistics of performance on posits vs IEEE754 and 2. no given overview paper or structured collection of arguments/properties of what guarantees IEEE754 gives in contrast to posits and vice versa.

The latter is discussed in this paper. Similarly I did not find a collection with statement of posit authors yet ie to wikipedia critique.

There are several applications in machine learning (no big zig community yet) and general error handling stuff.

A nice blog post on the arguments of the pro-posit paper with some reasons as list.

A related discussion on the julia forum that mentions that IEE754 will fix some shortcomings (although the next few years nothing is expected to change).

So to me it looks a lot of the stuff is still quite messy and posit team does not give a current working status on critique etc and instead tries to come up with examples where posit is clearly better. So my conclusion is that they currently want to evaluate further to come up with more tailored benchmarks/use cases.

[ghost]

John L. Gustafson: Posits are the End of Error.

Everybody else: Posits are a different way of sampling a bounded interval of the real number line, improving accuracy in some cases and reducing it in others (e.g. multiplication). Overall superiority to floating point and other number systems is unclear.

From the Dinechin paper linked by @matu3ba:

There are clear use cases for the posit system. Machine learning, graphics rendering, some Monte Carlo methods, integration-based methods where the magnitude of the result can be framed, and many other applications belong there. There are also clear situations where posits are worse than floating-point. Particle physics simulations are one example, integration methods where the result is unbounded a priori is another one. Between these extremes, writing the equivalent of a BLAS library that is both efficient and accurate whatever the combination of inputs is probably more of a challenge with posits than with floats.

When posits are better than floats of the same size, they provide one or two extra digits of accuracy [19]. When they are worse than floats, the degradation of accuracy can be arbitrarily large. This simple observation should prevent us from rushing to replace all the floats with posits.

In my opinion, posit support is not a bad idea as such, but firmly belongs in a library. The argument that posits will be the next big thing, and Zig should therefore take an active part in promoting them, does not strike me as convincing.

[FlyingWombat]

In my opinion, posit support is not a bad idea as such, but firmly belongs in a library.

Forgive my ignorance, but is it even possible in (current) Zig to add support for a new primitive type via a library? For composite types (e.g. Matrix, Vector, etc.), a library works -- and I agree that such things should be in libraries.

Also, as someone else mentioned (current) Zig does not support operator overloading, so the new primitive type implemented via library would not be able to use operators.

[ghost]

Forgive my ignorance, but is it even possible in (current) Zig to add support for a new primitive type via a library? For composite types (e.g. Matrix, Vector, etc.), a library works -- and I agree that such things should be in libraries.

With inline assembly, I don't see why not. But yes, there would be no neat mathematical expressions like a + 5 * b. You'd have to write it as a.add(b.mul(Posit.fromInt(5)). That's not very nice, but it's the same situation that any mathematical library would find itself in. Obviously, native posit support would be much more streamlined, but where do we draw the line? Should we also add native LNS support? Decimal (or even arbitrary-base) floating point? Complex numbers? Quaternions? Exact fractions?

In my opinion, one should simply accept that Zig is not shaping up to be an amazing language for numerical computing. The decision to dispense with operator overloading, RAII, built-in iteration, closures, etc. makes a lot of sense for a lot of use cases, but it does remove many conveniences that make mathematical programming more pleasant in languages like C++ or Julia. Zig aims to be a small language with a sharp focus, so sacrifices have to be made somewhere.

[ityonemo]

So I'm the OG implementor/namer of posits[0] (https://github.com/interplanetary-robot/SigmoidNumbers; https://www.youtube.com/watch?v=aP0Y1uAA-2Y&t=3180s), strong recommend to not really bother, but stuff like this maybe becoming useful (and other things really being useful - like GF256 arithmetic) did inform my suggestions about operator overloading/custom overrideable binary operators.

[0] JG claims to have created the name, but I have wanted to retcon positronic brains for about a decade now, so that's how I picked the name.