Suggestions for additional floating-point types

aaronfranke commented 5 years ago

I noticed that, like other languages, the only floating-point types built-in are f32 and f64. However, only having these can be limiting. I propose adding f128, and as mentioned in this thread f16 would likely be very useful for some workloads.

f128 would not be needed in most programs, but there are use cases for them, and it'd be nice to have it as a language built-in type. RISC-V is able to hardware accelerate them using the Q extension.

f16 is a more efficient type for workloads where you need tons of floats at low precision, like machine learning. Hardware using this is already widespread in Apple's neural engine and in mobile graphics.

Also, if covering IEEE-754 is desired, then there's also f256.

Original text:

I noticed that, like other languages, the only floating-point types built-in are `f32` and `f64`. However, I often have limitations with just these. I propose the following: ~~`fsize`, `freal`~~, and `f128` ~~`fsize` would be like `isize` but for floats. Basically, use the version that's most efficient for your processor. On modern 64-bit processors with wide FPUs and/or 256-bit SIMD this would become `f64`.~~ ~~Sometimes I want to be able to have a variable for real numbers, or I don't know what precision I want yet. In C++ I can do the following to have an abstract precision that I control via compiler flags:~~ ~~`#ifdef REAL_T_IS_DOUBLE`~~ ~~`typedef double real_t;`~~ ~~`#else`~~ ~~`typedef float real_t;`~~ ~~`#endif`~~ ~~I propose something similar in Rust, where you can just write `freal` or something and be able to change the precision later with compiler flags. The default would probably be `f32`.~~ Finally, it would be nice to have 128-bit floats (`f128`) in the language. These are not normally needed, but there are use cases for them, and it'd be nice to have it as a language built-in type. Some newer processors have 512-bit SIMD chipsets that can process these efficiently, though most don't. If you only implement some of these proposals, that's fine too. Originally posted at https://github.com/rust-lang/rust/issues/57928

duplexsystem commented 1 year ago

'f16' and 'fp128' would be particularly useful in combination with std:simd, especially for running rust on things like a GPU.

rdrpenguin04 commented 1 year ago

f128 would be useful for calculators that require advanced precision; I'm actually blocked on such a type existing at a critical point in development.

programmerjake commented 1 year ago

f128 would be useful for calculators that require advanced precision; I'm actually blocked on such a type existing at a critical point in development.

if you need high precision floats but can't wait, you can use https://docs.rs/rug/1.19.2/rug/struct.Float.html

tgross35 commented 1 year ago

At this point I think this more or less just needs a RFC, right? (No I am not volunteering to write it)

I think that most everyone here would be on board with a minimal implementation like this:

/// Available on platforms that support f16
/// ARM and AArch64 have this to my knowledge with __fp16
#[cfg(target_has_f16)]
f16

/// Available on platforms that support true 128-bit floats
#[cfg(target_has_f128)]
f128;

/// Exact semantics of c `long double`
core::ffi::c_longdouble;

/// ...maybe? I don't know enough about it
core::ffi::c_bfloat16;

And just not supporting 80-bit fake f128 as a native rust type, only via c_longdouble.

Not positive about f16 as it hasn't been discussed much here. Apple's M1 is the only mainstream platform I know of that has a half precision unit in CPU

aaronfranke commented 1 year ago

C long double is not suitable to use as-is because it is usually not 128-bit. For example it's 64-bit on Microsoft platforms, equivalent to double. I don't think Rust has to depend on C's types, right?

tgross35 commented 1 year ago

Not depend on, but be able to interface with. E.g. core::ffi::{c_long, c_ulong} are defined as i32/u32 on Windows and i64/u64 on linux.

So providing c_longdouble would give a way for rust <-> C FFI, plus provide an escape hatch for platforms that have 80 bit precision but not true 128. Good point about microsoft though - what is their 128 bit type?

aaronfranke commented 1 year ago

GCC and Clang/LLVM support __float128 as a compiler-specific keyword for 128-bit floats. As far as I know MSVC does not support 128-bit floats at all, so you have to use a library if you want 128-bit floats. There isn't yet a standardized keyword in C for 128-bit floats (and as mentioned long double won't work).

programmerjake commented 1 year ago

There isn't yet a standardized keyword in C for 128-bit floats (and as mentioned long double won't work).

afaict the standardized C keyword is _Float128: https://en.cppreference.com/w/cpp/types/floating-point

tgross35 commented 1 year ago

I was imagining that target_has_f128 would only be true if (1) there is a __float128 or similar, and (2) it is truly f128 (not 80 bit). So in this case, you just couldn't use f128 on Windows with MSVC if it doesn't support it - similar to the target_has_atomic gates.

tgross35 commented 1 year ago

The wiki page actually has a good summary on this, under the "computer language support" page. I suppose that f128 could be allowed anywhere that C's _Float128 is available, with the same rules/targets. Same story for f16/_Float16 (thanks Jake for pointing out the canonical names)

I guess the use of c_longdouble in an initial RFC would be debatable since it would mean a possible 80bit float that's otherwise not available - but something is still kind of needed for FFI. Maybe c_longdouble could only be available on targets where it cleanly maps to f64 or f128?

I think somebody just needs to take a stab at writing an RFC and trim out features that can't reach consensus. I haven't seen anybody arguing things like "rust should not have a f128/f16 type at least on platforms that support real f128/f16", so it's just a matter of structuring the details into something better than a long github issue discussion :)

You've been super active on this issue for 4 years @aaronfranke - do you maybe want to write it up? Should be a pretty easy RFC. Just fill out this template https://github.com/rust-lang/rfcs/blob/master/0000-template.md and create a PR to that repo (no pressure of course)

tgross35 commented 1 year ago

~~Unfortunate note - I think the LLVM bug that causes this https://github.com/rust-lang/rust/issues/54341 might be applicable here too. But this doesn't change anything with respect to next steps~~ edit: I take this back - seems like this might actually not be a problem because it seems to only affect llvm integers, based on the LLVM patch submissions

ecnelises commented 1 year ago

Some questions/notes regarding float128 (also applied to float16):

Which 'long' float type should be supported?

According to LLVM, there're three float types longer than double in IR: fp128, ppc_fp128, x86_fp80. Only the first is IEEE-comformant, while ppc_fp128 is legacy format and only available on PowerPC (see my answer on StackOverflow), and x86_fp80 is also non-standard and only available on x86.

__float128 is available on i386, x86_64, IA-64, and hppa HP-UX, as well as on PowerPC GNU/Linux targets that enable the vector scalar (VSX) instruction set. __float128 supports the 128-bit floating type. On i386, x86_64, PowerPC, and IA-64 other than HP-UX, __float128 is an alias for _Float128. On hppa and IA-64 HP-UX, __float128 is an alias for long double.

__float80 is available on the i386, x86_64, and IA-64 targets, and supports the 80-bit (XFmode) floating type. It is an alias for the type name _Float64x on these targets.

__ibm128 is available on PowerPC targets, and provides access to the IBM extended double format which is the current format used for long double. When long double transitions to __float128 on PowerPC in the future, __ibm128 will remain for use in conversions between the two types.

— GCC Document 'Floating Types'

Although many targets still does not support fp128, we should use this one as primitive f128 type to avoid ambiguity, and leave the rest two into arch or other target related parts.

Which targets should support `f128`

Very few architectures support hardware instructions for IEEE float128 (I can only recall PowerPC after Power9?), but compiler-rt provides a complete set for lowering these float operations, as long as the target ABI accepts such 'type' (like storing it in 128-bit vector register). Not only by architecture, but also by OS, vendor, or ABI.

In clang, not every target with such width type can accept __float128, if forcing to emit fp128 in IR, backend may crash and interop with C may be broken. So let every target decide it.

// clang f128.c -S -emit-llvm -O -o - -mfloat128 -target <YOUR_TARGET>
__float128 foo(__float128 a, __float128 b) { return a + b; }

And yes, we need something like #[cfg(target_has_f128)] to guard it.

Inter-op with C

We need to define separate types respective to __float128 _Float128 long double in libc. (their meaning may vary between targets!)

It should be specially noted that definition of long double is really a mess. In clang/GCC, options can be used to control semantics of long double, like -mlong-double-64 and -mabi=ieeelongdouble (on PowerPC). We need to use or create cfg directives to differentiate different case, carefully.

Backend support

I believe LLVM and GCC support the additional types well. But cranelift does not support them: https://github.com/bytecodealliance/wasmtime/blob/main/cranelift/docs/ir.md#floating-point-types . Will this be an issue?

I'd like to write such an RFC as my first one if no other volunteer raises hands. :)

tgross35 commented 1 year ago

Although many targets still does not support fp128, we should use this one as primitive f128 type to avoid ambiguity, and leave the rest two into arch or other target related parts.

...

but compiler-rt provides a complete set for lowering these float operations, as long as the target ABI accepts such 'type' (like storing it in 128-bit vector register). Not only by architecture, but also by OS, vendor, or ABI.

This all sounds totally reasonable 👍

I believe LLVM and GCC support the additional types well. But cranelift does not support them: [link]. Will this be an issue?

I don't think that any sort of decisions like this are typically blocked on cranelift support. I suspect they will quickly add support if they know Rust will soon have f128 and f16, but probably just haven't had a reason to do it until now.

I'd like to write such an RFC as my first one if no other volunteer raises hands. :)

Go for it! It sounds like you certainly have the knowledge to write it. It's a team effort anyway, you can create a draft PR as soon as you have some of it typed up, and everyone can help finish/polish it (link it here whenever you do)

aaronfranke commented 1 year ago

Which 'long' float type should be supported?

f128 would be IEEE quadruple-precision only. The behavior MUST be standardized and consistent between architectures. It should NOT use 80-bit floats or double-doubles ever.

Very few architectures support hardware instructions for IEEE float128

RISC-V has native hardware support for IEEE quadruple-precision 128-bit floats via the Q extension. Most other CPU architectures will need to use software emulation; I would look into how C/C++ _Float128 does it.

It should be specially noted that definition of long double is really a mess. ... We need to use or create cfg directives to differentiate different case, carefully.

There is no configuration, the rule should be that we forget about long double. As mentioned above it's not a useful type because it's not consistent across target platforms. It's out-of-scope for a f128 RFC proposal.

I'd like to write such an RFC as my first one if no other volunteer raises hands. :)

That would be great! (also small note, my text above isn't disagreeing with you, just clarifying)

thomcc commented 1 year ago

Note that compiler-rt isn't always available, so we'd still have to port implementations of these for compiler-builtins. It would be good to avoid more situations where this is assumed, which causes significant pain.

lygstate commented 1 year ago

Which 'long' float type should be supported?

f128 would be IEEE quadruple-precision only. The behavior MUST be standardized and consistent between architectures. It should NOT use 80-bit floats or double-doubles ever.

Very few architectures support hardware instructions for IEEE float128

RISC-V has native hardware support for IEEE quadruple-precision 128-bit floats via the Q extension. Most other CPU architectures will need to use software emulation; I would look into how C/C++ _Float128 does it.

It should be specially noted that definition of long double is really a mess. ... We need to use or create cfg directives to differentiate different case, carefully.

There is no configuration, the rule should be that we forget about long double. As mentioned above it's not a useful type because it's not consistent across target platforms. It's out-of-scope for a f128 RFC proposal.

I'd like to write such an RFC as my first one if no other volunteer raises hands. :)

That would be great! (also small note, my text above isn't disagreeing with you, just clarifying)

long double is for ABI compat with libc only. Maybe we only neede is the convert function between long double and f128 and implement full functional in f128, I think that's would be enough. Even though have performance hurt on x86 or ppc double double, but that at least we won't have linkage error

ecnelises commented 1 year ago

Hi there, here is my initial draft for the planned RFC: https://github.com/ecnelises/rust-rfcs/blob/additional-floats/text/0000-additional-float-types.md

I know the author has chance to revise it before the decision period, but I’d like to gather some basic comments before it goes into a real RFC, since this is my first attempt to write a Rust RFC. Thanks!

tgross35 commented 1 year ago

@ecnelises are you able to open a PR to the RFC repo? It's easier to provide feedback that way.

Quick review:

The reference-level explanation section could use more detail on e.g. C interop and architecture support. Specifically things like "f128 will be available on architectures that have hardware support or can emulate true IEEE-xxx 128-bit floats without using 80-bit extended precision, including: ...". I'd also break up the "reference-level explanation" section into f16, f64, and ffi components to help keep things organized.
Specifying core::fii::c_longdouble (only when it's exactly f64 or f128) is useful - these varying types are exactly what core::ffi is for.
I'm not sure whether any support of f80 type is desirable: that is, does Rust really gain much from adding it as part of the minimum viable product? Imho this could go under "future possibilities"
Things like the From/TryFrom implementations are often easier to understand as a code snip rather than paragraph form, e.g.

impl From<f16> for f128 { /* ... */ }
impl From<f32> for f128 { /* ... */ }
impl From<f64> for f128 { /* ... */ }
impl From<i8> for f128 { /* ... */ }
// ...

But again, easier to shape these things once you open a PR. Thanks for putting it together!

aaronfranke commented 1 year ago

@ecnelises Looks nice to me overall, I did spot one grammar mistake:

Implementing in Rust compiler help to maintain a stable codegen interface. To fix: help -> helps

Also I agree with @tgross35 on mentioning architectures, for example we can mention RISC-V which has support for 128-bit quadruple-precision floats via the Q extension (without the Q extension, emulation would be required).

For the discussion of what to put in this RFC or what to leave out (@tgross35 mentioned maybe leaving out f80 but including core::fii::c_longdouble), I'm not sure, but it is indeed an important discussion. I would tend towards prioritizing IEEE standards over language-specific and architecture-specific formats.

For the drawbacks section, note that not all architectures support f32 and f64 natively either. For example RISC-V without the F or D extensions does not support those formats respectively. So however Rust handles f32 and f64 on that architecture will be very similar to how it will handle f128 without the Q extension.

tgross35 commented 1 year ago

@ecnelises would you mind opening a PR to this repo with your draft? You can create it as a draft PR to indicate that it isn't ready for final review.

Even if you haven't had the time to work on it (completely understandable), I think it would be good for everyone to start thinking about it, and to provide reviews with suggested changes.

ecnelises commented 1 year ago

@ecnelises would you mind opening a PR to this repo with your draft? You can create it as a draft PR to indicate that it isn't ready for final review.

Even if you haven't had the time to work on it (completely understandable), I think it would be good for everyone to start thinking about it, and to provide reviews with suggested changes.

Sorry for the delay! I've created https://github.com/rust-lang/rfcs/pull/3451 . Although I still have some changes not finished, I'd be glad to see more comments and revise it.

rust-lang / rfcs