Decimals and other binary bit sizes

[sirinath]

How does this change with decimals and other binary bit sizes? Can they be added?

[abolz]

Do you mean support for e.g. half-float's or long doubles?

Larger floating-point numbers, i.e. numbers with a greater exponent range than doubles, would require enormously large tables. But I think implementations for half-floats would be quite easy to add.

[sirinath]

Name	Common
binary16	Half precision
binary32	Single precision
binary64	Double precision
binary128	Quadruple precision
binary256	Octuple precision
bfloat16	Brain Floating Point
minifloats
TensorFloat-32
FP16
FP24
FP32
Extended precision formats
decimal32
decimal64
decimal128

Found in:

• https://en.wikipedia.org/wiki/IEEE_754
• https://en.wikipedia.org/wiki/Bfloat16_floating-point_format
• https://en.wikipedia.org/wiki/Minifloat
• https://medium.com/@moocaholic/fp64-fp32-fp16-bfloat16-tf32-and-other-members-of-the-zoo-a1ca7897d407

Can the table be properly generated than hardcoded?

[abolz]

Implementations for the minifloat formats can certainly be added. I think it wouldn't even require too much work (since most of the binary32 implementation can be reused).

The table for binary128 would ~2.5MB large. The table for binary256 would be much larger. (Both tables could probably be compressed, though) So binary80/128/256 are out of scope for this library and will certainly not be implemented. Generating the tables at run-time would require a big-integer implementation, which is also out of scope for this library.

[sirinath]

I am looking to port this to Java. Is it possible to add a float/binary32 implementation?

[abolz]

I'm not sure what you are referring to.

There are already binary32 implementations of Ryu and Schufach. I'm not planning to implement binary32 versions of Grisu2/3 currently. (These are mostly for comparison with the other algorithms.) @jk-jeon has an implementation of Dragonbox which works for 32-bit floats here.

And there are already Java implementations of Ryu and Schubfach.

Note that porting the implementations in this repository to Java is not just a straightforward source code transformation. The implementations here all follow JavaScript's Number.ToString method: The output is always shortest. Java has slightly different requirements, where sometimes more digits than neccessary will have to be emited, and the Java implementations linked above both already do this correctly.

[sirinath]

@jk-jeon version of DragonBox has a lot of meta programming hence it is difficult to translate into Java easily. @jk-jeon himself suggested this would be the better stating point for porting into Java.

I am only interested in Dragonbox.

[sirinath]

I am also interested in simplified yet fast string parsing if this is also possible.

[abolz]

I am also interested in simplified yet fast string parsing if this is also possible.

You may find fast strtod/strtof implementations in ryu_32/64.h. There is also the more recent fast_float library, which you may want to check out.

Since I do not have any time in the near future to work on other floating-point representations, I'm closing this issue. But I'll drop a note here, if/when I get the time to work on this.

[newbie-02]

hello @abolz,
in comment #3 you stated: 'The table for binary128 would ~2.5MB large. The table for binary256 would be much larger. (Both tables could probably be compressed, though) So binary80/128/256 are out of scope for this library and will certainly not be implemented.'
you didn't say how big a table for bin-80 will be, a naive exponential interpolation might assume ~85k, if the trick of ryu to use a shrinked table at the cost of few more calculations is applicable 17k could be practicable?
I think / wish it could be possible to produce a working 80-bit implementation, would like that very much, but lack the skills to code myself ... why I'd like it? it's one of very few approaches to check the correctness of double calculations other than manual evaluation ...

[abolz]

The table would be ~1.6 MB large.

The folder scripts/ryu contains some scripts to compute the table sizes. compute_bit_sizes computes the number of bits required for each constant. Which is 152 bits for binary80, which rounded up the next multiple of 32 gives 160 bits. compute_table_sizes computes the number of entries required for each table. For binary80 ~10000 entries are required.

You can then feed these number into compute_powers which will actually compute the constants in the table.

The size of the tables for the larger floating-point formats is mostly due to the huge exponent range of these formats.

[newbie-02]

hello @abolz,
thank you for your fast reply. still flat part of my learning curve ... the exponential behavior is oriented at the exponent size and that is 15 for 80-bit as well as 128-bit floats? ... :-(
the 'small table' approach of ryu could reduce that to ~300 kB?, still impracticable I think. Will try to dive into once I find some time to try to understand the basic math ... [ edit ] meant kB, not MB [ /edit ]

[abolz]

You're welcome.

But maybe instead of focusing on these large floating-point formats, you may want to focus on the implementation of Ryu for some of the smaller floating-point formats. Like binary16. Or some toy format, which you may invent yourself. The advantage is that you're able to easily test all of the numbers in these formats.

[newbie-02]

hello @abolz,

'small formats' - yes, that may be an interesting field in nearby future, I have heard rumors that KI and big-data are going that direction reg. bandwith usage. Also for other usage small formats will become practical once they deliver less deviating results. I'm working on 'last bit accuracy' which changes ' = 0.6 + 0.3 ' -> 0.90000004 with floats to better behavior as well as ' = 0.2 + 0.1 ' -> 0.30000000000000004 with doubles.
But:

• for the moment I need to continue from the point I am, and that is checking corrected double results with different calculations to prove them right.
• as a somehow viable migration path I have to start from what is currently in use, and that is for spreadsheets 64-bit doubles.

[newbie-02]

hello @abolz,

two questions:

• found the note of Ulf Adams about 'small tables' in d2s.c:
  '// -DRYU_OPTIMIZE_SIZE Use smaller lookup tables. Instead of storing every // required power of 5, only store every 26th entry, and compute // intermediate values with a multiplication. This reduces the lookup table // size by about 10x (only one case, and only double) at the cost of some // performance. Currently requires MSVC intrinsics.' thus I'd expect a factor 26 ( stored values ) or factor 10 ( mentioned ) shrinking -> ~75 or ~160kB, could that be manageable?

• thinking about use of long doubles more for enhanced precision than for extremely small or big values ... would it be practicable to split / triage the long doubles in A.) those inside the double range from -/+( 5E-324 .. 1.7E308 ) and B.) the others outside that ranges, develop a fast algorithm which doesn't need extra exponent capa but only account the better precision of the significant, and call that for ( real life common ) values in ranges A.) while using the slow 'generic' algo as a fallback for ( IMHO rare ) occurrence of B.) figures? Near to try to code it by myself, but would need reasonable amount of help and Ulf Adams is too busy at the moment.

And - if I understood the graph in issue#2 correct that Schubfach is nearly twice as fast as ryu for short strings - it would be interesting to build such as a variant of Schubfach ...