ext2str (fpstr) has multiple issues with exponent part

[tingtron]

Summary of issues:

1. When Exp ends with "0" it should not be truncated. E.g. 1.23E+10 should not be returned as 1.23E+1
2. In Exp form, the number of fractional digits should not exceed the fractional digits parameter (observed: it is increased by 1)
3. Last digit before E should be properly rounded. (Observed: it is simply truncated. E.g. 1234567 with frac=5 becomes 1.23456E+06 instead of 1.23457E+06.
4. When a small number is less than 10^-f (fractional digits), it should automatically switch to exp. form (observed: with frac=4 0.00001 returns 0.0000 instead of 1.0000E-05.
5. If the exp. part causes the display width (int+frac+1) to increase, it should reduce the frac. digits. (e.g. with int=6, frac=9, the number 123456789.123E+200 should return 1.23456789E+208 [16 digits] instead of 1.2345678912E+208 [18 digits])
6. If the integer part fits within the display width (int+frac+1), it should not use the Exp. form, even if it will result in fewer fractional digits than frac. parameter. This is because the exp. suffix (E+xx) will take up space and reduce the overall number of significant digits displayed.

Using FPPOW.PLA with different variations of ext2str and multipliers:

ext2str(@xT,@sT,12,6,1) // don't strip trailing zeros (1=fixed count of fractionals)

    Max. Disp. Digits is 19 = 12 + 6 + 1  (12 includes sign)

Issues:

begin: 123456789.123456789[19]
x 1/1 => 123456789.123457[17]
x100/1 => 12345678912.345679[19]
x100/1 => 1.2345678E+12[14]   (*) expected 6 (not 7) fractional digits
...
x100/1 => 1.2345678E+2[13]    (*) expected "E+20"
x100/1 => 1.2345678E+22[14]
...
begin: 123456789.123456789[19]
x 1/100=> 1234567.891235[15]
x 1/100=> 12345.678912[13]
x 1/100=> 123.456789[11]
x 1/100=> 1.234568[9]
x 1/100=> 0.012346[9]
x 1/100=> 0.000123[9]
x 1/100=> 0.000001[9]
x 1/100=> 0.000000[9] (*) expected neg.exp.
...

ext2str(@xT,@sT,9,6,1) // don't strip trailing zeros (1=fixed count of fractionals)

    Max. Disp. Digits is 16 = 9 + 6 + 1  (9 includes sign)

begin: 123456789.123456789[19]
x 1/1 => 1.2345678E+08[14] - OK: 9 int digits + sign = 10, so exp.
x10/1 => 1.2345678E+09[14] (*) but expected 6 (not 7) fractional digits
x10/1 => 1.2345678E+1[13]  (*) expected "E+10"
x10/1 => 1.2345678E+11[14]
x10/1 => 1.2345678E+12[14]
...
x 1/10=> 12345678.912346[16] - OK: 8 int + sign = 9, so no exp.
x 1/10=> 1234567.891235[15]
x 1/10=> 123456.789123[14]
x 1/10=> 12345.678912[13]
x 1/10=> 1234.567891[12]
x 1/10=> 123.456789[11]
x 1/10=> 12.345679[10]
x 1/10=> 1.234568[9]
x 1/10=> 0.123457[9]
x 1/10=> 0.012346[9]

ext2str(@xT,@sT,6,9,1) // don't strip trailing zeros (1=fixed count of fractionals)

    Max. Disp. Digits is 16 = 6 + 9 + 1  (9 includes sign)

begin: 123456789.123456789[19]
x 1/1 => 1.2345678912E+08[17]     (*) expected 6 (not 7) fractional digits
x32000/1 => 3.9506172519E+12[17]  (*) ... so here we also exceed out planned 16 Disp. digits
x32000/1 => 1.2641975206E+17[17]
x32000/1 => 4.0454320659E+21[17]
x32000/1 => 1.2945382611E+26[17]
x32000/1 => 4.1425224355E+3[16]  (*) expected "E+30"
...
x 1/2 => 6.1728394561E+07[17]     (*) also expected 6 (not 7) fractional digits
x 1/2 => 3.0864197280E+07[17]
...

begin: 123456789.123456789E+80[23]      
x 1/1 => 1.2345678912E+88[17]       (*) expected 9 (not 10) fractional digits    
x32000/1 => 3.9506172519E+92[17]    
x32000/1 => 1.2641975206E+97[17]        
x32000/1 => 4.0454320659E+101[18]   (*) also when exp digits increase,       
x32000/1 => 1.2945382611E+106[18]       it should reduce fractional digits
x32000/1 => 4.1425224355E+11[17]    (*) exptected "E+110"    
x32000/1 => 1.3256071793E+115[18]       
...

To compare, some rules observed from HP-42s (using Free42):

1) Prioritize Int part over fixed decimals (if Int part fits into Display digits, truncate the fractional digits)

2) Use positive Exp. when Int. part won't fit in Display digits.

3) Use negative Exp. when the value is less than 10^(-FracDigits), if otherwise all frac digits would be zero

4) In Exp. form, the number of decimal digits is observed.

5) In Exp. form, limit Decimal digits <= Disp.digits - Exp digits - 4 (=sign + period + "E+") (This is not an issue in 42s because it has max Fix 11 and 16 Display digits)

123,456,789.123456789
                  FIX 06
  123,456,789.123    *** (1)
     1,000.000000      /
   123,456.789123    ***
     1,000.000000      /
       123.456789    ***
     1,000.000000      /
         0.123457    ***
     1,000.000000      /
         0.000123    ***
        10.000000      /
         0.000012    ***
        10.000000      /
         0.000001    *** (3)
        10.000000      /
      1.234568E-7    *** (3)
        10.000000      /
      1.234568E-8    *** (4)
            1E200      /
    1.234568E-208    *** (4)
           1E1000      /
   1.234568E-1208    *** (4)
123,456,789.123456789
                   ENTER
        10.000000      x
 1,234,567,891.23    *** (1)
        10.000000      x
 12,345,678,912.3    *** (1)
        10.000000      x
 123,456,789,123.    *** (1)
        10.000000      x
      1.234568E12    *** (2)
        10.000000      x
      1.234568E13    ***
        10.000000      x
      1.234568E14    ***
     1,000.000000      x
      1.234568E17    ***
            1E100      x
     1.234568E117    *** (4)
           1E1000      x
    1.234568E1117    *** (4)
                  FIX 04
      1.2346E1117    ***
           1E1000      /
       1.2346E117    ***
            1E100      /
        1.2346E17    ***
             1E14      /
       1,234.5679    ***
          10.0000      x
      12,345.6789    ***
         100.0000      x
   1,234,567.8912    ***
          10.0000      x
  12,345,678.9123    ***
          10.0000      x
  123,456,789.123    ***
          10.0000      x
 1,234,567,891.23    ***
          10.0000      x
 12,345,678,912.3    ***
          10.0000      x
 123,456,789,123.    *** (2)
          10.0000      x
        1.2346E12    *** (2)
          10.0000      x
        1.2346E13    ***

[dschmenk]

Once again, great test cases! Pretty printing floating point numbers is surprisingly challenging. These test cases should help get it sorted. I'll include this test in the next release, too.

[tingtron]

Yes, in many cases the particular formatting approach depends on the goal. I vaguely recall working with sprintf and crafting the format strings for it and then adding or removing trailing zeros manually -- for a particular consistent look within different ranges in a visualization or a report.

In Apple Numerics 2nd ed, they showed Dec2Str. Interesting how that would look like and if there are sources for those routines. I haven't found yet. (Also don't know if the ORCA sources have it.)

In the case of RPN calculator, the requirements are to fit within the display width; maximize visible significant digits and follow a selected fixed number of decimals.

One question how is the intdigits supposed to be used? And is it more useful than a displayWidth parameter instead?

In the approach in ext2str (fpstr) is reasonable by using and adapting Bin2Dec (aka Ext2Dec or Num2Dec). One problem I see is using the DecForm.style=1 (Fixed) even for a "fixed" decimal format in calculator fashion -- without losing small decimals, and when dealing with large exponents and many significant digits.

So I am thinking it makes sense to always use DecForm.style=0 (Float), and then based on exponent and display width decide whether to use the Exp. format or keep non-Exp. format (unless explicit FPSTR_EXP is requested).

In addition, in case non-exp. choice above, in case of FPSTR_FLOAT, all fractionals would be kept, whereas for FPSTR_FIXED, the fractionals would be either rounded or padded with zeors (depending on FPSTR_STRIP). For FPSTR_FIXED this fractional dressing could be done manually or by invoking Bin2Dec a second time, but with DecForm.style=1 (Fixed).

The reason for this two-stage processing and initial Bin2Dec call with DecForm.style=0 (Float) is to obtain the Exp. to make the above decisions. Alternatively, the Exp. from the ext representation could be accessed directly -- not sure if there's a SANE call for that or could be just read from memory (but it's not very trivial, and may have to deal with de-normalized representation,)

This example FPSANE.PLA shows which sizes of exp are candidates for float and exp representation. The output here is:

• <input string>
• [len] <sig> e<exp> dp=<decimal pos.>

/PLASMA/BLD/:+fpsane                    
Float Format: disp.width=16 frac=6      

....|....|....|....|....|....|....|.... 
123456789.1234E+1000                    
     [14] 12345678912340 e995 dp=1009   
1.123456789012E+1000                    
     [14] 11234567890120 e987 dp=1001   
123456789.12345E+100                    
     [14] 12345678912345 e95  dp=109    
123456789.123456E+10                    
     [14] 12345678912346 e5   dp=19     
12345678901234567890                    
     [14] 12345678901235 e6   dp=20     
1234567890.123456789                    
     [14] 12345678901235 e-4  dp=10     
12345.12345678901234                    
     [14] 12345123456789 e-9  dp=5      
0.000000000123456789                    
     [14] 12345678900000 e-32 dp=-18    
0.12345678901234E-10                    
     [14] 12345678901234 e-24 dp=-10    

[tingtron]

Another issue, but in str2ext, when parsing: Numbers in the form 0.0dddd get their exponent doubled. So

• 0.01 -> 0.001
• 0.001 -> 0.00001
• 0.0001 -> 0.0000001

This can be seen in the RPN Calc:

[dschmenk]

I've made wholesale updates tp FPSTR to address most of the issues you brought up. Rounding will be up to the SANE library, but everything else should be working. I decided to add an additional flag to ext2str to switch to exponential form if overflow/underflow occurs with the FPSTR_FLOAT or FPSTR_FIXED flags. However, I could revert to the case where it is done automatically in the over/underflow cases. Just wasn't sure the best approach would be. I think the additional flag is more flexible (and the flag is called FPSTR_FLEX) to give the option.

I've updated the plasma.2mg image with the latest if you want to check it out.