Adds a builtin decimal type (FixedPoint)

[03a875e3-f930-4515-877c-2a0b50d291b6]

BPO	653938
Nosy	@loewis, @brettcannon, @rhettinger

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = created_at = labels = ['interpreter-core'] title = 'Adds a builtin decimal type (FixedPoint)' updated_at = user = 'https://bugs.python.org/mclay' ``` bugs.python.org fields: ```python activity = actor = 'brett.cannon' assignee = 'none' closed = True closed_date = None closer = None components = ['Interpreter Core'] creation = creator = 'mclay' dependencies = [] files = [] hgrepos = [] issue_num = 653938 keywords = ['patch'] message_count = 5.0 messages = ['42000', '42001', '42002', '42003', '42004'] nosy_count = 4.0 nosy_names = ['loewis', 'brett.cannon', 'rhettinger', 'mclay'] pr_nums = [] priority = 'normal' resolution = 'rejected' stage = None status = 'closed' superseder = None type = None url = 'https://bugs.python.org/issue653938' versions = ['Python 2.3'] ```

[03a875e3-f930-4515-877c-2a0b50d291b6]

This patch integrates the fixedpoint module, that was created by Tim Peters, into Python as a first class object. That is, the patch adds this new number type so that it has direct syntax support like float, int, long, str., etc. within the interpreter.

I use Tim's module to implement the type. This limits the patch to a small bit of code that adds the a syntax interface to this module. Syntax recognition for the new number format required a change to Parser/tokenizer.c and to Python/compile.c. This patch allows the new decimal type (I renamed the FixedPoint type to decimal in the fixedpoint.py file because the name is shorter and it is sufficient to distinguish the type from a binary float.) to be entered as a literal that is similar to a long, int, or float. The new syntax works as follows:

>>> 12.00d
12.00d
>>> .012d
0.012d
>>> 1d
1.d
>>> str(1.003d)
'1.003'

As you can see from the example, the default precision for decimal literals are determined by the precision entered expressed in the literal.

The patch also adds glue to Python/bltinmodule.c to create a builtin decimal() function. This builtin decimal (Formally FixedPoint) function invokes the constructor of the Python implementation of the decimal class defined in the fixedpoint module. The implementation follows the familiar pattern for adding special builtin functions, much like the implementation of apply and abs. It does not follow the typical method for adding built in types. That would require a more invasive patch. The builtin decimal function works as follows:

>>> decimal(1.333)
1.33d
>>> decimal(1.333,4)
1.3330d
>>> decimal(400,2)
400.00d

The semantics of the precision parameter may be incorrect because it is only addressing the precision of the fraction part of the number. Shouldn't it reflect the number of significant digits? If that were the case then the following would have been the result:

>>> decimal(400,2)
4.0e2d

This problem is more noticable when exponents are large relative to the number of significant digits. For instance:

>>> 40.e3d
40000.d

The result should have been 40.e3d, not 40000. This implies more precision than is declared by the constant.

As it currently is implemented the type pads with zeros on the integer side of the decimal point, whic implies more accuracy in the number than is true. I ran into this problem when I tried to implement the automatic conversion of string number values to decimal representation for numbers in 'e' format.

>>> 3.03e5d
303000.00d

The representation is incorrect. It should have returned 3.03e5d but it padded the result with bogus zeros.

For this first cut at a new decimal type I am primarily interested in investigating the semantics of the new native decimal number type in Python. This type will be useful to bankers, accountants, and newbies. The backend implementation could be replaced with a C implementation without affecting the Python language semantics.

The approach used in this patch makes it very easy to experiment with changes to the semantics because all of the experimentation can be done by changing the code in Lib/fixedpoint.py. The compiled wrapper doesn't have to be modified and recompiled.

Unit testing so far just reuses the _test function in the fixedpoint.py module. Since I am not sure which way to go with the semantic interpretation of precision I decided to post the patch before making a significant change to fixedpoint.py. Some feedback on the interpretation of precision would be appreciated.

Documentation:

The following needs to be added to section 2.1 of the library reference manual:

decimal(value, precision)
    Convert a string or a number to fixed point decimal
number. If the argument is a string, it must contain a
possibly signed decimal or floating point number,
possibly embedded in whitespace. The precision
parameter will be ignored on string values. The
precision will be set based on the number of
significant digits. Otherwise, the argument may be a
plain or long integer or a floating point number, and a
decimal number with the same value ( the precision
attribute will set the precision of the fraction
roundoff) is returned.

Section 3.1.1 of the tutorial will add a brief description of decimal number usage after the description of floating point:

There is full support for floating point; operators with mixed type operands convert the integer operand to floating point:

>>> 3 * 3.75 / 1.5
7.5
>>> 7.0 / 2
3.5

Python also supports fixed point decimal numbers. These numbers do not suffer from the somewhat random roundoff errors that can occur with binary floating point numbers. A decimal number is created by adding a 'd' or 'D' suffix to a number:

>>> 3.3d + 3d
6.3d
>>> 3.3d + 3.03
6.3d
>>> 3.3d + decimal(3.03,3)
6.330d
>>> decimal(1.1, 16)
1.1000000000000001d
>>>
The builtin decimal constructor provides a means for
converting float and int types to decimal types. Since
floats are approximations of decimal floating point
numbers there are often roundoff errors introduced by
using floats.  (The 1 in the last place of the
conversion of 1.1 with 16 digits is a binary round off
error.)  For this reason the decimal function requires
the specification of precision when converting a float
to a decimal. This allows the significant digits of the
number to be specified. (Accountants and bankers love
this because it allows them to balance the books
without pennies being lost due to the use of binary
numbers.)

>>> 3.3d/2
1.6d

Note that in the example above the expression 3.3d/2 returned 1.6d. The rounding scheme for Python decimals uses "banking" rounding rules. With floating point numbers the result would have been as follows:

>>> 3.3/2
1.6499999999999999

So as you can see the banking rules round off the .04999999999 portion of the number and calls it a an even 1.6.

[61337411-43fc-4a9c-b8d5-4060aede66d0]

Logged In: YES user_id=21627

There's no uploaded file! You have to check the checkbox labeled "Check to Upload & Attach File" when you upload a file.

Please try again.

(This is a SourceForge annoyance that we can do nothing about. :-( )

[rhettinger]

Logged In: YES user_id=80475

I don't think a syntax change is warranted. Most use cases I can think of do not have decimal constants in the program; rather, they are always data. FixedPoint is valuable in and of itself without any need for a new syntax.

[brettcannon]

Logged In: YES user_id=357491

Is this patch worth keeping open now that we have nondist/ sandbox/decimal/ ? Or are these two implementations going to duke it out in the end?

[brettcannon]

Logged In: YES user_id=357491

With Michael's blessing I am closing this patch as rejected.