CE-Programming / toolchain

Toolchain and libraries for C/C++ programming on the TI-84+ CE calculator series
https://ce-programming.github.io/toolchain/index.html
GNU Lesser General Public License v3.0
511 stars 54 forks source link

Integers rounding mid operation. #454

Closed CoolModder closed 11 months ago

CoolModder commented 11 months ago

In code such as: 10 * ((11 / 10) ^ x) It results in inaccurate numbers due to int rounding in the middle of operations. Was expecting 11 when x = 1 (quantity of item), not 0 (End price), as shown here: image Option for rounding at end of operations? (Clicker game, late game prices could get astronomical, and no need for decimals in prices.)

runer112 commented 11 months ago

There are two reasons why that expression is not performing the calculation you'd like. Both relate to how C/C++'s notation differs from mathematical notation:

  1. ^ is not exponentiation, but is bitwise XOR. To perform exponentiation, #include <math.h> and use the pow function.
  2. Unless you include a decimal point or use scientific notation, numbers are assumed to be of type int. And operations on only integral types don't automatically promote to non-integral types. For the result of an operation to produce a floating-point value, at least one of the operands must be a floating-point value. I'm not sure what the type of x is, but 11 / 10 is int / int, which will produce an int (1). If you really want to dive into what the resulting type of a binary operation will be, check out "Usual arithmetic conversions".

Combining these two, corrected code could be (and don't forget #include <math.h>):

10.0 * (pow(11.0 / 10.0, x))

Note that the following would produce the same compiled code because floating-point types are "dominant" over integer types:

10 * (pow(11.0 / 10, x))

commandblockguy commented 11 months ago

Note that the ^ operator in C isn't exponentiation; instead, it does bitwise XOR.

You'll either need to use floats, or do all of the multiplication first and then all of the division, e.g.

uint64_t result = 10;
for (int i = 0; i < x; i++) result *= 11;
for (int i = 0; i < x; i++) result /= 10;

Note that division by non-powers-of-two is a very slow operation, so it might be better to use floats instead or to hard-code the values.

It's also worth noting that, depending on your definition of "astronomical," you might run into the limits of the int type fairly quickly. On the calculator, sizeof(int) is 3, so the largest value an unsigned integer can hold is 2^24-1 or around 16.8 million. Other data types like uint64_t can hold much larger integers (around 1.8e19), but there aren't printing routines in graphx that directly support them, so you would need to convert them to a string first and then display that.

CoolModder commented 11 months ago

Thank you!