search

boricj / numworks-rpn

RPN calculator for NumWorks
Other
36 stars 12 forks source link

Unexpected undef result #26

Closed jgoizueta closed 4 years ago

jgoizueta commented 4 years ago

I've been playing with the RPN App in Omega and I really like it.

But I've also found something weird: This produces an undef result: 1.0000001 x^2 x^2 x^2 x^2 x^2 (enter 1.0000001 and press the square function 5 times). It happens both in exact and approximate modes.

It seems that results are kept in exact form and the integers in the fraction grow too large and become undef.

The Calculation App doesn't have this problem: 1.0000001 EXE ans^2 EXE ans^2 EXE ans^2 EXE ans^2 EXE ans^2 EXE produces 1.000003 (1000003200004560003200001/1E24; so it seems to limit the carried precision)

I have mixed feelings with this problem: I like the idea of RPN maintaining internal exact results always, but as this example shows, that can get out of hand quickly. In this example, since I entered the number as a decimal (and specially If I'm using approximate mode) I would expect an possibly inexact answer, but not an undefined value.

I think precision should be limited at some point like in the Calculation App, maybe only in the approximate mode. In this example the result is OK while numerator and denominator have 113 decimal digits. The operation that fails would have produced integers with 225 digits, so I guess the limit lies somewhere in between.

boricj commented 4 years ago

I need to look into it, but your analysis is sound and is in line with RPN's current behavior. I think an easy fix would be to store an approximation in the internal stack if the exact result does not fit in the statically-allocated buffers.

boricj commented 4 years ago

So I've looked into it and the problem is a bit trickier than that. In the RPN app, at the fourth squaring Poincare tries to write out an exact representation of the fraction. The numerator fits in the buffer and is fully serialized (as with the /), but the denominator doesn't in what's left of it and the denominator gets serialized as undef. The fifth squaring results in undef because undef is the NaN of Poincare.

The calculation app doesn't hit this problem because merely typing x^2 inputs ans^2, which takes the approximation of the last result, not the exact result. Repeatedly selecting the exact representation in the results to place it in the input box and squaring that will end up producing the approximation of the result when the representation gets too large. Forcing the calculation app to keep squaring the exact representation beyond that tends to crash the simulator when scrolling around.

A somewhat dirty solution is to approximate the result whenever it contains the substring undef after simplification. It keeps computation exact until the stack entry buffer would overflow.

jgoizueta commented 4 years ago

Thanks for looking into that, I see there's no an obvious and clean way to fix it.

boricj commented 4 years ago

I did fix it with 1b5f6c3b34601450724c6e6d4f52f0f940f5cd13. It's just that it's not a very elegant way of fixing this problem.

Also, please note that you're using the Omega fork of this RPN app (https://github.com/Omega-Numworks/Omega-RPN). Therefore, the fix must be ported to that fork before it'll end up in Omega.

quentinguidee commented 4 years ago

@jgoizueta This is now fixed on the omega-dev branch and will be publicly available with Omega 1.20 😉 (https://github.com/Omega-Numworks/Omega/commit/482c7df15d34b2b74d0c7c3a04812c3255ad1991)