search

valdezt / sigfig

A python module aimed to aid the user with visual display of scientific numbers, i.e. significant figures, rounding, spacing, etc.
12 stars 4 forks source link

Problem with round_unc function when changing decimal places #1

Open bannanc opened 8 years ago

bannanc commented 8 years ago

Hello @drakegroup ,

My group recently started using your module. However we've found a problem with the output of sigfig.round_unc, specifically it seems to mishandle decimal placement when a rounded number should increase one decimal place. Below are some examples we found where the rounding works or doesn't work:

sigfig.round_unc (0.99626791663, 0.1) '0.10' sigfig.round_unc (1.0, 0.1) '1.0'

correct as soon as we hit 1

sigfig.round_unc (0.99626791663, 0.5) '0.10' sigfig.round_unc (0.99626791663, 0.9) '0.10' sigfig.round_unc (0.99626791663, 1.0) '1'

Correct if uncertainty gets bigger

sigfig.round_unc (0.99626791663, 1) '1'

Correct if uncertainty is bigger even if it's not a float

sigfig.round_unc (0.099626791663, 0.01) '0.010'

Value and uncertainty 10x smaller is still problematic

sigfig.round_unc (0.99626791663, 0.1) '0.10' sigfig.round_unc (9.9626791663, 1.0) '10'

This is correct