Closed mhulse closed 11 years ago
mhulse
commented
11 years ago Based on conversations I've had with a couple of people, it seems the consensus is that the last decimal should round up.
I don't think this rounding should affect any browsers in terms of computed pixel values.
My personal opinion? The number should truncate and not round.
For now, I can live with the rounding.
mhulse
commented
11 years ago For future reference:
it should round to 1.66...67. Having said that, what is far more important is what happens when you multiply either answer by 6. In both instances the answer should be 7.
... more precisely what I am saying is that:
... [picking the number] 6 was not random. I picked 6 because 1/6 = 0.1666666... where the 6 repeats FOREVER. So you want to make sure the calculating device treats 0.16666...666 or 0.16666...667 appropriately, that is, as 1/6. Multiplying 1/6 (or in your question 7/6) should result in 1 (or 7).
Multiply your original problem by 6 (not the 14 digit answer to the problem -- which is an approximating to the correct answer because the correct answer has infinite 6's) and hopefully you will get 7 on both devices.
Big thanks goes out to Paul H. for the math help! :+1:
My precision is set to
14; why does sass round up that last decimal place?For example:
This math on my calculator app:
(1.5 / 1.125) * .875 = 1.16666666666666
When using SCSS to calculate the above formula, I get:
1.16666666666667
Which result is more accurate? In my opinion, the last decimal place should not round (on repeating decimal) up unless I've asked for it to do so.
Possible solutions: 1) Query the SASS peeps/community to understand why it does this and if there is a fix. 2) Use grunt to strip off the last decimal.
Note to self: Reasoning for 14 decimal places is that I've found that Webkit needs at least 7 decimal places to meet my requirements for optimal baseline alignment.