search

tgrosinger / md-advanced-tables

A text editor independent library to enable formatting and Excel-style navigation, and spreadsheet formulas to Markdown tables.
MIT License
151 stars 30 forks source link

Floating point error #46

Closed henrebotha closed 1 year ago

henrebotha commented 2 years ago

A simple sum across decimal numbers, as in the below example, demonstrates classic floating-point error.

| Num                 |
| ------------------- |
| 0.1                 |
| 0.2                 |
| 0.30000000000000004 |
<!-- TBLFM: @>$1=sum(@I..@-1) -->

I am aware that formatting directives can be used to prevent this, but I wonder if a decimal library would be helpful. I don't think any user of this plugin wants to see a decimal representation of a binary value, but rather a proper decimal value.

ebdavison commented 2 years ago

I have a similar issue with decimals and would love to see a proper resolution. In my case, the result should be 0.

| date       | total                  |
| ---------- | ---------------------- |
| 2022-01-17 | 303.09                 |
| 2022-01-17 | -140.09                |
| 2022-01-17 | -25                    |
| 2022-01-17 | 0                      |
| 2022-01-28 | -20                    |
| 2022-02-04 | -20                    |
| 2022-02-04 | 0                      |
| 2022-04-20 | -8                     |
| 2022-04-20 | -90                    |
|            | -2.842170943040401e-14 |
<!-- TBLFM: @>$2=sum(@I..@-1) -->
tgrosinger commented 1 year ago

There are some suggestions for resolving this here: https://github.com/tgrosinger/advanced-tables-obsidian/issues/220#issuecomment-1348928275