dankamongmen / growlight

notcurses block device manager / system installation tool
https://nick-black.com/dankwiki/index.php/Growlight
GNU General Public License v3.0
85 stars 12 forks source link

Calculation of decimal part in genprefix() is incorrect #16

Closed dankamongmen closed 5 years ago

dankamongmen commented 5 years ago

While working on #10, I looked at this, and was like, "that can't be right":

diff --git a/src/growlight.h b/src/growlight.h
index 1671930..19ecf5e 100644
--- a/src/growlight.h
+++ b/src/growlight.h
@@ -454,8 +454,11 @@ genprefix(uintmax_t val,unsigned decimal,char *buf,size_t bsize,
                dv /= mult;
                val /= decimal;
                if((val % dv) / ((dv + 99) / 100) || omitdec == 0){
                   snprintf(buf,bsize,"%ju.%02ju%c%c",val / dv,(val % dv) / ((dv + 99) / 100),
                                   prefixes[consumed - 1],uprefix);
                }else{
                        snprintf(buf,bsize,"%ju%c%c",val / dv,prefixes[consumed - 1],uprefix);
                }

So I'm clearly trying to get a percentage for the mantissa, but...that's not how it's done, son. And indeed:

│  16.46KiB 23437770719→23437770751 partition table metadata       

Unless there's something else at work here, 23437770751 - 23437770719 + 1 -> 33 sectors. At 512 each, that's 16896 bytes. 16896 % 1024 is 512, which is exactly half a kibibyte. We instead show 16.46, which is indeed 512 / ((1024 + 99) / 100). The correct display is 16.50.

The formula we want, I'm pretty sure, is (val % dv) * 100 / dv. I'm not sure where that other one came from.

dankamongmen commented 5 years ago

As the values get larger, this obviously approaches the correct value more and more closely:

> (23437752319 - 2048 + 1) * 512
12000128139264
> 12000128139264 % (1024 * 1024 * 1024 * 1024)
1005011861504
> (1005011861504 * 100) / (1024 * 1024 * 1024 * 1024)
91.405296325683594
> (1005011861504) / ((1024 * 1024 * 1024 * 1024 + 99) /100)
91.405296317453463

but at low values, it's quite off, and either way it's weird as hell.

dankamongmen commented 5 years ago

Fixed in whatever follows 1.6.0.2.