Standardize our usage of data volume units: SI vs. IEC, GB vs. GiB

[marians]

Internal Slack thread

The interchanging use of SI and IEC units like GB and GiB, MB and MiB etc. cause a challenge to our end users, as this forbids exact comparison. To avoid this, we should decide for the preferred unit system to use throughout our docs, user interfaces, and APIs.

We should survey the biggest cloud providers and most important tools in our ecosystem to see if there is alignment already.

This affects

• Documentation
• User interfaces
• CLIs
• CRDs
• App value schemas and templates
• Dashboards

[marians]

Survey

• Azure: IEC (MiB, GiB, TiB, ...)
• AWS: Mixed. It seems they use SI (MB, GB, TB) for block storage (S3 etc.) and IEC for memory (e. g. in instance type details).
• Google Cloud: Mixed. It seems they use SI (MB, GB, TB) in the web UI, but I see IEC (GiB) used in the gsutil CLI.
• GitHub: SI (MB, GB, TB)

[marians]

Proposal A: Use SI units

I propose to use SI units (KB, MB, GB, TB...) for the sake of simplicity. The conversion factor 1000 makes it very easy to calculate the relationship between the different units. Also the units are currently more common to pronounce ("megabyte" vs. "mebibyte").

The cloud providers, with the exception of Azure, seem to use both units, so unfortunately alignment with the ecosystem as a whole isn't possible.

[marians]

Searching through our docs, I found occurrences of IEC units (KiB, MiB, GiB etc.) in these contexts:

• kubectl gs template cluster flags --vsphere-control-plane-memory-mib and --vsphere-worker-memory-mib, --vsphere-control-plane-disk-gib, --vsphere-worker-disk-gib
• kubectl gs template cluster output for CAPV (VSphere): memoryMiB, diskGiB
• Various docs content snippets use GiB
• Various docs code snippets use the format [0-9]+Gi for resource requests/limits
• https://github.com/giantswarm/cluster v0.7.0 has quotaBackendBytesGiB

Within dashboards, I found

• loki-deletion
• persistentvolume
• persistentvolume
• Search for "unit": "bytes" finds panels where the unit is set to "Date / bytes (IEC)". (with SI it would be `"unit": "decbytes".)

[marians]

In yesterday's KaaS sync I learned that it's common to use IEC ("power-of-two") units when the subject is memory. For disk storage, on the other hand, power of 10 is more common. So the picture is more complex than I assumed.

We decided to gather some more input asynchronously before we decide.

I found some interesting posts on Quora which indicate that

• Only hardware manufacturers have traditionally used power-of-ten calculation to claim, for example, that a disk has 100 GB when the power-of two-calculation would in fact have yielded a lower number.
• The IEC has introduced their unit system KiB, MiB, GiG etc. to make it obvious that a power-of-two calculation (factor 1024 instead of 1000) is used.

My conclusion here:

• Technically even hard disk's (which are solid state memory often) sizes should rather be given as a power-of-two unit, as this is the way computers address memory.
• The only way to make it clear and explicit which factor is used (1024 or 1000), is to choose IEC units (KiB, MiB etc.). When using e. g. GB, there is always uncertainty on the reader's side whether the author got the concept right.

To me, this speaks for Microsoft Azure's approach of using IEC units consistently.

So I'm reverting my proposal A and hereby suggest

Proposal B: Use IEC units

I propose to use the units KiB, MiB, GiB, calculated with a conversion factor of 1024, consistently for storage size, memory size, data I/O and all other data volume measures.

[marians]

We have a PDR now here: https://intranet.giantswarm.io/docs/product/pdr/011-iec-units/

The implementation will happen over time, and we don't need to keep this issue open for that. Closing.