caicloud / cyclone

Powerful workflow engine and end-to-end pipeline solutions implemented with native Kubernetes resources. https://cyclone.dev
Apache License 2.0
1.07k stars 168 forks source link

The result of pvc memory usage is not religious #1381

Open zhujian7 opened 4 years ago

zhujian7 commented 4 years ago

Cyclone use du -sh to get memory usage, read the annotation of -h for du command:

-h "Human-readable" output. Use unit suffixes: Byte, Kilobyte, Megabyte, Gigabyte, Terabyte and Petabyte.

Please pay attention to the unit : Kilobyte, Megabyte, Gigabyte, Terabyte and Petabyte. Instead of Kibibyte, Mebibyte, Gibibyte, Tebibyte and Pebibyte.

Unfortunately, we use the following formula to calculate:

var unitMap = map[byte]int64{
    'B': 1,
    'K': 1024,
    'M': 1024 * 1024,
    'G': 1024 * 1024 * 1024,
    'T': 1024 * 1024 * 1024 * 1024,
}

And It is obviously wrong, since 1 Mebibyte=1000Kibibyte (1M=1000K).

There are two ways to fix it:

Related Reads:

/kind bug

What happened:

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?: @supereagle @hyy0322 @qiuxiaolian

supereagle commented 4 years ago

What are the preferred units? Could we convert to these preferred units after 'du -sh'?

zhujian7 commented 4 years ago

A piece of output of man du:

...
       Display values are in units of the first available SIZE from --block-size, and the DU_BLOCK_SIZE, BLOCK_SIZE and BLOCKSIZE environment variables.  Otherwise, units default to 1024  bytes  (or
       512 if POSIXLY_CORRECT is set).

       SIZE is an integer and optional unit (example: 10M is 10*1024*1024).  Units are K, M, G, T, P, E, Z, Y (powers of 1024) or KB, MB, ... (powers of 1000).
...

So I rethought the description of the issue is wrong And It is obviously wrong, since 1 Mebibyte=1000Kibibyte (1M=1000K).

The implementation of command du treats 1M=1024K. So I think the only thing we need to do is change the unit from K M G T to Ki Mi Gi Ti to make them more readable. Besides Kubernetes uses Ki Mi Gi Ti instead of K M G T as memory units: Unit of Memory in Kubernetes