footprint memory cost! - Githubissues

facebookresearch / kill-the-bits

Code for: "And the bit goes down: Revisiting the quantization of neural networks"

Other

636 stars 123 forks source link

footprint memory cost! #21

Closed rafikg closed 5 years ago

rafikg commented 5 years ago

The example given in 4.1(metrics), I think the indexing cost is not well calculated: cost for save the centroid: 9 16(float16)256 / 1024 =4.5kB cost for indexing : 16bits 128(cin subvectors) 128(cout vector) / 1024 =32KB can you confirm?

pierrestock commented 5 years ago

Hey DeeperDeeper,

Thanks for reaching out! The example given in 4.1 is correct.

You got the cost of storing the centroids in fp16 right, it is 4.5 KB in this example
For indexing, we index indeed 128 * 128 vectors. Since we are using 256 centroids, our index is any integer between 0 and 255, thus it can be stored on 1 byte (= 8 bits) (see PyTorch's ByteTensor here). Thus the cost is 16 KB.

Hope this helps,

Pierre