TT embedding provides good a embedding compression ratio. However, I don't know how to control the compression rate by setting the shapes and TT-rank. My goal is to apply tt_embedding for CTR tasks. I have the following two featues and each has been embeded by tt_embedding. But the following setting has led to a large accuracy drop (3%). I wonder if I can only use a small compression rate, e.g., 2x~5x. How can best setting the shapes to achieve a target compression rate? Thanks!
voc_size=820508 emb_size=192
A TT-Matrix of size 900000 x 192, underlying tensorshape: [90, 100, 100] x [4, 6, 8], TT-ranks: [1, 160, 160, 1] on device 'cpu' with compression rate 11.12
voc_size=2903321 emb_size=192
A TT-Matrix of size 3000000 x 192, underlying tensorshape: [125, 150, 160] x [4, 6, 8], TT-ranks: [1, 160, 160, 1] on device 'cpu' with compression rate 24.69
TT embedding provides good a embedding compression ratio. However, I don't know how to control the compression rate by setting the shapes and TT-rank. My goal is to apply tt_embedding for CTR tasks. I have the following two featues and each has been embeded by tt_embedding. But the following setting has led to a large accuracy drop (3%). I wonder if I can only use a small compression rate, e.g., 2x~5x. How can best setting the shapes to achieve a target compression rate? Thanks!
voc_size=820508 emb_size=192 A TT-Matrix of size 900000 x 192, underlying tensorshape: [90, 100, 100] x [4, 6, 8], TT-ranks: [1, 160, 160, 1] on device 'cpu' with compression rate 11.12
voc_size=2903321 emb_size=192 A TT-Matrix of size 3000000 x 192, underlying tensorshape: [125, 150, 160] x [4, 6, 8], TT-ranks: [1, 160, 160, 1] on device 'cpu' with compression rate 24.69