Open chloeskt opened 1 year ago
1 replace n_ref_tokens
with n_tokens
:
ys = self.create_ys(normalized_cdf, n_tokens).unsqueeze(
dim=2
)
2 replace N
with n_tokens
:
unique_indices = self.get_unique_indices(indices=tokens_to_pick_ind, max_value=n_tokens - 1)[:, : n_tokens - 1]
Thanks for your answer !
Hi, @chloeskt . I replaced the code in ats_block.py as @cuguniang's comment. Though the code could run without any errors, I still could not reproduce the results given in Fig 5 (c).
I ran the testing with batchsize = 1, 1024, and NUM_TOKENS= 1.0, 0.87, 171 (for NUM_TOKENS <= 1.0, it would be view as ratio). And the results are: batchsize=1, NUM_TOKENS=1.0, top1_acc = 77.05% batchsize=1, NUM_TOKENS=171, top1_acc = 74.52% batchsize=1024, NUM_TOKENS=1.0, top1_acc = 78.34% batchsize=1024, NUM_TOKENS=0.87, top1_acc = 58.75%
Have you sucessfully reproduced the result? Thank you.
Hello,
I am currently trying to reproduce the results given in Fig 5 (b) and (c), in "not finetuned" mode.
Here is my conf for the GFLOPs level of 3, Stage 3 not finetuned:
And here is my conf for the GFLOPs level of 3, Multi-stage not finetuned:
However I am not able to reach the Top1-Accuracy you indicate in these figures. Could you please provide the config files leading to the creation of Fig5 (b) and (c) please ?
Thank you in advance !