-
Looking at the code, it seems that there are no weights for key, query, and value when implementing self attention. Is this the correct implementation?
-
Hi, I wonder how these figures were obtained.
SVD on self-attention map would produce U, S, and V.T.
How did you obtain the figures??
 WARNING 11-15 15:19:28 gemma2.py:351] Some weights are not initialized from checkpoints: {'layers.18.mlp.gate_up_proj.…
-
Hi,
First thank your very much for your work. It adds a huge improvement to DETR family.
And your paper was really well explained and written.
Also thank you for publishing your code & models, i…
-
Hi, I tried a test about compiling unet(torch.float16), which is the part of StableDiffusionXLPipeline in Inferentia2.8xlarge and it failed.
When the latent size of unet is (64, 64), it did not fai…
-
Traceback (most recent call last):
File "/home/yy/MSSR-main/MSSR-main/run_model.py", line 39, in
run_result = run_recbole(model=args.model, dataset=args.dataset, config_file_list=config_file_…
-
Just realized i get the below warning with Salesforce/blip-image-captioning-large ; i think i already ran results for it, but they're probably random in that case; maybe someone could check the result…
-
Thank you for developing this!
## Context
Due to lenghty computation time and in order to speed things up, I thought about using the `flash_attention_2` and smaller floating points `torch.float16`…