search

FoundationVision / LlamaGen

Autoregressive Model Beats Diffusion: 🦙 Llama for Scalable Image Generation
https://arxiv.org/abs/2406.06525
MIT License
1.33k stars 55 forks source link

Issues about the 3B model #7

Open Con6924 opened 5 months ago

Con6924 commented 5 months ago

Thanks for your fascinating work!

I'm now trying on the 3B model and encountered two issues:

  1. The json of 3B model is missing. I tried to modify from the json of the XXL version to match the checkpoint and statistics in the paper, but meet another issue;
  2. ValueError: Head size 100 is not supported by PagedAttention. Supported head sizes are: [64, 80, 96, 112, 128, 256]. from xformers.
ShoufaChen commented 5 months ago

Hello, @Con6924

This issue has been fixed in https://github.com/FoundationVision/LlamaGen/pull/23. Please have a try with main branch.