Open overfiter opened 1 year ago
I think it would need 40GB VRAM for the 30b and even more for the 60b.
I disagree on 13b intrinsically being better, I've set up a batch script for captioning Stable Diffusion training and 13b tends to add way more unnecessary info than really is required.
Also.. I think they'd need to change the name if they implemented this 🤣
Is the 7b version faster? How much vram needed for that. Would love to see your version if it's available
7b with low resource off results in processing 1000 images in 1 hour.
With the live demo, interactivity takes a second or two.
vram usage is just over 16gb with low resource off.
With low resource on, it's marginally faster. With it off, you're looking at about a second or two on a 3090ti per query.
Will minigpt4 30b and 60B be released later? The 13b is clearly better than the 7b, so I'm looking forward to the release of the 30b and 65b models