Request: BLIP2 memory optimizations?

theovercomer8 / captionr

GIT/BLIP/CLIP Caption tool

MIT License

138 stars 15 forks source link

Request: BLIP2 memory optimizations? #10

Open jim-22 opened 1 year ago

jim-22 commented 1 year ago

This issue on the LAVIS repository https://github.com/salesforce/LAVIS/issues/118 shows someone who managed to fit BLIP2 onto his 24 GB VRAM card. Would it be possible to apply that to this repo?

theovercomer8 commented 1 year ago

You can try out the 8bit branch. It has the 8bit stuff integrated. I think there were a lot of problems though when I had some friends testing it.