Closed Keeo closed 9 months ago
Please turn off --quant for SPHINX. We are going to support quantized SPHINX in the future.
Thanks, in the meantime is there anything else I can toggle to fit it in 24gb memory?
@Keeo Please refer to the following issue for quantized SPHINX. https://github.com/Alpha-VLLM/LLaMA2-Accessory/issues/114
Hi Team, wanted to try Sphinx out but got following error:
AttributeError: 'NonDynamicallyQuantizableLinear' object has no attribute 'weight'
. Any idea what could be wrong?I am running this on 4090 in up to date Manjaro and I checked hashes of the downloaded files.
This boots up and starts gradio, there I select image, provide prompt and pres Enter.
Please expand the details to see the full log from the terminal: