Closed sigmareaver closed 1 year ago
Came here to see how we can load https://huggingface.co/mosaicml/mpt-7b-storywriter into oogabooga... guessing the answer is we can not
https://www.youtube.com/watch?v=QVVb6Md6huA&t=1s ubuntu
https://www.youtube.com/watch?v=O9Y_ZdsuKWQ windows
https://github.com/oobabooga/text-generation-webui/issues/354
https://github.com/oobabooga/text-generation-webui/issues/1927
https://github.com/oobabooga/text-generation-webui/issues/1915
https://github.com/oobabooga/text-generation-webui/issues/1856 tie in other similar issues making them easier to close when solved
The model type is "mpt", which they created themselves, and is not one of the model types with GPTQ support.
https://github.com/oobabooga/text-generation-webui/issues/1828 trunk for this item. should answer most questions
I am having the same problem trying to load OccamRazor/mpt-7b-storywriter-4bit-128g.
I can get the og mosaicml/mpt-7b-storywriter working.
I get the same ERROR:Can't determine model type from model name. Please specify it manually using --model_type argument as the original poster when i try launch from inside the webui or from the cmd env.
I am having the same problem trying to load OccamRazor/mpt-7b-storywriter-4bit-128g.
I can get the og mosaicml/mpt-7b-storywriter working.
I get the same ERROR:Can't determine model type from model name. Please specify it manually using --model_type argument as the original poster when i try launch from inside the webui or from the cmd env.
Have you seen this issue with storywriter or any other mpt models? I keep having the output devolve into madness and haven't figured out yet if this is common for others or a me thing: https://github.com/oobabooga/text-generation-webui/discussions/1815#discussioncomment-5819492
@CarlKenner Where do you get the "mpt" model type definition?
@CarlKenner Where do you get the "mpt" model type definition?
It seems someone managed doing a 4bit model for MPT: https://github.com/0cc4m/GPTQ-for-LLaMa/blob/latestmerge/gptq/mpt.py
https://github.com/abetlen and https://github.com/ggerganov and https://github.com/jllllll/GPTQ-for-LLaMa/commits?author=jllllll in case they are not aware? [pretty sure they know, but just in case as always]
And still have not managed to get any version of mpt running. (Update: Well, I just got mpt-7b-storywriter to load before hitting the out of memory error thanks to the tutorial video I found here, but still have not gotten the 4bit-128g model to run.)
Error Messages: (with model_type mpt)
(with model_type removed)
(with --model-menu in place)
In addition I have to create a separate .bat file because there is no dropdown menu in the webui to select these options. I am not a strong coder so don't like to dabble with webui.py files and instructions are scattered across different tutorials often related to unrelated models.
I hear as someone mentioned: occam made offers mpt support: https://github.com/0cc4m/GPTQ-for-LLaMa But I have no idea how to get that to work in Oobabooga.
The main page says oobabooga/text-generation-webui supports MPT here: Instruct mode compatible with various formats, including Alpaca, Vicuna, Open Assistant, Dolly, Koala, ChatGLM, MOSS, RWKV-Raven, Galactica, StableLM, WizardLM, Baize, Ziya, Chinese-Vicuna, MPT, INCITE, Wizard Mega, KoAlpaca, Vigogne, Bactrian, h2o, and OpenBuddy
But I have not had any luck with it out of the box so not sure how it supports MPT out of the box if it is causing this much of a headache to run.
Having Oobabooga support MPT out of the box, with trust-remote-code selectable within the WebUI without a custom .bat file, and without tweaking or playing around/breaking code would be ideal. Also having proper versioning would be nice, to know if this is Oobabooga/text-generation 1.4 vs. 2.8 and be able to compare the changes between them, without overriding and breaking everything that used to work -- but then doesn't.
Any one managed to get it working?
This issue has been closed due to inactivity for 30 days. If you believe it is still relevant, please leave a comment below.
Describe the bug
Unable to load the model because it can't determine model type.
Is there an existing issue for this?
Reproduction
Attempt to load mpt-7b-storywriter-4bit-128g.
Screenshot
No response
Logs
System Info