search

mzbac / AutoGPTQ-API

Host the GPTQ model using AutoGPTQ as an API that is compatible with text generation UI API.
91 stars 14 forks source link

How add an API if the model is already downloaded separately via text-generation-webui #5

Open vishagar81 opened 1 year ago

vishagar81 commented 1 year ago

Hello, Thanks for sharing your samples for the vsc extension and API. I have already downloaded the 'TheBloke_WizardCoder-15B-1.0-GPTQ' model via text-generation-webui. Would it possible to reference that model via the API? If you could help me in this regard then that would be deeply appreciated. image

mzbac commented 1 year ago

It used to be compatible with the text generation web UI. If there are any changes in the web UI, you may have to fork the VSC extension repository and make the change there to make it compatible with the web UI API. https://github.com/mzbac/wizardCoder-vsc/blob/main/src/WizardCoderInlineCompletionItemProvider.ts#L53-L57 https://github.com/mzbac/wizardCoder-vsc/blob/main/src/chatWithWizardCoder.ts#L50-L59

vishagar81 commented 1 year ago

Thank you so much for your response.