-
found your message on exllama for fastapi : https://github.com/turboderp/exllama/issues/37
what am i supposed to do, use your exllama version with the fastapi you created, or can i run it with the …
-
Hello,
I'm queueing up some updates for new entries and have some questions:
- Do you prefer pull requests with multiple new entries, or separate pull requests for each new entry?
- There seem to…
-
`
/Users/andyblocker/mambaforge/envs/llm-as-chatbot/lib/python3.9/site-packages/transformers/generation/utils.py:725: UserWarning: MPS: no support for int64 repeats mask, casting it to int32 (Trigger…
-
Hi @PromtEngineer
I have followed the README instructions and also watched your latest YouTube video, but even if I set the `--device_type` to `cuda` manually when running the `run_localGPT.py` or…
-
I use this code to load the model:
```
model = 'WizardLM/WizardCoder-15B-V1.0'
def load_model(model = model):
tokenizer = AutoTokenizer.from_pretrained(model)
model = AutoModelForCausal…
-
PC Hardware:
- i7-9700kf
- Nvidia gtx 1660(6Gb)
- 16Gb memory
model used:
- TheBloke/wizardLM-7B-GPTQ
- wizardLM-7B-GPTQ-4bit-128g.no-act.order.safetensors
env:
Running in windows10 WSL2…
-
### Feature request
plug in to
https://github.com/lmstudio-ai/model-catalog
seems like this is all ggml format.
https://github.com/lmstudio-ai/model-catalog/blob/main/catalog.json
### Motiv…
-
- OS: **Ubuntu 22.04**
- GPUs: **2x 4090** (2x 24GB)
- CUDA: **11.8**
- CPU: **Ryzen 3800X**
- RAM: **64GB**
- vLLM build: **main** `400b8289`
Started the API server with this command:
```sh
…
-
All modern transformer models has problems with math. that's because they tokenise numbers weirdly.
Your model supposed to be better in math than others. But main reason, why are transformers so bad …
-
# Far-fetched app idea - Junior coder
This is more of a stretch goal than a must.
## Idea
Currently with advent of coding focused LLM like StarCoder, an enticing but difficult use case of let…