-
torchrun --nproc_per_node 1 llamacpp_mock_api.py \
--ckpt_dir CodeLlama-7b-Instruct/ \
--tokenizer_path CodeLlama-7b-Instruct/tokenizer.model \
--max_seq_len 128 --max_batch_size 4
…
HwJhx updated
2 months ago
-
### Describe the issue
There is my code:
```
import autogen
config_list_codellama = [
{
'base_url': "http://localhost:1234/v1",
'api_key': 'NULL',
}
]
llm_con…
-
The puzzle `75-712e2839` from #75 has to be resolved:
https://github.com/h1alexbel/fakehub/blob/851f56de607b37ef5b4f894ff6d412b0407a5849/server/src/xml/storage.rs#L80-L82
The puzzle was created by …
0pdd updated
2 months ago
-
Type: Bug
I don't know how to set it, because I have missed the first time setting....
I reinstalled again and again, but he disappeared in the extension module...
Finally, i find it with ctrl+ l
B…
-
Hi guys
I had a report earlier today from a user telling me that he tried one of my new AWQ models, and got an error indicating that only float16 is supported with AWQ.
I tested it myself with t…
-
The puzzle `79-d0a80208` from #79 has to be resolved:
https://github.com/h1alexbel/fakehub/blob/6932d8bc8d0c334bc7dba4106a820a77e305f622/cli/src/args.rs#L28-L30
The puzzle was created by @rultor on…
0pdd updated
2 months ago
-
使用PPO训练13B的模型,内存占用特别高,我应该怎么解决
-
**Describe the bug**
Use wsl start server and follow docs, the command is
`./tabby serve --model $LLM_MODEL_PATH --chat-model $LLM_MODEL_PATH`
The log shows that the model exists and is loading.…
-
First, this app works great on a MacBook Pro M3 Max 128GB and for lots of transformers and LLM models. One of the few RAG app's where I have been able to run it without the internet (well, once all o…
-
### What happened?
./llama-infill -t 10 -ngl 0 -m ../../models/Publisher/Repository/codellama-13b.Q3_K_S.gguf --temp 0.7 --repeat_penalty 1.1 -n 20 --in-prefix "def helloworld():\n print(\"hell" -…