-
hi please see below,
I can download these directly no problem, but when I try though LMStudion this fails.
I'm on win10, been using your amazing software for a few good months now, this is happeni…
-
I encountered an issue when trying to export a GGUF model file for Mistral Nemo and Mistral 7B finetunes using the `unsloth` library. The error occurs during the `save_pretrained_gguf` function call, …
-
### Describe the issue as clearly as possible:
I got a problem while creating a generator with ```outlines.generate.json(model, schema)``` and as a schema using Pydantic Model with the fields like …
-
I am not getting error but after running q&a generation it took 20 minutes and got empty datasets
please let me know what can be the cause.
(granite1) sankar@Sankars-MacBook-Pro test1 % ilab dat…
-
### Proposal to improve performance
_No response_
### Report of performance regression
Model: meta-llama/Meta-Llama-3-8B-Instruct
GPU: 1x A6000
| SamplingParams.logprobs | Generation Throughput…
-
**Describe the bug**
question:
1. when I use flops_profiler to profile TFLOPS of Megatron GPT, I get the value not correct.
2. when I use flops_profiler to train LLAMA2, I can't get any infos.
…
-
Currently when trying to run with a local model that isn't downloaded, the app crashes with an error such as the following:
```
⇒ npx humanifyjs local --disableGpu foo.js
(node:96922) [DEP0040] Dep…
-
Thank you for your solid work. I would like to ask if the current version is suitable for GQA architecture models, such as LLaMA-2-70B and LLaMA-3.
-
`CUDA_VISIBLE_DEVICES=0,1 lm_eval --model vllm \
--model_args pretrained=/home/jovyan/data-vol-1/models/meta-llama__Llama3.1-70B-Instruct,tensor_parallel_size=2,dtype=auto,gpu_memory_utilization=…
-
### Describe the issue as clearly as possible:
I am trying the mlx-lm examples from your documentation:
[https://dottxt-ai.github.io/outlines/latest/reference/models/mlxlm/](https://dottxt-ai.github…