-
Hi, I'm trying to fine-tune the Llama3.1 8b model but after fine-tuning it uploading it to HF, and when trying to run it using vLLM I get this error "KeyError: 'base_model.model.model.layers.0.mlp.dow…
-
Implement an Importer so that semantic models conformant to https://github.com/eclipse-esmf/esmf-semantic-aspect-meta-model can be used as semantic description for a submodel. Aspect Models are used f…
-
### Your current environment
vllm v0.5.4
Setup A) single docker container with vllm, no pipeline-parallelism
```
docker run ... vllm/vllm-openai:v0.5.4 --model "meta-llama/Meta-Llama-3.1-70B-…
-
llm = LMDeployServer(path='internlm/internlm2_5-7b-chat',
model_name='internlm2',
meta_template=INTERNLM2_META,
top_p=0.8,
…
-
### Description
_WIP_
Meta ticket for improving the logical modeling and planning of inlinestats & co. Relates #110923
- [ ] remove Stats interface
- [ ] keep inline model untouched until the optim…
-
### What is the issue?
Trying to run codestral:22b on a 6800xt but get this error everytime :
Error: llama runner process has terminated: signal: segmentation fault (core dumped)
I have 16G RAM …
-
Hugginface hub login successful
Used gemma2-27b LLM to testing:
cargo run --release -- -m "google/gemma-2-27b-it" -c
Finished release [optimized] target(s) in 0.03s
Running `target/re…
-
Instantiating the WatsonxLLM using:
```
model = WatsonxLLM(model_id = meta-llama/llama-3-1-8b-instruct,
params = self.default_params,
…
-
### What happened?
for given IR
```mlir
module {
func.func @"torch-jit-export"(%arg0: !torch.vtensor) -> !torch.vtensor attributes {torch.onnx_meta.ir_version = 6 : si64, torch.onnx_meta.opse…
-
[meta engineering blog post](https://engineering.fb.com/2024/06/12/data-infrastructure/training-large-language-models-at-scale-meta/)
- Meta requires massive computational power to train large lang…