-
DJL does not support (or has not documented support) for FP8 quantization ([docs](https://demodocs.djl.ai/docs/serving/serving/docs/lmi/user_guides/trt_llm_user_guide.html#quantization-support)).
…
-
The problem only happens with LLaMA 65B, LLaMA 7B/13B/30B work well. Below is the reproduce code：
```
from vllm import LLM, SamplingParams
args_model = '/mnt/sdb/ly/models/hf_converted_llama/65B/'
…
-
**What would you like to be added/modified**:
A benchmark suite for large language models deployed at the edge using KubeEdge-Ianvs:
1. Interface Design and Usage Guidelines Document;
2. Implem…
-
### 先决条件
- [X] 我已经搜索过 [问题](https://github.com/open-compass/opencompass/issues/) 和 [讨论](https://github.com/open-compass/opencompass/discussions) 但未得到预期的帮助。
- [X] 错误在 [最新版本](https://github.com/open-…
-
```bash
2024-09-21:20:19:56,843 INFO [_logging.py:101] Running EleutherEvalRecipe with resolved config:
batch_size: 1
checkpointer:
_component_: torchtune.training.FullModelHFCheckpointer
…
-
### Reminder
- [X] I have read the README and searched the existing issues.
### System Info
目前llamafactory提高的评测集有没有可以直接评估增量预训练后的模型的，结果是得分，还是困惑度。可以直接评估大模型增量预训练后是否能力遗忘 …
-
问题发现于swift/swift/llm/eval.py:295
```python
nlp_datasets = set(OpenCompassBackendManager.list_datasets()) & set(args.eval_dataset)
```
直接运行
```python
from evalscope.backend.opencompass import Ope…
-
What are `mmlu_continuation` and `mmlu_generative`? Where can I find their description?
I am going to test `mmlu` in the `cloze` way. Like the following illustration:
:
https://www.reddit.com/r/ChatGPT/comments/11t389v/wow_gpt4_beats_all_the_other_translators/
-
## Background
The current schema for `Instance` is very limited - it allows a single `Input` and a `List[Reference]`. This is appropriate for most question answering scenarios and traditional NLP t…