-
问题描述:
使用peft微调llama3 8b,训练代码基本是按照样例稍作修改,在训练的时候 前10个steps,loss稍高,后面输出的loss,一直都是0.0了
微调代码:
```python
import torch
from datasets import Dataset
import pandas as pd
from transformers impo…
-
0:00 - Introduction
0:42 - Sponsors & Contact information
1:31 - Transformers from the ground up
2:37 - Examples for transformers
4:48 - Outline
6:29 - Disclaimer
7:11 - Augmenting RNNs with at…
-
- [LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day](https://arxiv.org/abs/2306.00890)
- [MEDITRON-70B: Scaling Medical Pretraining for Large Language Models](http…
-
-
用的云服务器
/root/miniconda3/lib/python3.8/site-packages/pydub/utils.py:170: RuntimeWarning: Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work
warn("Couldn't find ffmpeg or avconv…
-
Hello everyone, thank you very much for your contribution. I appreciate the effort and consistency in uploading the code for such many models and maintaining this repository.
I saw Kosmos-2 and I q…
-
==((====))== Unsloth: Fast Llama patching release 2024.4
\\ /| GPU: NVIDIA GeForce RTX 2060 SUPER. Max memory: 7.785 GB. Platform = Linux.
O^O/ \_/ \ Pytorch: 2.3.0. CUDA = 7.5. CUDA Too…
-
Hi,
I just got stuck with some "FetchPhaseExecutionException" when using the highlighting and the decomp filter:
InvalidTokenOffsetsException[Token verzinnte exceeds length of provided text sized 83…
-
CUDA SETUP: Highest compute capability among GPUs detected: 8.0
CUDA SETUP: Detected CUDA version 116
CUDA SETUP: Loading binary /opt/conda/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_…
-
Thanks for your great work!
I have some questions about the code which you uploaded on github.
- where is the file whose Directory address is 'tools/llm/llm.py' ?
- I also wonder whether t…