-
## 개요
- LLM.int8() + LoRA를 활용한 memory¶meter efficient fine tuning
- BitsAndBytes + Peft 활용한 모델 학습 예정
- Backbone은 [polyglot-ko-5.8b](https://huggingface.co/EleutherAI/polyglot-ko-5.8b) 활용(KoGPT는…
-
General
- [ ] README.md may list directory layout and what is contained in each folder. Alternatively, each folder should have a self-explanatory name so documentation is not needed. e.g. "data" f…
-
### Describe the feature
I want to continue pre-training llama 2 70b using my own data. My data is about 1b tokens. I have read [Fine-tuning Llama 2 70B using PyTorch FSDP ](https://huggingface.co/bl…
-
Hi @NielsRogge
I have finetuned my paligemma for custom data for image to JSON use case, but when I inference it some key values I got wrong like 3000 is extracted as 9000 so to get the data is corr…
-
https://lightning.ai/pages/community/finetuning-falcon-efficiently/
-
Hi,
I'm interested in finding near-duplicate audio files. My dataset is about 3000 thousands short audio files, between 0.5 seconds to 5 seconds. Unlike Shazam, both the "target" audio (i.e. the song…
-
Hi,
I'm interested in finding near-duplicate audio files. My dataset is about 3000 thousands short audio files, between 0.5 seconds to 5 seconds. Unlike Shazam, both the "target" audio (i.e. the song…
-
# LoRA: Low-Rank Adaptation of Large Language Models
基于large pre-trained model,把基于某个任务的微调存储在低秩矩阵对中,low intrinsic dimension $r=4$ 就够。
Pro:
- 并行化不影响速度、任务特化的信息相对很少。
- 该方法对超参数极其不敏感。
另外:
- 对于模型…
-
### Zoom: https://navercorp.zoom.us/j/92208940283
### 페이스북: https://www.facebook.com/weeklyaiarxivpage
### News
- Conference
- ICLR 2024
- Abs: 9.23 AoE (9.21에서 변경) , Full paper: 9.28…
-
Hi,
I'm interested in finding near-duplicate audio files. My dataset is about 3000 thousands short audio files, between 0.5 seconds to 5 seconds. Unlike Shazam, both the "target" audio (i.e. the song…