-
Implement minimum risk trainer as described in http://arxiv.org/abs/1512.02433.
- sampling is done by computing top-k translations (beam search is already done)
-
As a user of vocascan, I'd like to have the feature of importing and exporting vocabularies as csv files.
I can imagine that many starts to learn by creating tables and have it already there and wo…
-
hello
I want to build a speech translation model for an Arabic dialect using the fairseq toolkit, so I start by preparing my data to start to train my model. For preparing data for training I have in…
-
Hello so i was fine-tuning a llama-2 model with unsloth using a tokenizer of my own, it has an extended vocabulary of around 48000 tokens in total, the tokenizer is compatible and checks have been mad…
-
Thanks for making the repo public. I am new to the machine translation and your repo seems promising to me. could you please explain about how to go about train and evaluate the model for my datasets.
-
To better plan and have an overview of the tasks still open to implementing the Pushshift Reddit benchmark I ask you to go through the following functions with regard to the Pushshift Reddit benchmark…
-
在32G 的卡上跑,clip 默认设置的200。 batch-size=1都会溢出。也是服了。
然后发现preprocess里面的clip 其实没有clip掉。最长的句子竟然会出现1146个char 溢出应该也正常吧。。。
修改了代码,贴上来,有遇到同样问题的人可以参考一下。
```
import argparse
import numpy as np
import os
imp…
-
I was training skip-thought model with my own data following the README in training directory.
But I encountered TypeError and I'm not familiar with Theano, so I don't know how to do about it.
Her…
-
==((====))== Unsloth: Fast Llama patching release 2024.4
\\ /| GPU: NVIDIA GeForce RTX 2060 SUPER. Max memory: 7.785 GB. Platform = Linux.
O^O/ \_/ \ Pytorch: 2.3.0. CUDA = 7.5. CUDA Too…
-
采用命令运行:python finetune.py --datasets_path /Fengshenbang-LM/fengshen/examples/finetune_taiyi_stable_diffusion/demo_dataset --datasets_type txt --model_path models/Taiyi-Stable-Diffusion-1B-Chinese-E…