-
Hi, I was wondering how exactly the source and target vocabs for the Dual2Seq experiment are retrieved.
Are you using one of your get_vocab scripts?
Do you simply concatenate your sentence-words wi…
-
Hello all,
I'm using my own data for training a transformer model for machine translation. I am using the standard pipeline with t2t-datagen and t2t-trainer and it's fine to train the model. In som…
-
I would like to ask for your advice on the following two questions.
1. DPO train does not seem to support DeepSpeed ZeRO. After manually integrating `DPOAlignerArguments` with the `FinetunerArguments…
-
# Implementing Proximal Policy Optimisation
I've used some of the [PyTorch RFC](https://github.com/pytorch/rfcs/blob/master/README.md) template here for clarity.
**Authors:**
* @salmanmohammadi…
-
Hi
I want to train a RNNLM. My vocabulary size is 48603 (cutoff=100) or 72294 (cutoff=50). Data is stored in CTF format which is suitable for sparse data. Training RNN with vocabulary size of 4860…
-
I am not able to figure out how to set the interactive model config to load `checkpoint_path='gs://scenic-bucket/ul2/ul220b/checkpoint_2650000'` from [here](https://github.com/google-research/google-r…
-
I'd like to see a feature that lets the user switch out main display of a word with an alternative writing. Or at least a way to disambiguate a word that has several alternate writings and meanings, b…
K-410 updated
9 years ago
-
### 🐛 Describe the bug
i use pytorch==2.3.0 and peft to train llama3 8b , when i run my code, its raise error like:
```text
torch._amp_foreach_non_finite_check_and_unscale_(
RuntimeError:…
-
I checked there is a pretrained model in repo "https://github.com/rewicks/ersatz-models/tree/main/monolingual/en".
As I cannot find the tokenizer Vocabulary, I am not sure how to finetune the existed…
-
When I run:
thumt-trainer \
--input corpus.tc.32k.zh.shuf corpus.tc.32k.en.shuf \
--vocabulary vocab.32k.zh.txt vocab.32k.en.txt \
--model transformer \
--validation newsdev2017.tc.32k.zh…