-
I have been trying to follow the steps listed under "reproducing GPT-2" from the README.md. Unfortunately, when I run the model, my training always diverges. I have tried switching up my learning rate…
-
Thanks for your great work! Will the model that have undergone Alignment Pretraining be open-sourced (not the models after SFT)? When will they be open-sourced?
-
Hello, I'm encountering an issue reproducing your experiment. I attempted to use the provided checkpoint for generation without pretraining or fine-tuning. However, the results are significantly worse…
-
I know the general recommendation is to leave the backbone frozen and train task-specific heads. However I'm interested in continuing pre-training to better fit the backbone features to my dataset. Is…
-
Dear author, you have a pre-trained model on github, on which dataset was this model pre-trained? In your paper, you mentioned using the CATH dataset for pre-training. I think it is an interesting dat…
-
I'm new to deep learning but have some experience with training boosted-decision-trees.
Is this just for fine-tuning or pretraining as well? When I look inside train_gpt2.c I see the first thing it…
-
I appreciate this awesome work and I am currently going to try finetune DNTR to my own dataset.
But along the way I have a trouble with the training configuration.
https://github.com/hoiliu-0801…
-
If I want to generate an embedded representation based on a pre-trained model, given some SIMLES sequences, how do I modify the code and preprocess the data.
-
Hi,
a bit of noob question - should I add some shuffling, or this is expected for homography pretraining?
- images (total amoun…