Fine tune on p3dn.24xlarge works great!

minimaxir / gpt-2-simple

Python package to easily retrain OpenAI's GPT-2 text-generating model on new texts

Other

3.4k stars 676 forks source link

Fine tune on p3dn.24xlarge works great! #229

Open EmilyWebber opened 4 years ago

EmilyWebber commented 4 years ago

HI folks, great job on this project! One note - I had zero issues downloading the large 774M model and fine-tuning this on 16MB using an AWS GPU. This was the p3dn.24xlarge, available on a SageMaker training job. The entire process took less than 30 minutes!

Instance details - https://aws.amazon.com/sagemaker/pricing/instance-types/

Here's my example repo - https://github.com/aws-samples/amazon-sagemaker-architecting-for-ml/tree/master/Example-Project

bala1802 commented 3 years ago

I have a dataset of size 640MB, can you please suggest me the right sagemaker instance for fine tuning 774M GPT2 model ?