CUDA out of memory.

[mrcatman]

I'm trying to finetune a medium Russian model with my small amount of data.

After 5-10 seconds I get an error: RuntimeError: CUDA out of memory. Tried to allocate 96.00 MiB (GPU 0; 7.79 GiB total capacity; 6.59 GiB already allocated; 74.06 MiB free; 6.79 GiB reserved in total by PyTorch)

My script:

`export TRAIN_FILE=./corpus/texts.txt

export CUDA_VISIBLE_DEVICES=0
export MODEL_SIZE=gpt2-medium
export OUTPUT=gpt2/m_checkpoint-3364613
export BS=3
export LR=3e-5

while true
do
    python run_lm_finetuning.py \
        --output_dir=$OUTPUT \
        --model_type=gpt2 \
        --model_name_or_path=$OUTPUT \
        --do_train \
        --train_data_file=$TRAIN_FILE \
        --per_gpu_train_batch_size $BS \
        --save_steps=10000 \
        --logging_steps=10 \
        --fp16 \
        --fp16_opt_level O2 \
        --warmup_samples 16000 \
        --learning_rate $LR \
        --overwrite_output_dir \
        --tokenizer_class YTEncoder \
        --tokenizer_name bpe/yt.model \
        --do_eval \
        --evaluate_during_training \
        --eval_steps 1000 \
        --eval_data_file=./data/classic/valid \
        --save_total_limit 30 \
        --num_train_epochs 10.0 \
        --unfreeze_level 0

    sleep 1
done

What should I do to lower the amount of memory used?

[mgrankin]

export BS=1

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.