search

in-context-learning-2024 / in-context

5 stars 1 forks source link

LoRA Finetuning #33

Open riensou opened 7 months ago

riensou commented 7 months ago

Pull request for Issue #26

riensou commented 7 months ago

examples of some training runs:

  1. training a gpt2 model on linear regression from scratch
train_linear
  1. lora fine-tuning the above model on sparse linear regression
lora_sparse
  1. training a gpt2 model on sparse linear regression from scratch
train_sparse
  1. lora fine-tuning the above model on linear regression
lora_linear