GPT-2 autoencoder checkpoint?

openai / sparse_autoencoder

MIT License

340 stars 35 forks source link

GPT-2 autoencoder checkpoint? #4

Closed GheorgheMorari closed 5 months ago

GheorgheMorari commented 5 months ago

Will a checkpoint be released for the gpt-2 sparse autoencoder alongside the training code? If not, how much vram is approximatively needed for the training for the gpt-2 autoencoder model?

WuTheFWasThat commented 5 months ago

yes, they are released. see https://github.com/openai/sparse_autoencoder/blob/main/sparse_autoencoder/paths.py