Publish to pypi

[winglian]

Can you make this pip installable from pypi? Makes it hard to integrate with other libraries with axolotl. Thanks!

[zyushun]

Hi @winglian ! Thanks for the great suggestions! This is exactly our next move. Will update in a week or so.

[zyushun]

Hi @winglian !

We have done the pip installization. You can now use as follows (as you may see in the updated readme).

git clone https://github.com/zyushun/Adam-mini
pip install -e .

from adam_mini import Adam_mini

optimizer = Adam_mini(
            named_parameters = model.named_parameters(), 
            lr = lr, 
            betas = (beta1,beta2), 
            eps = eps,
            weight_decay = weight_decay, 
            model_sharding = True,
            dim = model_config.dim,
            n_heads = model_config.n_heads,
            n_kv_heads = model_config.n_kv_heads,
            )

Thanks for your suggestion!

[winglian]

Thank you!