search

Lightning-AI / litgpt

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
https://lightning.ai
Apache License 2.0
10.69k stars 1.06k forks source link

litgpt download ..model_name... --convert_checkpoint false is performing conversion #1791

Open wiiiktor opened 1 month ago

wiiiktor commented 1 month ago

Bug description

The command litgpt download ..model_name... --convert_checkpoint false is performing conversion and deletes the safetensor files (my workaround is to terminate work after safetensors are downloaded, just before they are deleted):

image

What operating system are you using?

Unknown

LitGPT Version


Version: 0.4.13
rasbt commented 3 weeks ago

Thanks for reporting!

The --convert_checkpoint false flag is not for the .safetensor file conversion but for the HF -> LitGPT conversion.

Some models on the HF Hub have .bin files and some have .safetensor files. So, for simplicity, .safetensor files are always converted to .bin files as a first step (this is part of the download logic, not the checkpoint conversion logic, so it happens whether --convert_checkpoint is true or false.

To prevent the .safetensor files from being deleted, you could comment out the following line:

https://github.com/Lightning-AI/litgpt/blob/5a771c7cef6315750a5641ac21b60bbc8fbe0fde/litgpt/scripts/download.py#L119

Anyways, thanks for reporting this though. It would be nice to improve the HF -> LitGPT conversion logic some time to read from .safetensor files directly (I do that in some of my other code bases too and should be too hard).

wiiiktor commented 3 weeks ago

Just after converting from safetensor to bin, I terminate this operation and safetensor files stay in my folder ;) Anyway, thanks for explanation. Safetensor files are oftentimes handy, as in practice I use various tools/libraries and some of them use safetensors instead of bins.