search

Adamliu1 / SNLP_GCW

3 stars 0 forks source link

Add special case for saving model when running with ZERO-3 optimisation. #149

Open Willmish opened 2 months ago

Willmish commented 2 months ago

Tiny fix so a model sharded with Zero-3 can be saved! (To test if it doesnt also save the ugly placeholder confusing huggingface model loader)

Willmish commented 2 months ago

Weird thing is: when saving the model, it saves it fine, but also saves the placeholder model.safetensors file, hierarchy looks like:

config.json
generation_config.json
model-00001-of-00004.safetensors
model-00002-of-00004.safetensors
model-00003-of-00004.safetensors
model-00004-of-00004.safetensors
model.safetensors
model.safetensors.index.json
special_tokens_map.json
tokenizer_config.json
tokenizer.json

But model.safetensors shouldnt be there! (it causes things like AutoModelForCausalLM.from_pretrained( to fail, unless the placeholder file is removed!