Not pushing merged_16bit model.

[pandora-s-git]

When I try pushing the model to a Hugging Face repo, everything seems to work fine, except when I push a 'merged_16bit' version.

For reference, I managed to push all the versions I wanted (Lora, GGUF Q2 up to F16).

But when I try with model.push_to_hub_merged("username/repo", tokenizer, save_method="merged_16bit"), it completes the process without any problems or errors. It even creates the repository, begins loading the model, and claims that everything was uploaded successfully with a clear Done. Saved to https://huggingface.co/usernamep/repo. However, upon checking the repository, it only contains the README file...

[pandora-s-git]

Okay so, I understood what happen on my end, but It should be a bug on your side I believe. For some weird reason, the model was uploaded to my main account and not the Organization repo it was supposed to... even tho I did point to the correct repo and organization name.

[danielhanchen]

@pandora-s-git Oops you're correct - working on it!

[danielhanchen]

@pandora-s-git Should be fixed hopefully! You were correct on not uploading to another organization's username! Not 100% if it works now - it'll be super awesome if you could test it out - highly appreciate it :) To update with no dependency changes on a local machine: pip install --upgrade --force-reinstall --no-cache-dir git+https://github.com/unslothai/unsloth.git. For Colab - no need to update! Thanks again!

[pandora-s-git]

I've bad news, this is a bit awkward... I did a new test and the outcome was the same, it still pushed to the my main account and not the Organization. I did it on colab btw, so it should have reinstalled Unsloth. So, I poorly reinstalled it, or the issue remains.

As stated before, the repo is created on the Organization account, and the README file too, but the model itself is pushed to my main account.

I will gladly test it again if needed, am I doing something wrong tho?

[danielhanchen]

@pandora-s-git Oh no - will test later today - sorry! :(

[pandora-s-git]

So uh- last week I tested and the problem was still there. Just to know if there are any news?

[danielhanchen]

@pandora-s-git OMG so so sorry I totally forgot about this............... Extreme extreme apologies - I'll put a fix in by today - I'll label this as URGENT!!

[danielhanchen]

@pandora-s-git Extreme apologies - I think I fixed it! I confirmed I could upload to my own organization account - it'll be fabulous if you can try it out!! Again massively sorry on the horrible delay!

[480]

@danielhanchen

I had the same issue.

• model.push_to_hub_gguf -> uploaded under the organization account as specified as normal
• model.push_to_hub_merged -> removed the specified organization account and uploaded under a personal account.

Until yesterday, it was working as above, but after today's new running, it uploads correctly to the specified organization account.

I was tinkering with the code to fix the issue, so thanks for the quick fix. 👍

[danielhanchen]

Glad it got fixed - again apologies on the issue and the horrid delay!

[480]

This is an upload capture of lora, gguf, and tensors in a single repo to verify that the issue is resolved.

[danielhanchen]

@480 Fabulous!! Glad it works as intended!

[pandora-s-git]

Ugh... this is weird, it did not work for me once again (tried yesterday), same issue, made a repo on the organization but pushed in my private account the model files.

A code snippet :

model_path_16bit = f"ORGANIZATION NAME/NAME OF MODEL"
if save_16bit:
  model.push_to_hub_merged(model_path_16bit, tokenizer, save_method = "merged_16bit", private=True)

I'm not doing anything special so I still think it's not a problem on my side, but yet 480 says its solved so... Im not sure anymore.

[danielhanchen]

@pandora-s-git I can check again - is this via Colab / Kaggle or a local machine? If local machine, you'll have to update Unsloth. If Colab / Kaggle, it should work - I'll check this today! Sorry :(

[pandora-s-git]

It's on Colab T-T

[danielhanchen]

@pandora-s-git Ok not good :( Let me check now! Sorry on the issue :(

[danielhanchen]

@pandora-s-git Sorry on the delay! Weirdly I managed to upload to https://huggingface.co/unslothai/06042024/tree/main for GGUF, merged 16bit and lora. Maybe you used HF_TOKEN? If that's the case - I added a fix a few minutes ago to use that!

I used `model.push_to_hub_merged("unslothai/06042024", tokenizer, save_method = "merged_16bit", token = "hf_xxx") for the model

[federicotorrielli]

@danielhanchen I have a very similar problem. I cannot push my model (16) to the hub because of this strange error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In [13], line 2
      1 model.save_pretrained_merged("Meta-Llama-3-8B-Instruct-conceptnet_UsedFor_en_en", tokenizer, save_method = "merged_16bit")
----> 2 model.push_to_hub_merged("EvilScript/Meta-Llama-3-8B-Instruct-conceptnet_UsedFor_en_en", tokenizer, save_method = "merged_16bit", token = "...")

File /usr/local/lib/python3.9/dist-packages/unsloth/save.py:1106, in unsloth_push_to_hub_merged(self, repo_id, tokenizer, save_method, use_temp_dir, commit_message, private, token, max_shard_size, create_pr, safe_serialization, revision, commit_description, tags, temporary_location, maximum_memory_usage)
   1104 del arguments["self"]
   1105 del arguments["repo_id"]
-> 1106 unsloth_save_model(**arguments)
   1107 for _ in range(3):
   1108     gc.collect()

File /usr/local/lib/python3.9/dist-packages/torch/utils/_contextlib.py:115, in context_decorator.<locals>.decorate_context(*args, **kwargs)
    112 @functools.wraps(func)
    113 def decorate_context(*args, **kwargs):
    114     with ctx_factory():
--> 115         return func(*args, **kwargs)

File /usr/local/lib/python3.9/dist-packages/unsloth/save.py:651, in unsloth_save_model(model, tokenizer, save_directory, save_method, push_to_hub, token, is_main_process, state_dict, save_function, max_shard_size, safe_serialization, variant, save_peft_format, use_temp_dir, commit_message, private, create_pr, revision, commit_description, tags, temporary_location, maximum_memory_usage)
    642     hf_api.upload_folder(
    643         folder_path = new_save_directory,
    644         path_in_repo = ".",
   (...)
    648         ignore_patterns = "*.md",
    649     )
    650 else:
--> 651     internal_model.save_pretrained(**save_pretrained_settings)
    652 pass
    654 # Revert config back

File /usr/local/lib/python3.9/dist-packages/transformers/modeling_utils.py:2606, in PreTrainedModel.save_pretrained(self, save_directory, is_main_process, state_dict, save_function, push_to_hub, max_shard_size, safe_serialization, variant, token, save_peft_format, **kwargs)
   2598     logger.info(
   2599         f"The model is bigger than the maximum size per checkpoint ({max_shard_size}) and is going to be "
   2600         f"split in {len(shards)} checkpoint shards. You can find where each parameters has been saved in the "
   2601         f"index located at {save_index_file}."
   2602     )
   2604 if push_to_hub:
   2605     # Eventually create an empty model card
-> 2606     model_card = create_and_tag_model_card(
   2607         repo_id, self.model_tags, token=token, ignore_metadata_errors=ignore_metadata_errors
   2608     )
   2610     # Update model card if needed:
   2611     model_card.save(os.path.join(save_directory, "README.md"))

File /usr/local/lib/python3.9/dist-packages/transformers/utils/hub.py:1142, in create_and_tag_model_card(repo_id, tags, token, ignore_metadata_errors)
   1140 if tags is not None:
   1141     for model_tag in tags:
-> 1142         if model_tag not in model_card.data.tags:
   1143             model_card.data.tags.append(model_tag)
   1145 return model_card

TypeError: argument of type 'NoneType' is not iterable

EDIT: to fix it, I had to manually put my tags into the huggingface platforms for it to work

[danielhanchen]

Ok that is a strange error - I will fix this - sorry on the issue!

[asimniazi63]

any update on this? @danielhanchen

[zihualiu]

+1 on this. trying to use model.merge_and_unload() and then saving also does not work

[danielhanchen]

@zihualiu You shouldn't use that sadly - instead use model.push_to_hub_merged("location", tokenizer, save_method = "merged_4bit")