How to convert Meta-Llama-Guard-2-8B to pre_trained models

[xtrycatchx]

So, basically I managed to download the model Meta-Llama-Guard-2-8B following the instructions described at https://llama.meta.com/llama-downloads

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a----         4/17/2024   5:01 AM    16060563346 consolidated.00.pth
-a----         7/18/2024  11:20 AM           1042 download.sh
-a----         12/6/2023   4:59 PM           7020 LICENSE
-a----         4/17/2024   5:01 AM            212 params.json
-a----         4/17/2024   5:01 AM        2183982 tokenizer.model
-a----         12/6/2023   4:59 PM           4790 USE_POLICY.md

Do I need to run a script to convert these into a model that I can load directly with Huggingface transformers?

I just blindly tried the snippets from Meta's repo in Huggingface and replacing the model_id with the directory containing the above downloaded files and the device configured as cpu like this below:

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# model_id = "meta-llama/Meta-Llama-Guard-2-8B"
model_id = "./downloaded-Meta-Llama-Guard-2-8B"
device =  "cpu" #"cuda"
dtype = torch.bfloat16

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=dtype, device_map=device)

def moderate(chat):
    input_ids = tokenizer.apply_chat_template(chat, return_tensors="pt").to(device)
    output = model.generate(input_ids=input_ids, max_new_tokens=100, pad_token_id=0)
    prompt_len = input_ids.shape[-1]
    return tokenizer.decode(output[0][prompt_len:], skip_special_tokens=True)

moderate([
    {"role": "user", "content": "I forgot how to kill a process in Linux, can you help?"},
    {"role": "assistant", "content": "Sure! To kill a process in Linux, you can use the kill command followed by the process ID (PID) of the process you want to terminate."},
])

But I got these errors ( looking for a config.json file, etc.. )

(environment) PS C:\Users\batman\Documents\EXPERIMENTS> python .\meta-llama-guard-2_1.py
Traceback (most recent call last):
  File "C:\Users\batman\Documents\EXPERIMENTS\meta-llama-guard-2_1.py", line 9, in <module>
    tokenizer = AutoTokenizer.from_pretrained(model_id)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\batman\Documents\EXPERIMENTS\environment\Lib\site-packages\transformers\models\auto\tokenization_auto.py", line 846, in from_pretrained 
    config = AutoConfig.from_pretrained(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\batman\Documents\EXPERIMENTS\environment\Lib\site-packages\transformers\models\auto\configuration_auto.py", line 965, in from_pretrained   
    config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\batman\Documents\EXPERIMENTS\environment\Lib\site-packages\transformers\configuration_utils.py", line 632, in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\batman\Documents\EXPERIMENTS\environment\Lib\site-packages\transformers\configuration_utils.py", line 689, in _get_config_dict
    resolved_config_file = cached_file(
                           ^^^^^^^^^^^^
  File "C:\Users\batman\Documents\EXPERIMENTS\environment\Lib\site-packages\transformers\utils\hub.py", line 373, in cached_file
    raise EnvironmentError(
OSError: ./downloaded-Meta-Llama-Guard-2-8B does not appear to have a file named config.json. Checkout 'https://huggingface.co/./downloaded-Meta-Llama-Guard-2-8B/tree/None' for available files.

[JFChi]

If you want to use the Huggingface format of the checkpoint, please download our model in HF format. Note that the model downloaded using https://llama.meta.com/llama-downloads is not in HF format.

[xtrycatchx]

Thanks. I confirm the HF models works as expected