ltoniazzi / reduce-llms-for-testing

Reduce LLMs size for testing
0 stars 0 forks source link

Gemma not working #14

Closed ltoniazzi closed 3 months ago

ltoniazzi commented 3 months ago

Check PR #5 description for more details.

Llama.ccp gguf-py/scripts/gguf_dump.pygguf-py/scripts/gguf_dump.py saving code:

    for n, tensor in enumerate(reader.tensors, 1):
        if tensor.name.endswith(".lora_a") or tensor.name.endswith(".lora_b"):
            name_file = "_lora_phi.txt"
        else:
            name_file = "_base_phi.txt"

        with open(name_file, "a") as file:
            if tensor.name.endswith(".lora_b"):
                skip = True
            elif tensor.name.endswith(".lora_a"):
                skip = False
                t_name = tensor.name.replace(".lora_a", "")
            else:
                skip = False
                t_name = tensor.name
            if not skip:
                file.write(t_name + "\n")

        prettydims = ', '.join('{0:5}'.format(d) for d in list(tensor.shape) + [1] * (4 - len(tensor.shape)))
        print(f'  {n:5}: {tensor.n_elements:10} | {prettydims} | {tensor.tensor_type.name:7} | {tensor.name}')  # noqa: NP100
ltoniazzi commented 3 months ago

Solved in #15.