search

ollama / ollama-python

Ollama Python library
https://ollama.com
MIT License
2.71k stars 223 forks source link

From ollama to transformer.AutoModelForCausalLM #71

Closed Demirrr closed 4 months ago

Demirrr commented 4 months ago

Dear all,

Thank you for ollama-python.

I was wondering whether it would be possible to create transformers.AutoModelForCausalLM from ollama, e.g.,

import ollama
model, tokenizer = ollama.from('mixtral:8x7b ')

where

mxyng commented 4 months ago

It's not (currently) possible to convert an ollama model back into a transformers model