ericcurtin
commented
3 months ago Note /library/ is only used for models in the default namespace, substitute library with the namespace name for non-default namespaces
Open ericcurtin opened 3 months ago
ericcurtin
commented
3 months ago Note /library/ is only used for models in the default namespace, substitute library with the namespace name for non-default namespaces
mtrmac
commented
3 months ago What is the actual feature you are requesting?
From a minimal experiment, https://registry.ollama.ai/v2/ returns 404
I don’t know why we should eagerly add a hack instead of sending a patch, or trying to convince them, to modify the server to be a a compliant registry if that is what it is. (Or, if they explicitly don’t want third-party clients, long-term they are going to win, so I’m not inclined to start fighting them.)
ericcurtin
commented
3 months ago I don't think we will be able to define the ollama protocol.
We could try and support it. It's not very complex as of today:
https://registry.ollama.ai/v2/library/llama3/manifests/latest
doesn't return null.
We don't have to continue to be compatible if indefinitely it proves to constantly change, but as of today it's not incredibly complex.
ericcurtin
commented
3 months ago I just don't want to assume that Ollama will deliberately try and break third-party solutions. There's already at least 2 solutions that support it, LocalAI and ramalama. And the ramalama solution is curl, I think long-term ollama should be more consolidated with the other OCI transports.
mtrmac
commented
3 months ago I mean
What intersection is there with this project? No credentials, no format conversion, no reason to use the same mirrors, no point in copying the data to other non-Ollama OCI registries, I think.
Anyone can write a client using a HTTP client library, just as you did. That’s fine. Not all of those should be inside the Skopeo binary.
ericcurtin
commented
3 months ago This is where I don't agree:
no point in copying the data to other non-Ollama OCI registries
I do think there is a point in this. Lets say a person in enterprise is not allowed to reach out to external OCI registries like Ollama, quay.io, etc. They might pick some models to copy/cache in an internal enterprise OCI registry like helm, etc.
ericcurtin
commented
3 months ago But yeah it doesn't have to be part of skopeo, but at the moment lets take ramalama, it shows different progress bars (and uses different clients) for huggingface, ollama (and the upcoming oci: pr shows download progress in a different way also)
huggingface is completely different I think, but ollama and other OCI registries could be consolidated.
mtrmac
commented
3 months ago Lets say a person in enterprise is not allowed to reach out to external OCI registries like Ollama, quay.io, etc. They might pick some models to copy/cache in an internal enterprise OCI registry like helm, etc.
And how is that going to be consumed? If the Ollama tools specifically required an Ollama server (and artifact formats), that doesn’t help.
The way I think about it:
It amounts to the same thing for me either way.
ericcurtin
commented
3 months ago Lets say a person in enterprise is not allowed to reach out to external OCI registries like Ollama, quay.io, etc. They might pick some models to copy/cache in an internal enterprise OCI registry like helm, etc.
And how is that going to be consumed? If the Ollama tools specifically required an Ollama server (and artifact formats), that doesn’t help.
Yeah this is something that's under discussion @tarilabs has some ideas in this space.
The way I think about it:
- Either the server maintainers want to use interoperable registry protocols (and maybe even artifact formats); great, help them run an OCI distribution-spec compliant registry. No action needed in c/image or Skopeo
- The server maintainers don’t want to be interoperable. I guess we can track a RFE for adversarial interoperability here, but personally I’m going to put it at the very bottom of the list of the >90 c/image and Skopeo items, I’m not going to work on it, and if I can stop myself, I’m not going to even be reviewing PRs.
It amounts to the same thing for me either way.
thxCode
commented
3 months ago I am very confused about why Ollama doesn't use OCI standards to store its models. So I created an alternative to find more answers. https://github.com/gpustack/gguf-packer-go
ericcurtin
commented
3 months ago We are doing very similar things:
https://github.com/containers/ramalama https://github.com/containers/omlmd
I am hoping skopeo can be compatible with Ollama as it's one of our primary OCI registry tools.
mtrmac
commented
3 months ago I think this went way too fast into an implementation suggestion without sufficient analysis.
If no-one knows why they are using a registry-like but registry-incompatible protocol, shouldn’t the first step be to find out?
github-actions[bot]
commented
2 months ago A friendly reminder that this issue had no activity for 30 days.
github-actions[bot]
commented
1 month ago A friendly reminder that this issue had no activity for 30 days.
As you can see from the curl commands below from the ramalama project (toggle x = True in the ramalama python script to print the curl commands) the "Accept: application/vnd.docker.distribution.manifest.v2+json" header shows this format is an OCI transport protocol of sorts.
We should support this in our stack, it's becoming a very popular OCI repository for pulling models.
https://github.com/containers/ramalama
$ ./ramalama pull llama3 curl -f -s --header Accept: application/vnd.docker.distribution.manifest.v2+json -o /var/lib/ramalama/repos/ollama/manifests/registry.ollama.ai/library/llama3/latest https://registry.ollama.ai/v2/library/llama3/manifests/latest curl -f -L -C - --progress-bar --header Accept: application/vnd.docker.distribution.manifest.v2+json -o /var/lib/ramalama/repos/ollama/blobs/sha256:6a0746a1ec1aef3e7ec53868f220ff6e389f6f8ef87a01d77c96807de94ca2aa https://registry.ollama.ai/v2/library/llama3/blobs/sha256:6a0746a1ec1aef3e7ec53868f220ff6e389f6f8ef87a01d77c96807de94ca2aa