search

solo-io / wasm

Web Assembly tools and SDKs for extending cloud-native infrastructure
Apache License 2.0
305 stars 39 forks source link

spec is incompatible with some registries and standard CLI tools #208

Closed dgn closed 3 years ago

dgn commented 3 years ago

wasme containers cannot be pulled/pushed by standard tooling such as docker or podman:

$ docker pull webassemblyhub.io/haruband/simple-filter:v0.1
v0.1: Pulling from haruband/simple-filter
aa27b32e866b: Pulling fs layer
bc694dc922ea: Downloading [==>                                                ]  3.302kB/63.13kB
invalid rootfs in image configuration

$ podman pull webassemblyhub.io/haruband/simple-filter:v0.1
Trying to pull webassemblyhub.io/haruband/simple-filter:v0.1...
Getting image source signatures
Copying blob bc694dc922ea done  
Copying blob aa27b32e866b done  
Copying config aa27b32e86 done  
Writing manifest to image destination
Storing signatures
  unsupported docker v2s2 media type: ""
Error: unable to pull webassemblyhub.io/haruband/simple-filter:v0.1: Error committing the finished image: error parsing manifest: unsupported docker v2s2 media type: ""

Also, even if using the wasme CLI, images cannot be pushed to AWS ECR (see https://github.com/solo-io/wasm/issues/129) or to quay.io:

$ wasme push quay.io/dgrimm/test:latest
INFO[0000] Pushing image quay.io/dgrimm/test:latest    
Error: All attempts fail:
#1: oras push failed: failed commit on ref "manifest-sha256:abc3813ff7d5af3ff5dc06bbd12b0d4b9501d09cd06e106140678c212c714248": unexpected status: 415 UNSUPPORTED MEDIA TYPE

This seems to be caused by the custom mediaTypes used in the OCI manifest spec.

Do we really count on these vendors to implement support for the custom mediaTypes in the spec? Why not use existing mediaTypes and labels to package the information required for extensions? That would allow users to stick with their CLI tools (and standard Dockerfiles), and push to the registries they already use, with no interoperability issues.

Sodman commented 3 years ago

While these images are OCI images, they're not quite docker images. The wasme CLI should be used to push and pull them. While you might imagine docker pull should work, it gets more complicated for commands like docker run where it's expecting a container but it receives a WASM module. When it comes to repositories and CLIs, some vendors have chosen to whitelist the mediaTypes they accept, and they likely have some strong assumptions throughout their own ecosystems that depend on those assumptions. For example ECR is Elastic Container Registry, so any tooling around that which assumes its pulling docker containers might have some unexpected behaviors when they receive wasm modules instead.

WASM images also have some unique challenges of their own, for example determining which runtimes are currently compatible with a given image. These and other problems are what we're trying to solve for with webassemblyhub.io - giving users a place dedicated to WASM images specifically.

Happy to hop on a call to discuss further!

dgn commented 3 years ago

I think we're talking about different things here. In my opinion, the biggest advantage to using an OCI format is the compatiblity with existing tooling like registries and CLI tools. Due to the usage of custom mediaTypes, this compatibility is not given. We cannot expect external vendors to support any arbitrary mediaType there could be in an OCI manifest. At the same time, I don't see a need for these custom mediaTypes, as the required data can be packaged in a standards-compliant mediaType as well (ie a tarball)