Envoy should be able to describe (Wasm) ABIs it supports

[yskopets]

Title: Envoy should be able to describe (Wasm) ABIs it supports

Context:

• consider a use case of a marketplace with Wasm extensions for Envoy
• from a user perspective, it's important to know whether a given extension is supported by a particular Envoy distribution he is using right now
• the assumption is that an answer like "use Envoy vX.Y.Z+" is not good enough (a user might be using either a 3rd party closed source Envoy distro, or a nightly build of upstream Envoy, etc)

Proposal:

• Add to Envoy the ability to describe (Wasm) ABIs it supports
• The initial focus is on the offline scenario where it is a user, or a marketplace, or a vendor, who needs to know this info
• The online scenario, where it is a Control Plane who needs to know this info, is currently out of scope

Example use cases:

• a user runs envoy --list-abis to get the precise list of (Wasm) ABIs this particular Envoy distro supports, e.g.

  envoy:
  version:
   ... 
  abis:
    v1:
      host:
        network_filter:
          ...
        http_filter:
          ...
        access_log:
          ...
        http_client:
          proxy_http_call:   SUPPORTED
        grpc_client:
          proxy_grpc_call:   SUPPORTED
          proxy_grpc_stream: SUPPORTED
          proxy_grpc_send:   SUPPORTED
        tcp_client:
          ...
        udp_client:
          ...
      ...
      extension:
        network_filter:
          ...
        http_filter:
          ...
        access_log:
          ...
        http_client:
          proxy_on_http_call_response:           SUPPORTED
        grpc_client:
          proxy_on_grpc_create_initial_metadata: SUPPORTED
          proxy_on_grpc_receive:                 DEPRECATED
        tcp_client:
          ...
        udp_client:
          ...
      ...

• a user then comes to a marketplace site, provides info from above and gets back the exact info on what extensions (and their versions) are applicable to him
• a marketplace runs envoy --list-abis on popular Envoy distributions (e.g. upstream, getenvoy, istio, solo, etc) to provide compatibility info in advance
• a vendor of a closed source Envoy distribution runs envoy --list-abis on his own binaries and submits this info to a marketplace in order to be present in the compatibility matrix for each extension in the catalog

[Shikugawa]

I think it is easy to implement this with a dynamic approach by fetching symbols with wasm_vm_->registerCallback and any other functionality to provide ABI support.

[mattklein123]

Agreed. This is similar also for Envoy expressing what extensions it supports. cc @jpeach

[yskopets]

Here is the re-worked schema that is meant to meet the following requirements:

• it should support not only proxy-wasm but also Envoy-specific ABIs (see 3 examples below)
• it should have the same form both for proxy-wasm and Envoy-specific ABIs (unlike in the original proposal)
• it should (minimally) support dependencies between ABIs, in particular, to make extensions reuse the same ABI for memory management
• it should be simple (given that the entire Wasm support in Envoy is new and might evolve significantly)
  • e.g., every patch version of proxy-wasm ABI will be described separately, which means a lot of duplication. However, that is simple

'@type': type.googleapis.com/envoy.data.abi.v3alpha.ABI
abis:
- name: proxy-wasm:0.1.0 # reference name 
  description: Wasm for Proxies (github.com/proxy-wasm/spec)

  abi: proxy-wasm # name of the ABI without a version component
  version:
    semantic:
      major: 0
      minor: 1
      patch: 0

  # ABI provided by host.
  host:
    # Module through which host ABI will be exposed to the extension.
    module: env

    functions: # use Wasm terms
    - name: proxy_log
      status: IMPLEMENTED # by default

    - name: proxy_get_configuration
      status: DEPRECATED # to flag the extensions that are still using this function

    - name: proxy_get_buffer

    - name: proxy_open_grpc_stream

  # ABI expected from the extension.
  extension:  
    functions: # use Wasm terms
    - name: proxy_abi_version_0_1_0
      implementation: MANDATORY # Envoy will reject those extensions that are not exposing this function

    - name: proxy_on_memory_allocate
      implementation: MANDATORY

    - name: proxy_on_vm_start
      implementation: OPTIONAL # by default

    - name: proxy_on_new_connection

- name: envoy.extensions.memory_management.basic.v1alpha0 # reference name 
  description: Wasm ABI for memory management in cross-boundary calls, i.e. Envoy => Extension and Extension => Envoy

  abi: envoy.extensions.memory_management.basic # name of the ABI without a version component
  version:
    grpc:
      major: v1
      stability: alpha0

  host: {} # ABI implemented only by the extension

  extension:
    functions:
    - name: envoy_extensions_memory_management_basic_v1alpha0_alloc # functions exposed by the extension must have fully qualified names
      implementation: MANDATORY

    - name: envoy_extensions_memory_management_basic_v1alpha0_realloc
      implementation: OPTIONAL

    - name: envoy_extensions_memory_management_basic_v1alpha0_free
      implementation: MANDATORY

- name: envoy.extensions.tracers.v1alpha0 # reference name 
  description: Wasm ABI for tracing providers

  abi: envoy.extensions.tracers # name of the ABI without a version component
  version:
    grpc:
      major: v1
      stability: alpha0

  memory:
    management: # memory management in cross-boundary calls, i.e. Envoy => Extension and Extension => Envoy
      abi: envoy.extensions.memory_management.basic.v1alpha0 # announces ABI for memory management (implicit dependency between ABIs)

  host: {} # ABI implemented only by the extension

  extension:
    functions:
    - name: envoy_extensions_tracers_v1alpha0_tracer_create # functions exposed by the extension must have fully qualified names
      implementation: MANDATORY

    - name: envoy_extensions_tracers_v1alpha0_tracer_destroy
      implementation: MANDATORY

    - name: envoy_extensions_tracers_v1alpha0_span_start
      implementation: MANDATORY

    - name: envoy_extensions_tracers_v1alpha0_span_set_operation

    - name: envoy_extensions_tracers_v1alpha0_span_set_tag

    - name: envoy_extensions_tracers_v1alpha0_span_finish
      implementation: MANDATORY

- name: envoy.sdk.udp.client.v1alpha0 # reference name 
  description: Wasm ABI for Envoys-specific features, such as UDP client

  abi: envoy.sdk.udp.client # name of the ABI without a version component
  version:
    grpc:
      major: v1
      stability: alpha0

  memory:
    management: # memory management in cross-boundary calls, i.e. Envoy => Extension and Extension => Envoy
      abi: envoy.extensions.memory_management.basic.v1alpha0 # announces ABI for memory management (implicit dependency between ABIs)

  host:
    # Module through which host ABI will be exposed to the extension.
    module: envoy.sdk.udp.client.v1alpha0

    functions:
    - name: create_client
      status: NOT_IMPLEMENTED # explicitly indicate that this ABI is not implemented yet

    - name: destroy_client
      status: NOT_IMPLEMENTED

    - name: send_message
      status: NOT_IMPLEMENTED

  extension: {} # ABI implemented only by the host