Cloud Native Application Bundles (CNAB) support

[MedAnd]

Reading the documentation and faq for mesh and cannot seem to find examples of how I would run existing docker compose images.

As the 6.3 preview cluster is a regular Service Fabric cluster, wondering what is the longer term plan for manifest based applications & support for docker compose etc? If I wanted to deploy docker Linux images from hub.docker.com, is this supported (documented)?

Thx.

[mikkelhegn]

We only support docker container images at this point, but you have to describe the deployment in ARM. Using Docker Compose as the format is not supported.

You can see an example of how to deploy an existing container from Docker Hub here: https://github.com/Azure/service-fabric-mesh-preview-pr/blob/master/docs/conceptual-docs/application-deployment-quickstart.md - this is the ARM template being used, and it references a container image from Docker Hub: https://seabreezequickstart.blob.core.windows.net/templates/quickstart/sbz_rp.linux.json

[jmeijrink]

An intermediate way could be a docker compose to application template converter...

[MedAnd]

Hand rolling ARM json is always fun :grin: can we assume support for docker compose will come in future releases + tooling in VS/Code? Being able to easily deploy existing docker images (for example a db/search cluster) in Mesh, with state & location transparency would be a killer feature...

Any idea if the following docker compose Linux cluster template is "portable" to ARM Mesh and how would I express the dependency and order between master, worker & manager containers?

services: master: container_name: "mc_master" image: 'xyz' ports: ["${PORT:-8080}:8080"] worker: container_name: "mc_worker" image: 'xyz' depends_on: { manager: { condition: service_healthy } } manager: container_name: "mc_manager" image: 'abc' volumes: ['/var/run/docker.sock:/var/run/docker.sock'] depends_on: { master: { condition: service_healthy } }

[mikkelhegn]

In the preview tooling we released for Windows and Visual Studio, the tooling defines the application in a resource YAML format, which is how we would encourage users to describe their deployments. E.g.

application: schemaVersion: 0.0.1 name: VotingApp description: VotingApp description. services:

• name: VotingWeb description: VotingWeb description. osType: Windows codePackages:
  • name: VotingWeb image: votingweb:dev endpoints:
    • name: VotingWebListener port: 20005 environmentVariables:
    • name: ASPNETCORE_URLS value: http://+:20005
    • name: Voting_BackendHostName value: votingdata
    • name: Voting_BackendHostPort value: 20000 resources: requests: cpu: 0.5 memoryInGB: 1 replicaCount: 1 networkRefs:
  • name: VotingAppNetwork

network: schemaVersion: 0.0.1 name: VotingAppNetwork description: VotingAppNetwork description. addressPrefix: 10.0.0.4/22 ingressConfig: layer4:

• name: VotingWebIngress publicPort: 20005 applicationName: VotingApp serviceName: VotingWeb endpointName: VotingWebListener

That tooling packace has a conversion tool from this format to the JSON and ARM JSON for Service Fabric and Mesh. Our plan is to provide this directly in cli, so users only have to deal with the YAML definitions.

To the compose question, we have limited support for Compose in Service Fabric today, and our experience based on that is that it's hard to adopt a definition, which is built around the capabilities of docker and not Service Fabric. So we are not likely to build support for compose directly in Mesh.

Considering the above example, it should be an easy transition from compose to the resource YAML.

See here as well: https://github.com/Azure/service-fabric-mesh-preview-pr/issues/101

[MedAnd]

Hi @MikkelHegn,

Thought the below article from Docker was interesting & relevant, it seems the industry and Microsoft are moving towards supporting Docker Compose and Kubernetes YAML in the same cluster:

This allows an organization that already has a containerized Windows application with Docker Compose to migrate the application to Kubernetes simply by re-deploying it.

https://blog.docker.com/2018/06/kubernetes-on-windows-server-with-docker-ee/

Given the open source community are building and packaging whole application clusters (complex multi-container solutions), wondering what the vision is for Mesh when customers want to use off the shelf Linux Docker Stream Processing & Analytics containers alongside in-house developed .Net Core Windows containers?

[vipul-modi]

All of the sample templates published in the "How-to" documentation use the images from dockerhub. Mesh applications can be authored using YAML files in VS. The command line tooling support is coming.

[MedAnd]

Hi @VipulM-MSFT ... believe this is a longer term goal as per comments from @ChackDan but being able to use existing Compose solutions without having to write or know any Mesh YAML would be the ideal... even if this is supported via a conversion process built-into the Mesh tooling.

Imagine in VSO a step definition which allows one to pick any Linux Docker Compose solution and then deploy this to Mesh... followed by another step definition which deploys a .Net Core container (Nano Server, Alpine Linux, Minimal Ubuntu) and all this in the same network and cluster...

To borrow from the above quote, this would allows organizations to migrate to Mesh simply by re-deploying :relaxed:

[mikkelhegn]

@MedAnd - If the goal is some sort of platform agnostic definition languages, I would consider Terraform or Pulumi providers to be better candidates, as these are designed to be that, and not a descriptive language for a specific orchestration / container run-time. Our experience with compose support in Service Fabric, is that it provides an easy move, but you very quickly want to get to the primitives / specifics of the platform you are running in, but which cannot be described in the common descriptive language (in this case concepts like stateful services with replicas etc.)

Maybe a client tool to transform docker-compose to SF Mesh YAML is all that's needed for a one-time conversion?

[MedAnd]

@MikkelHegn - I'm following Terraform (at present still lacks on-prem and cross cloud Service Fabric support) and Pulumi (have been in touch with the guys to recommend Service Fabric support) closely :relaxed: but above is more about how can I run existing docker + compose container solutions on Service Fabric Mesh without any developer intervention and without having to fork/own the open source images and yml where not required.

I'm thinking wouldn't it be a great adoption story if Mesh allowed organizations that already have containerized Windows & Linux applications with Docker Compose to migrate these applications to Mesh simply by re-deploying?

So when the open source community release updates to these docker + compose applications, I want to be able to pull the latest images and do a rolling upgrade across my Mesh. If I use a client tool to first transform and then publish my own forked Mesh yml version of the application, wondering how this would work with CI/CD pipelines... as in I would have to git my transformed Mesh version of docker-compose first, is it fully 100% auto transformed etc?

If we don't end up supporting Compose at all, I'm worried that customers who need slightly more complex container functionality and who want to leverage existing open source solutions will look elsewhere.

[mikkelhegn]

@MedAnd - I don't disagree that it would be a great feature for quickly trying out Mesh, but I see it more as a convenience feature at this point. There are a few app-level definition languages out there, but they are all tied to one specific orchestrator: Compose to Docker, Helm to Kubernetes and SF Mesh / ARM to Mesh, and I think they should continue to be that.

A few alike from the Kubernetes World as inspiration on how Compose has been supported there: http://kompose.io/ and https://github.com/kelseyhightower/compose2kube

[MedAnd]

Hey @MikkelHegn - updating this request to Cloud Native Application Bundles (CNAB) support... any chance this is already on the radar for Mesh or Service Fabric?

Adding @technosophos to the conversation ☺️