Network vs Gateway

[guibirow]

Looking through the SF schemas, I've noticed that the Microsoft.ServiceFabricMesh/networks template has changed from apiVersion 2018-07-01-preview to 2018-09-01-preview

In the apiVersion 2018-07-01-preview, the network could expose the services via 'ingressConfig' properties, in the 2018-09-01-preview we don't see that anymore.

2018-07-01-preview

{
  "name": "string",
  "type": "Microsoft.ServiceFabricMesh/networks",
  "apiVersion": "2018-07-01-preview",
  "location": "string",
  "tags": {},
  "properties": {
    "description": "string",
    "addressPrefix": "string",
    "ingressConfig": {
      "qosLevel": "Bronze",
      "layer4": [
        {
          "name": "string",
          "publicPort": "integer",
          "applicationName": "string",
          "serviceName": "string",
          "endpointName": "string"
        }
      ]
    }
  }
}

2018-09-01-preview

{
  "name": "string",
  "type": "Microsoft.ServiceFabricMesh/networks",
  "apiVersion": "2018-09-01-preview",
  "tags": {},
  "location": "string",
  "properties": {
    "description": "string",
    "kind": "string"
  }
}

**We can see in the new api version the the gateway definition,

is the plan to Gateway replace the 'ingressConfig' in the future?**

[aloneguid]

I think gateway resource will be the new one where you can define routing rules, something similar to Azure Application Gateway. It will still need to rely on network resource and will route traffic to it. However, as long as I understand network resource doesn't understand HTTP routing or SSL termination, it's just a virtual network that routes you to SF internal load balancer. In other words gateway understands layer 7 routing, whereas network can only do level 4 routing. Correct me if I'm wrong.

[guibirow]

I got the benefits of the gateway resource.

The only thing I didn't understand is if ingress configuration in a network was done by mistake and they are trying to fix that by replacing it with Gateway or they are planning to keep both. Because from the schema, does not have any ingress configuration in the network anymore.

IMHO, I would expect to have a Network and a LoadBalancer resource, and if I have multiple services types, I could add multiple LB as L4 routing(one to each type splitting the load among the service instances) and we could implement our own gateways using other solutions like nginx, traefikk and so on.

Currently I see the network resource doing multiple functions, and to implement the scenario above, we would need multiple networks per service.

It is not clear if the gateway will be entirely based on envoy as advertised, or you can configure the gateway to be a L4 as a Load Balancer or a L7 based on envoy, that I assume it won't because envoy also handle L4 load.

Doing a gateway based purely on envoy is a naive idea and only make SF Mesh tight coupled to this solution, even though we have many alternatives available.

Another thing I can't understand on this network resource is the security configuration, Lets say I have a networkA that can talk to networkB, networkB can talk to networkC, but networkA can't talk to networkC. No documentation or sample has been provided talking about it, will it be a network configuration, will they bring the NSG as a resource, what approach will be taken.

SF Mesh concept seems so nice, but the solution seems very raw right now that is difficult to even consider any production workload on it.

[aloneguid]

the docs seems to be updated now with now it works, I guess this issue can be closed

[mattrowmsft]

@sridmad in case he wants to correct anything I say or add to it. Gateways were an evolution from where we started. Just networks wouldn't allow for multiple services to expose the same port for example. Networks now represent more closely the concept of virtual networks from Microsoft.Networking namespace. In fact, the next big thing you will see with networks is bring your own VNET, where the two concepts get married together and ties in the broader Azure ecosystem. What people need out of the system often is slightly different, so the best thing you can do is be vocal here about the pieces that are missing and the pieces that need polished to light up your scenarios.

[guibirow]

My only fear is to expose any service to the internet we will be dependent on this Gateway, in case a the service has any limitation and we want to replace it with a custom made Gateway, this gateway would still be in front of any solution.

From the docs, it says it will be an envoy proxy, for both L4 and L7, L4 makes it more flexible, but can still be a point of failure and a bottleneck in the system.

The same concerns I have to vnet to vnet configuration.

[aloneguid]

@guibirow I don't think you have to use gateway resource, you can have your custom gateway pointing to network resource.

[mattrowmsft]

You need to have gateway to expose anything to public internet (or between private networks in future). I imagine it would look like public -> azure LB (l4) -> mesh gateway (l7/l4) -> user gateway (l7/l4) -> user service.

[guibirow]

@aloneguid on the apiVersion 2018-09-01-preview the ingress configuration has been removed from the network and the only option to expose services will be via the gateway. The docs are not updated to reflect this, it says you can use a network to expose applications to the internet, but it is misleading, as it will require the gateway in front of the service.

@mattrowmsft, This was my original concern, it makes me think how reliable and scalable this gateway will be and if it will be a bottleneck to the application.

[aloneguid]

I wasn't aware that network can't be exposed anymore, that's definitely a showstopper for orgs that want to join services to an existing gateway. Or rather a limitation. However if we are not billed for gateway (not sure about this) and it scales well it doesn't matter.

[SamirFarhat]

@aloneguid : Why is this a stopper ?

[guibirow]

@SamirFarhat This is a limitation and could be a stopper for some.

The main issue here, is that we are adding points of failure to the solution, and the overall performance will be determined by the slowest element.

Also the other concern is the network to network communication will depend on this gateway.

Another issue is that it becomes a single entry point for the service, if I want to do the load balancing in the client for example, I couldn't.

You could argue that a solution could be adding multiple gateways, it is a valid option(if possible) but will increase the cost to solve a problem that should not exist on first place.

[markheath]

I've been playing with the 2018-09-01-preview API version and gateways, but I can't work out how I can expose two endpoints. I've tried with http and tcp definitions, and even with two gateways (both of which end up with the same IP address), but in all cases, only one of the exposed services is reachable. Is this something that ought to be possible, and if so, is there a demo template I can examine?

[guibirow]

@markheath according to the latest release notes and to this docs, it should be possible.

I didn't have the chance to play around with it on multiple services, but should be very straight forward, Something like this:

{  
  "apiVersion": "2018-09-01-preview",
  "name": "ingressGatewayLinux",
  "type": "Microsoft.ServiceFabricMesh/gateways",
  "location": "[parameters('location')]",
  "dependsOn": [  
    "Microsoft.ServiceFabricMesh/networks/meshNetworkLinux"
  ],
  "properties": {  
    "description": "Service Fabric Mesh Gateway for Linux mesh samples.",
    "sourceNetwork": {  
      "name": "Open"
    },
    "destinationNetwork": {  
      "name": "[resourceId('Microsoft.ServiceFabricMesh/networks', 'meshNetworkLinux')]"
    },
    "http": [  
      {  
        "name": "web",
        "port": 80,
        "hosts": [  
          {  
            "name": "*",
            "routes": [  
              {  
                "name": "helloRoute", //this is missing in the docs
                "match": {  
                  "path": {  
                    "value": "/hello",
                    "rewrite": "/",
                    "type": "Prefix"
                  }
                },
                "destination": {  
                  "applicationName": "meshAppLinux",
                  "serviceName": "helloWorldService",
                  "endpointName": "helloWorldListener"
                }
              },
              {  
                "name": "counterRoute", //this is missing in the docs
                "match": {  
                  "path": {  
                    "value": "/counter",
                    "rewrite": "/",
                    "type": "Prefix"
                  }
                },
                "destination": {  
                  "applicationName": "meshAppLinux",
                  "serviceName": "counterService",
                  "endpointName": "counterServiceListener"
                }
              }
            ]
          }
        ]
      }
    ]
  }
}

---

Note: keep in mind that the route in the docs, misses the name of the route, as noted by @aloneguid on https://github.com/Azure/service-fabric-mesh-preview/issues/308

[markheath]

@guibirow thanks, I did try this option, but my containers (which are websites) don't work well with a prefix on their route (messes up getting their css etc). I want to expose the two services on two different port numbers, and can't find a way to make that happen.

[guibirow]

If you pay attention, the Routing has a URL rewrite mechanism ("rewrite": "/"), where you can host them on something like http://domain.com/api/users and the rewrite send to the service only http://domain.com/users

To host the services on different ports, you could try creating multiple hosts. Something like this:

    "http": [  
      {  
        "name": "web",
        "port": 80,
        "hosts": [{... route info here...}]
      },
      {  
        "name": "api",
        "port": 81,
        "hosts": [{... route info here...}]
      }
    ]

if you have problems with these approaches, I would recommend you opening a separate issue to track the problem!

[markheath]

Yes that is exactly what I tried, but couldn't get it working. I'll give it another go later and if it still doesn't work I'll raise an issue. Here's one of my recent attempts which deploys but only exposes one service (port 80) - https://github.com/markheath/azure-docker-talk/blob/4825ac7501f2bd7a38a537bbbe5141db7b23b293/azurethamesvalley/sfmesh-example-voting-app.json.

[SamirFarhat]

@mattrowmsft Is the new Gateway a managed Envoy proxy?

[mattrowmsft]

Yes, sorry about not being more active on thread. @sridmad and Artur who I'm getting to join the thread should be able to explain what exists today and where things are headed better.

[mattrowmsft]

• @kavyako as well

[arturenault]

The understanding in this thread seems to be correct about what a gateway is and how it works--short answer is yes, it replaces the IngressConfig in networks, with added layer 7 routing functionality. If you use l4 routing (the "tcp" property), behavior should be exactly the same.

@markheath I took a look at your sample and I believe it should be valid, except potentially for the fact that you have two ports pointing to the same endpoint. @sridmad and @kavyako can confirm if this is valid.

Additionally one thing I've noticed in this thread is plenty of routes in the form:

        "http": [
          {
            "name": "vote",
            "port": 80,
            "hosts": [
              {
                "name": "*",
                "routes": [
                  {
                    "name":  "voteRoute",
                    "match": {
                      "path": {
                        "value": "/",
                        "rewrite": "/",
                        "type": "Prefix"
                      }
                    },
                    "destination": {
                      "applicationName": "votingApp",
                      "serviceName": "voteService",
                      "endpointName": "VoteListener"
                    }
                  }
                ]
              }
            ]

While this is valid, you're not actually using any l7 routing capability, so you can do the same thing with this much more concise version:

        "tcp": [
          {
            "name": "vote",
            "port": 80,
            "destination": {
              "applicationName": "votingApp",
              "serviceName": "voteService",
              "endpointName": "VoteListener"
            }
          }
        ]

[guibirow]

@arturenault, @markheath has opened the #315 to track the issue with his application, regarding your comment, it is possible to route two ports to the same endpoint, the sample I posted on his issue worked this way.

@mattrowmsft @arturenault @sridmad @kavyako, I would like the bring the focus back to the main thread and discuss the future for the network and gateway, some points has been raised above but we didn't receive any official feedback from SF team.

Could you guys provide any information related to our concerns?

Are we limited to expose our services only via the gateway?

Are there any plan to expose the services behind a load balancer if we want to implement our own gateway?

Being it based on software(envoy) how reliable will it be? Can we expect any SLA from it?

Has any decision been taken related to the use of this gateway on communication between networks?