httproutes removal and gateway validation together cause circular dependencies.

[nithyatsu]

Overview

Radius is able to create services as part of rendering containers today, without the need to define a HTTP Route. This allows for higher degree of parallelization in deploying resources, since by defining a container's connection's source to be a URL, we remove the dependency it has on another Radius resource. Radius gateways are implemented using Contour HTTPProxy. As part of deploying gateways, today we validate the HTTPProxy objects that are created. One of the validations is to make sure the HTTPProxy is pointing to an existing service.

The ability to create services without HTTPRoutes introduces a new scenario in deployment of applications with gateways.

Current State

Since we do not have any dependency between the rendering of gateway and that of container, it is possible that the gateway rendering fails validation since the service it depends on is still being created. As a result, rad deploy gateway.bicep can fail.

gateway.bicep

import kubernetes as kubernetes {
  kubeConfig: ''
  namespace: 'default'
}
import radius as radius

@description('Specifies the location for resources.')
param location string = 'local'

@description('Specifies the environment for resources.')
param environment string

@description('Specifies the port for the container resource.')
param port int = 3000

@description('Specifies the image for the container resource.')
param magpieimage string

@description('Specifies tls cert secret values.')
@secure()
param tlscrt string
@secure()
param tlskey string

resource app 'Applications.Core/applications@2023-10-01-preview' = {
  name: 'corerp-resources-gateway-sslpassthrough'
  location: location
  properties: {
    environment: environment
  }
}

resource gateway 'Applications.Core/gateways@2023-10-01-preview' = {
  name: 'ssl-gtwy-gtwy'
  location: location
  properties: {
    application: app.id
    tls: { 
      sslPassthrough: true 
    } 
    routes: [
      {
        destination: 'https://ssl-gtwy-front-ctnr'# change line to this to get the bicep to deploy successfully-> destination: 'https://${frontendContainer.name}:${frontendContainer.properties.container.ports.web.port}'
      }
    ]
  }
}

resource frontendContainer 'Applications.Core/containers@2023-10-01-preview' = {
  name: 'ssl-gtwy-front-ctnr'
  location: location
  properties: {
    application: app.id
    container: {
      image: magpieimage
      env: {
        TLS_KEY: tlskey
        TLS_CERT: tlscrt
      }
      ports: {
        web: {
          containerPort: port
          port: 443
        }
      }
      readinessProbe: {
        kind: 'tcp'
        containerPort: port
      }
    }
  }
}

Workaround / Potential fixes

In simple scenarios we can get around the issue by injecting a dependency as below destination: 'https://${frontendContainer.name}:${frontendContainer.properties.container.ports.web.port}'

However, this might introduce cyclic dependencies in some scenarios.

A potential fix is to check for the validation error, and use the specific details ( if the failure is due to "Service Not Found") to ignore errors that we know to be due to the new model.

AB#9673

[shalabhms]

triage: This is not blocker for nov release. Removing the tag.

[rynowak]

A potential fix is to check for the validation error, and use the specific details ( if the failure is due to "Service Not Found") to ignore errors that we know to be due to the new model.

@vinayada1 @nithyatsu do we know how easy this is to do? How good is the error reporting from contour? If we special-case this error are we likely to miss other cases?

[AaronCrawfis]

Error message from recreating:

apiVersion: projectcontour.io/v1
kind: HTTPProxy
metadata:
  creationTimestamp: "2023-10-10T22:27:00Z"
  generation: 1
  labels:
    app.kubernetes.io/managed-by: radius-rp
    app.kubernetes.io/name: ssl-gtwy-gtwy
    app.kubernetes.io/part-of: corerp-resources-gateway-sslpassthrough
    radapp.io/application: corerp-resources-gateway-sslpassthrough
    radapp.io/resource: ssl-gtwy-gtwy
    radapp.io/resource-type: applications.core-gateways
  name: ssl-gtwy-gtwy
  namespace: default-corerp-resources-gateway-sslpassthrough
  resourceVersion: "1841"
  uid: 8aabf4e0-4eaa-40b6-bff8-07e869e050b7
spec:
  includes:
  - conditions:
    - prefix: /
    name: ssl-gtwy-front-ctnr
  tcpproxy:
    services:
    - name: ssl-gtwy-front-ctnr
      port: 443
  virtualhost:
    fqdn: corerp-resources-gateway-sslpassthrough
    tls:
      passthrough: true
status:
  conditions:
  - errors:
    - message: 'Spec.TCPProxy unresolved service reference: service "default-corerp-resources-gateway-sslpassthrough/ssl-gtwy-front-ctnr"
        not found'
      reason: ServiceUnresolvedReference
      status: "True"
      type: TCPProxyError
    lastTransitionTime: "2023-10-10T22:27:00Z"
    message: At least one error present, see Errors for details
    observedGeneration: 1
    reason: ErrorPresent
    status: "False"
    type: Valid
  currentStatus: invalid
  description: At least one error present, see Errors for details
  loadBalancer: {}

[willdavsmith]

Design notes: let's capture the ServiceUnresolvedReference error reason and not throw an error in that case.