search

confluentinc / ide-sidecar

Sidecar application used by Confluent for VS Code, as a local proxy for Confluent Cloud, Confluent Platform and local Kafka clusters, to help users build streaming applications.
Apache License 2.0
3 stars 3 forks source link

Update connection `status` to work for local, CCloud and direct connections #123

Open rhauch opened 2 weeks ago

rhauch commented 2 weeks ago

Continuation of #96 and #122, and followup to #116 that added initial and limited direct connections (with GraphQL support) but did not support credentials nor unify the status to work for all kinds of connections. (The status is heavily oriented around CCloud connections.)

Note that direct connections do not yet support credentials, so direct connections will only work with Kafka clusters and SR that have no authentication. We'll track adding support for other credential types in subsequent issues:

rhauch commented 1 hour ago

Every Connection resource includes a spec field and status field. The spec is the user-defined information about the connection, and the status is the system-generated information. The status field is defined in the OpenAPI spec as the ConnectionStatus schema:

    ConnectionStatus:
      required:
      - authentication
      type: object
      properties:
        authentication:
          $ref: "#/components/schemas/Authentication"

with the Authentication schema:

    Authentication:
      required:
      - status
      type: object
      properties:
        status:
          $ref: "#/components/schemas/Status"
        requires_authentication_at:
          $ref: "#/components/schemas/Instant"
        user:
          $ref: "#/components/schemas/UserInfo"
        errors:
          $ref: "#/components/schemas/AuthErrors"

and the Status schema:

    Status:
      enum:
      - NO_TOKEN
      - VALID_TOKEN
      - INVALID_TOKEN
      - FAILED
      type: string

New connection resources always have an "initial" status of (in a simplified not-quite-JSON representation):

status: {
  authentication: {
    status: NO_TOKEN
  }
}

and successfully-connected CCloud connections have this status:

status: {
  authentication: {
    status: SUCCESS,
    user_info: ...
    errors: ...
  }
}

The errors field is used only if the status is FAILED or INVALID_TOKEN.

This design was added for CCloud connections and the structure and terminology of the Status enum literals is very much oriented around CCloud and/or OAuth.

Meanwhile, the spec for connection resources is defined in the OpenAPI spec as:

    ConnectionSpec:
      description: The connection details that can be set or changed.
      type: object
      properties:
        id:
          description: The unique identifier of the connection resource.
          maxLength: 64
          minLength: 1
          type: string
        name:
          description: The user-supplied name of the connection resource.
          maxLength: 64
          type: string
        type:
          description: The type of connection resource.
          type: string
          allOf:
          - $ref: "#/components/schemas/ConnectionType"
        ccloud_config:
          description: The details for connecting to CCloud.
          type: object
          allOf:
          - $ref: "#/components/schemas/CCloudConfig"
        local_config:
          description: The details for connecting to Confluent Local.
          type: object
          allOf:
          - $ref: "#/components/schemas/LocalConfig"

Note the ccloud_config and local_config objects used for CCloud and local connections, respectively. The ccloud object, with the CCloud organization is to be used. The CCloudConfig schema is defined as:

    CCloudConfig:
      description: Configuration for Confluent Cloud connections
      required:
      - organization_id
      type: object
      properties:
        organization_id:
          description: The identifier of the CCloud organization to use. The user's
            default organization is used when absent.
          maxLength: 36
          minLength: 36
          type: string

As part of adding support for direct connections, the Connection spec was expanded to include a spec.kafka_cluster and spec.schema_registry:

    ConnectionSpec:
      description: The connection details that can be set or changed.
      type: object
      properties:
        id:
          description: The unique identifier of the connection resource.
          maxLength: 64
          minLength: 1
          type: string
        name:
          description: The user-supplied name of the connection resource.
          maxLength: 64
          type: string
        type:
          description: The type of connection resource.
          type: string
          allOf:
          - $ref: "#/components/schemas/ConnectionType"
        ccloud_config:
          description: The details for connecting to CCloud.
          type: object
          allOf:
          - $ref: "#/components/schemas/CCloudConfig"
        local_config:
          description: The details for connecting to Confluent Local.
          type: object
          allOf:
          - $ref: "#/components/schemas/LocalConfig"
        kafka_cluster:
          description: "The details for connecting to a CCloud, Confluent Platform,\
            \ or Apache Kafka cluster."
          type: object
          allOf:
          - $ref: "#/components/schemas/KafkaClusterConfig"
        schema_registry:
          description: The details for connecting to a Schema Registry.
          type: object
          allOf:
          - $ref: "#/components/schemas/SchemaRegistryConfig"

Now, local connections include a local_config object, CCloud connections include a ccloud_config object, and direct connections include a kafka_cluster object (when directly connecting to a Kafka cluster) and/or a schema_registry object (when directly connecting to a Schema Registry).

The status needs to change so that we can adequately reflect the connected-ness (and other errors) for each of the one or more local_config, ccloud_config, kafka_cluster and/or schema_registry objects in the spec. To do this, we will slowly evolve away from the status.authentication object and instead introduce an object in status for each spec configuration object:

    ConnectionStatus:
      required:
      - authentication
      type: object
      properties:
        ccloud:
          $ref: "#/components/schemas/CCloudStatus"
        kafka_cluster:
          $ref: "#/components/schemas/KafkaClusterStatus"
        schema_registry:
          $ref: "#/components/schemas/SchemaRegistryStatus"
        authentication:
          $ref: "#/components/schemas/Authentication"

where CCloudStatus is essentially the same as Authentication:

    CCloudStatus:
      description: The status related to CCloud.
      required:
      - state
      type: object
      properties:
        state:
          description: The state of the connection to CCloud.
          type: string
          allOf:
          - $ref: "#/components/schemas/ConnectedState"
        requires_authentication_at:
          description: "If the connection's auth context holds a valid token, this\
            \ attribute holds the time at which the user must re-authenticate because,\
            \ for instance, the refresh token reached the end of its absolute lifetime."
          type: string
          allOf:
          - $ref: "#/components/schemas/Instant"
        user:
          description: "Information about the authenticated principal, if known."
          type: object
          allOf:
          - $ref: "#/components/schemas/UserInfo"
        errors:
          description: Errors related to the connection to the Kafka cluster.
          type: object
          allOf:
          - $ref: "#/components/schemas/AuthErrors"

and the KafkaClusterStatus and SchemaRegistry status are defined similarly:

    KafkaClusterStatus:
      description: The status related to the specified Kafka cluster.
      required:
      - state
      type: object
      properties:
        state:
          description: The state of the connection to the Kafka cluster.
          type: string
          allOf:
          - $ref: "#/components/schemas/ConnectedState"
        user:
          description: "Information about the authenticated principal, if known."
          type: object
          allOf:
          - $ref: "#/components/schemas/UserInfo"
        errors:
          description: Errors related to the connection to the Kafka cluster.
          type: object
          allOf:
          - $ref: "#/components/schemas/AuthErrors"
...
    SchemaRegistryStatus:
      description: The status related to the specified Schema Registry.
      required:
      - state
      type: object
      properties:
        state:
          description: The state of the connection to the Schema Registry.
          type: string
          allOf:
          - $ref: "#/components/schemas/ConnectedState"
        user:
          description: "Information about the authenticated principal, if known."
          type: object
          allOf:
          - $ref: "#/components/schemas/UserInfo"
        errors:
          description: Errors related to the connection to the Schema Registry.
          type: object
          allOf:
          - $ref: "#/components/schemas/AuthErrors"

The major difference in these schemas compared to Authentication is the new ones use state rather than status (to avoid having a status.authentication.status object) and to use more generic enum literals for ConnectedState:

    ConnectedState:
      enum:
      - NONE
      - ATTEMPTING
      - SUCCESS
      - EXPIRED
      - FAILED
      type: string

where:

The extension needs to transition from using status.authentication to status.ccloud. In the short term, both will be kept (the former will be generated from the latter) so the extension can migrate in the near future, as it adds support for status.kafka_cluster and status.schema_registry.

Once that migration has been completed and the extension is no longer using status.authentication, the status.authentication object will be removed.