dotnet / tye

Tye is a tool that makes developing, testing, and deploying microservices and distributed applications easier. Project Tye includes a local orchestrator to make developing microservices easier and the ability to deploy microservices to Kubernetes with minimal configuration.
MIT License
5.28k stars 520 forks source link

Support replacement tokens in environment variables #377

Open davidfowl opened 4 years ago

davidfowl commented 4 years ago

This is a sample of an application that has an opiniated view of what env variables to use for ports. It wants to reference the tye's generated ports in the env variables (see below for an example).

name: cornflake

services:
  - name: cornflake
    replicas: 3
    project: cornflake/cornflake.csproj
    bindings:
    - name: gateway
      port: 11111
    - name: silo
      port: 30000
    - name: web
      port: 5000
    env:
     - name: ENV_SILO_PORT
       value: ${service:cornflake:silo:port}
     - name: ENV_GATEWAY_PORT
       value: ${service:cornflake:gateway:port}

This brings up one interesting challenge:

TimHess commented 4 years ago

Steeltoe supports resolving property placeholders in configuration values

davidfowl commented 4 years ago

@TimHess that's pretty cool! We have a request to support something similar in configuration.

rynowak commented 4 years ago

We want to do a different proposal to this. We're going to make it possible to add an alias to a binding, so that the value it provides can be used in a configurable environment variable.

Kralizek commented 4 years ago

Totally a +1 to this issue!

This is the case I am facing:

  - name: elasticsearch
    image: docker.elastic.co/elasticsearch/elasticsearch:7.9.3
    env:
      - name: discovery.type
        value: single-node
    tags:
      - resources
      - search
    bindings:
      - containerPort: 9200
        protocol: http
        connectionString: "${host}:${port}"
  - name: search-resource-access
    project: ./ResourceAccess/SearchResourceAccess/src/SearchResourceAccess.Host/SearchResourceAccess.Host.csproj
    tags:
      - ras
      - search
    env:
      - name: SearchUri
        value: "${env:CONNECTIONSTRINGS__ELASTICSEARCH}"

Like specified in the opening post, this application expects the connection string to elasticsearch in the configuration key "SearchUri". Being an older project, I don't have the possibility to use Configure<TOptions> and refactoring this application to accomodate this need is beyond the scope of my research.

    "replicas": {
      "search-resource-access_80a73db4-1": {
        "dockerCommand": null,
        "containerId": null,
        "dockerNetwork": null,
        "dockerNetworkAlias": null,
        "name": "search-resource-access_80a73db4-1",
        "ports": null,
        "exitCode": null,
        "pid": 3392,
        "environment": {
          "DOTNET_ENVIRONMENT": "Development",
          "ASPNETCORE_ENVIRONMENT": "Development",
          "DOTNET_LOGGING__CONSOLE__DISABLECOLORS": "true",
          "ASPNETCORE_LOGGING__CONSOLE__DISABLECOLORS": "true",
          "DOTNET_ROOT": "C:\\Program Files\\dotnet",
          "DOTNET_MULTILEVEL_LOOKUP": "0",
          "PATH": "<omitted>",
          "SearchUri": "${env:CONNECTIONSTRINGS__ELASTICSEARCH}",
          "CONNECTIONSTRINGS__ELASTICSEARCH": "localhost:10551",
          "APP_INSTANCE": "search-resource-access_80a73db4-1"
        },
        "state": "ready"
      }

As you can see, SearchUri contains the raw string and not a processed one.

This feature would make it easier to support also non ASP.NET Core applications with Tye.

cjaliaga commented 3 years ago

The case @Kralizek mentioned it's interesting and I think would be really useful. Imagine for example we are using the same service for starting a SQL Server but we'd like to define in another service 2 connection strings, one per database:

services:
  - name: sql
    image: mcr.microsoft.com/mssql/server:2019-latest
    bindings:
      - port: 1433
        connectionString: Server=tcp:${host},${port};User ID=SA;Password=Password123;
  - name: api
    project: API/API.csproj
    env:
      - name: CONNECTIONSTRINGS__APPDB
        value: "${env:CONNECTIONSTRINGS__SQL};Database=App"
      - name: CONNECTIONSTRINGS__UNICORNSDB
        value: "${env:CONNECTIONSTRINGS__SQL};Database=Unicorns"
marinasundstrom commented 3 years ago

@cjaliaga I'm looking for something like thos. I would like to have one instance of MSSQL service with multiple databases/connection strings.

alexdresko commented 3 years ago

In my case, I want to do something like:

services:
- name: api
  project: src/microservices/api/api.csproj
- name: web
  executable: cmd
  args: '/c npm run start --prefix src/web -- --web-port ${this.port} --api-port ${api.port}`'

The goal is to let web start on any available port, but also tell web where api is. web is a pure HTML/CSS/JS application, and we rely on the framework's (aurelia, in this case) dev server during development because it provides the best experience.

alexdresko commented 3 years ago

I don't know what's involved to satisfy my requirements, but it seems like a relatively easy problem to solve. Would I be wasting my time if I forked it and took a stab?

marinasundstrom commented 3 years ago

I ended up overloading GetConnectionString() with an extension method that appends the $"Database={database}" to the original output.

davidfowl commented 3 years ago

We would accept a PR for this if we can flesh out the design in this issue.

alexdresko commented 3 years ago

I'm considering compiling a list of requirements based on what is in this and a few other issues. For example, I wonder if the token/variable system could be extended to support multiple tye "instances", as in this issue. <-- a neat feature, BTW.

^^^^^ Edited to correct the link... Sorry for any confusion. :(

tungphuong commented 3 years ago

@TimHess and @davidfowl, Steeltoe is working fine with me. These are some additional steps

oising commented 3 years ago

I finally got far enough into Tye to figure out I need this. There must be plenty of people out there who aren't in a position to modify the source of a dotnet application so that it can consume the specific env vars generated by Tye. We should be able to bind arbitrary, custom environment variables to any dynamically generated string in Tye (binding, connection string etc.)

Kralizek commented 2 years ago

Bump :)

Trying to get this to work without relying on the hardcoded host.docker.internal:9200

- name: opensearch
  image: opensearchproject/opensearch:1.2.4
  env:
    - name: discovery.type
      value: single-node
    - name: DISABLE_SECURITY_PLUGIN
      value: "true"
  bindings:
    - protocol: http
      containerPort: 9200

- name: opensearch-dashboards
  image: opensearchproject/opensearch-dashboards:1.2.0
  bindings:
    - protocol: http
      containerPort: 5601
  env:
    - name: OPENSEARCH_HOSTS
      value: '[\"http://host.docker.internal:9200\"]'
    - name: DISABLE_SECURITY_DASHBOARDS_PLUGIN
      value: "true"
JarradPosey commented 2 years ago

I agree this would be a useful feature, when dealing with non-dotnet services. My use case is to add a pgweb dashboard to postgres without the need to add a new connection each time I restart tye. I would need to configure the DATABASE_URL variable with the value from the connection string CONNECTIONSTRINGS__POSTGRES__PGWEB.

- name: postgres
  image: docker.io/library/postgres
  env:
  - name: POSTGRES_USERNAME
    value: postgres
  env_file:  
  - .secrets/.env
  volumes:
  - source: .postgres
    target: /var/lib/postgresql/data
  bindings:
  - name: pgweb
    protocol: tcp
    containerPort: 5432
    connectionString: postgres://${env:POSTGRES_USERNAME}:${env:POSTGRES_PASSWORD}@${host}:${port}/postgres?sslmode=disable
  - name: backend
    protocol: tcp
    containerPort: 5432
    connectionString: Server=${host};Port=${port};Username=${env:POSTGRES_USERNAME};Password=${env:POSTGRES_PASSWORD};

- name: pgweb
  image: docker.io/sosedoff/pgweb
  env:
    - name: SESSIONS
      value: '1'
    - name: DATABASE_URL
      value: ${env:CONNECTIONSTRINGS__POSTGRES__PGWEB}
  bindings:
    - protocol: http
      containerPort: 8081
Phiph commented 1 year ago

Okay I've done some digging....

so the service bindings don't get computed until the host is running them.

There is already support for some patterns - maybe we could extend it a bit to get close to what you're looking for.

This line is of particular interest: Looks like there was some thought put into it!

https://github.com/dotnet/tye/blob/75465614e67b5f1e500d1f8b1cb70af22c0c683e/src/Microsoft.Tye.Hosting/TokenReplacement.cs#L68

However, I appreciate this is only for connection strings at the moment, But it does mean we could extend the API with the same pattern for env vars.

I think that the way each service executes is in series defined in the yaml file. I've looked at the steel toe implementation and they have a bucket that listens for replacements, then resolves them at runtime. Which is a bit easier with the configuration provider - where as here we have to set them at execution of the service.

https://github.com/SteeltoeOSS/Steeltoe/blob/8d67ec7a1863ebd7708b24d133c3ee790728a6b5/src/Common/src/Common/Configuration/PropertyPlaceHolderHelper.cs

image

I'll write a unit test and experiment.

Phiph commented 1 year ago

Okay I have it nearly working.... I'm going to do some code cleanup, and try to parse the binding and submit a PR with a E2E for this change.

Input:

image

Output:

image
Phiph commented 1 year ago
image

No way pretty - but it works.

https://github.com/Phiph/tye/commit/b256a12bfcb8a58ba55b8d7f8ad9abcff4c26b24