Open davidfowl opened 4 years ago
Steeltoe supports resolving property placeholders in configuration values
@TimHess that's pretty cool! We have a request to support something similar in configuration.
We want to do a different proposal to this. We're going to make it possible to add an alias to a binding, so that the value it provides can be used in a configurable environment variable.
Totally a +1 to this issue!
This is the case I am facing:
- name: elasticsearch
image: docker.elastic.co/elasticsearch/elasticsearch:7.9.3
env:
- name: discovery.type
value: single-node
tags:
- resources
- search
bindings:
- containerPort: 9200
protocol: http
connectionString: "${host}:${port}"
- name: search-resource-access
project: ./ResourceAccess/SearchResourceAccess/src/SearchResourceAccess.Host/SearchResourceAccess.Host.csproj
tags:
- ras
- search
env:
- name: SearchUri
value: "${env:CONNECTIONSTRINGS__ELASTICSEARCH}"
Like specified in the opening post, this application expects the connection string to elasticsearch in the configuration key "SearchUri". Being an older project, I don't have the possibility to use Configure<TOptions>
and refactoring this application to accomodate this need is beyond the scope of my research.
"replicas": {
"search-resource-access_80a73db4-1": {
"dockerCommand": null,
"containerId": null,
"dockerNetwork": null,
"dockerNetworkAlias": null,
"name": "search-resource-access_80a73db4-1",
"ports": null,
"exitCode": null,
"pid": 3392,
"environment": {
"DOTNET_ENVIRONMENT": "Development",
"ASPNETCORE_ENVIRONMENT": "Development",
"DOTNET_LOGGING__CONSOLE__DISABLECOLORS": "true",
"ASPNETCORE_LOGGING__CONSOLE__DISABLECOLORS": "true",
"DOTNET_ROOT": "C:\\Program Files\\dotnet",
"DOTNET_MULTILEVEL_LOOKUP": "0",
"PATH": "<omitted>",
"SearchUri": "${env:CONNECTIONSTRINGS__ELASTICSEARCH}",
"CONNECTIONSTRINGS__ELASTICSEARCH": "localhost:10551",
"APP_INSTANCE": "search-resource-access_80a73db4-1"
},
"state": "ready"
}
As you can see, SearchUri contains the raw string and not a processed one.
This feature would make it easier to support also non ASP.NET Core applications with Tye.
The case @Kralizek mentioned it's interesting and I think would be really useful. Imagine for example we are using the same service for starting a SQL Server but we'd like to define in another service 2 connection strings, one per database:
services:
- name: sql
image: mcr.microsoft.com/mssql/server:2019-latest
bindings:
- port: 1433
connectionString: Server=tcp:${host},${port};User ID=SA;Password=Password123;
- name: api
project: API/API.csproj
env:
- name: CONNECTIONSTRINGS__APPDB
value: "${env:CONNECTIONSTRINGS__SQL};Database=App"
- name: CONNECTIONSTRINGS__UNICORNSDB
value: "${env:CONNECTIONSTRINGS__SQL};Database=Unicorns"
@cjaliaga I'm looking for something like thos. I would like to have one instance of MSSQL service with multiple databases/connection strings.
In my case, I want to do something like:
services:
- name: api
project: src/microservices/api/api.csproj
- name: web
executable: cmd
args: '/c npm run start --prefix src/web -- --web-port ${this.port} --api-port ${api.port}`'
The goal is to let web
start on any available port, but also tell web
where api
is. web
is a pure HTML/CSS/JS application, and we rely on the framework's (aurelia, in this case) dev server during development because it provides the best experience.
I don't know what's involved to satisfy my requirements, but it seems like a relatively easy problem to solve. Would I be wasting my time if I forked it and took a stab?
I ended up overloading GetConnectionString() with an extension method that appends the $"Database={database}" to the original output.
We would accept a PR for this if we can flesh out the design in this issue.
I'm considering compiling a list of requirements based on what is in this and a few other issues. For example, I wonder if the token/variable system could be extended to support multiple tye "instances", as in this issue. <-- a neat feature, BTW.
^^^^^ Edited to correct the link... Sorry for any confusion. :(
@TimHess and @davidfowl, Steeltoe is working fine with me. These are some additional steps
name: postgres
image: postgres:11-alpine
env_file:
- .env
bindings:
- name: port
port: 5432
connectionString: Server=${host};Port=${port};User Id=${env:USER};Password=${env:PASSWORD};
Host.CreateDefaultBuilder(args)
.ConfigureAppConfiguration((context, config) =>
{
config.AddJsonFile("appsettings.json", optional: true);
....
config.AddPlaceholderResolver();
})
....
"ConnectionStrings": {
"default": "${connectionstrings:postgres?no_connection};Database=<your database>;"
},
However, if Tye can support this feature, it will be better.
I finally got far enough into Tye to figure out I need this. There must be plenty of people out there who aren't in a position to modify the source of a dotnet application so that it can consume the specific env vars generated by Tye. We should be able to bind arbitrary, custom environment variables to any dynamically generated string in Tye (binding, connection string etc.)
Bump :)
Trying to get this to work without relying on the hardcoded host.docker.internal:9200
- name: opensearch
image: opensearchproject/opensearch:1.2.4
env:
- name: discovery.type
value: single-node
- name: DISABLE_SECURITY_PLUGIN
value: "true"
bindings:
- protocol: http
containerPort: 9200
- name: opensearch-dashboards
image: opensearchproject/opensearch-dashboards:1.2.0
bindings:
- protocol: http
containerPort: 5601
env:
- name: OPENSEARCH_HOSTS
value: '[\"http://host.docker.internal:9200\"]'
- name: DISABLE_SECURITY_DASHBOARDS_PLUGIN
value: "true"
I agree this would be a useful feature, when dealing with non-dotnet services. My use case is to add a pgweb
dashboard to postgres
without the need to add a new connection each time I restart tye. I would need to configure the DATABASE_URL
variable with the value from the connection string CONNECTIONSTRINGS__POSTGRES__PGWEB
.
- name: postgres
image: docker.io/library/postgres
env:
- name: POSTGRES_USERNAME
value: postgres
env_file:
- .secrets/.env
volumes:
- source: .postgres
target: /var/lib/postgresql/data
bindings:
- name: pgweb
protocol: tcp
containerPort: 5432
connectionString: postgres://${env:POSTGRES_USERNAME}:${env:POSTGRES_PASSWORD}@${host}:${port}/postgres?sslmode=disable
- name: backend
protocol: tcp
containerPort: 5432
connectionString: Server=${host};Port=${port};Username=${env:POSTGRES_USERNAME};Password=${env:POSTGRES_PASSWORD};
- name: pgweb
image: docker.io/sosedoff/pgweb
env:
- name: SESSIONS
value: '1'
- name: DATABASE_URL
value: ${env:CONNECTIONSTRINGS__POSTGRES__PGWEB}
bindings:
- protocol: http
containerPort: 8081
Okay I've done some digging....
so the service bindings don't get computed until the host is running them.
There is already support for some patterns - maybe we could extend it a bit to get close to what you're looking for.
This line is of particular interest: Looks like there was some thought put into it!
However, I appreciate this is only for connection strings at the moment, But it does mean we could extend the API with the same pattern for env vars.
I think that the way each service executes is in series defined in the yaml file. I've looked at the steel toe implementation and they have a bucket that listens for replacements, then resolves them at runtime. Which is a bit easier with the configuration provider - where as here we have to set them at execution of the service.
I'll write a unit test and experiment.
Okay I have it nearly working.... I'm going to do some code cleanup, and try to parse the binding and submit a PR with a E2E for this change.
Input:
Output:
No way pretty - but it works.
https://github.com/Phiph/tye/commit/b256a12bfcb8a58ba55b8d7f8ad9abcff4c26b24
This is a sample of an application that has an opiniated view of what env variables to use for ports. It wants to reference the tye's generated ports in the env variables (see below for an example).
This brings up one interesting challenge: