dotnet / tye

Tye is a tool that makes developing, testing, and deploying microservices and distributed applications easier. Project Tye includes a local orchestrator to make developing microservices easier and the ability to deploy microservices to Kubernetes with minimal configuration.
MIT License
5.29k stars 521 forks source link

Seq memory limits, API key etc. #1608

Open rolfik-mycronic opened 1 year ago

rolfik-mycronic commented 1 year ago

I have tried using Seq logging extension and it launches Seq docker container and our services log to it when properly configured. But I have not found a way how to specify other parameters like:

To overcome it, I have tried this approach instead of the extension:

services:
- name: seq
  image: datalust/seq
  volumes:
    - source: ./.logs
      target: /data
  env:
  - ACCEPT_EULA=Y
  - SEQ_API_KEY=something
  bindings:
  - protocol: http
    port: 5341
    containerPort: 80
    connectionString: "http://${host}:${port}?apiKey$={env:SEQ_API_KEY}"

But while it launches Seq container, it does not propagate seq connectionString to our services where it should be used to configure Serilog with Seq.

Any ideas?

Thank You

Phiph commented 1 year ago

hey you should be able to access the SEQ connection string for your config using the CONNECTIONSTRINGS SERVICE-SEQ environment variable that gets injected to each of your services.

Question: Why do you need an API key for a local development when using SEQ?

rolfik-mycronic commented 11 months ago

API key is used to change logging level at runtime and it can be used in production as well

Phiph commented 11 months ago

Are you attempting to use tye to deploy SEQ aswell?

rolfik commented 11 months ago

No, we use docker compose for deployment, but reuse tye environment variables at runtime.

Phiph commented 11 months ago

Cool I'll happily take a look, but I'm struggling to understand why you need a SEQ api key for local dev environment - I don't use SEQ in my lifecycle so any more information you can give me will help.

We can easily extend the SEQ extension to include the other variables, but I'm uncertain whether including the APIkey value is worthwhile considering all config for SEQ may be removed when the container restarts, or when another dev in your project pulls and runs it.

rolfik commented 11 months ago

I have not investigated more why Seq connectionString is not available at other service start, but that is what I have seen when degugging a service. My idea was to specify Seq API key in environment variable file specific for a developer. Or even better in global environment variable. Seq config is stored on attached local data volume so it is kept there.

Phiph commented 11 months ago

I think the reason why SEQ as an extension works is because it set's itself as a dependency for all other services

https://github.com/dotnet/tye/blob/75465614e67b5f1e500d1f8b1cb70af22c0c683e/src/Microsoft.Tye.Extensions/Seq/SeqExtensions.cs#L59

There are some suggestions in #486 and #652 to control the start-up order. As there is an API available to us I could look into supporting a depend_on set up that may allow your custom SEQ container to properly inject the variables.

rolfik commented 11 months ago

Yes, I thought it was a dependency problem as well, but are not the connection strings setup for all services before any one of them is run?

Phiph commented 11 months ago

Yeah I use SQL server with connection strings and it works.

Where is your SEQ container in your tye file, is it at the bottom?

rolfik commented 11 months ago

Seq container yaml is included in app.yaml before other services.

Phiph commented 11 months ago

Ahh okay, I'll take a look and submit a PR for your use case as it seems very interesting!

I'll also look into the depends on feature

Phiph commented 11 months ago

Hey I've had a look, and I'll write some more E2E tests to validate the approach, but I'm using already existing API's so hopefully it works as expected :)