dotnet / aspire

An opinionated, cloud ready stack for building observable, production ready, distributed applications in .NET
https://learn.microsoft.com/dotnet/aspire
MIT License
3.78k stars 439 forks source link

Support for Custom Hostnames in Local Development #1116

Closed dotnetian closed 10 months ago

dotnetian commented 10 months ago

Hello,

I've been using .NET Aspire for developing microservices and it's been a great experience. However, I've encountered a limitation that I believe could be addressed to improve the development experience.

Currently, .NET Aspire uses localhost with different ports for each service during local development. This works well for most scenarios, but there are cases where having a custom hostname for a service would be beneficial. For example, when testing services that rely on absolute URLs or when simulating a production-like environment locally.

I propose adding support for custom hostnames during local development. This could potentially be configured through the WithServiceBinding method when adding a project to the builder, like this:

var basket = builder.AddProject<Projects.BasketService>("basket")
    .WithServiceBinding(hostPort: 8888, scheme: "http", name: "dashboard", hostName: "bargeh.testdev");

I understand that this feature might require significant changes and could have implications on how services are discovered and communicated during development. However, I believe that the flexibility it provides would be a valuable addition to .NET Aspire.

Thank you for considering this feature request.

davidfowl commented 10 months ago

For example, when testing services that rely on absolute URLs or when simulating a production-like environment locally.

Can you elaborate?

Host name support is non trivial to do without containers and projects don't run in containers by default so I'm not 100% sure what scenarios you are trying to accomplish.

dotnetian commented 10 months ago

Hello @davidfowl. I appreciate your reply. Let me explain my idea more clearly. We can assign custom hosting addresses in the launchSettings.json file in ASP.NET Core, like this:

      ...
      "https": {
        "commandName": "Project",
        "dotnetRunMessages": true,
        "launchBrowser": true,
        "inspectUri": "{wsProtocol}://{url.hostname}:{url.port}/_framework/debug/ws-proxy?browser={browserInspectUri}",
        "applicationUrl": "https://bargeh.net", // Here
        "environmentVariables": {
          "ASPNETCORE_ENVIRONMENT": "Development"
        }
      },
     ...

This allows us to set the Wapp (Web Application) URL to https://bargeh.net. This URL is not accessible on the internet, but we can make it work locally by modifying the Windows' hosts file located in C:\Windows\System32\Drivers\etc\hosts, and adding 127.0.0.1 bargeh.net to it. Then we can host the local Wapp with this address.

This feature is available in ASP.NET Core and it works well. My suggestion is to add a similar option to .NET Aspire too, so we can customize the format of each service that we want to host.

DamianEdwards commented 10 months ago

Modifying the HOSTS file is a privileged action and not something that a toolchain like .NET Aspire should reasonably do as an automatic part of the dev inner loop in my opinion. Or are you saying that even if you modify the HOSTS file manually, you can't use those URLs in service projects if they're launched via an AppHost project?

dotnetian commented 10 months ago

No, modifying the hosts file is not what I'm suggesting to add implement in Aspire. What I'm suggesting is adding the abality to make the AppHost's endpoints listen to other URLs than localhost too, as we have it in ASP.NET Core currently.

dotnetian commented 10 months ago

Modifying the HOSTS file is a privileged action and not something that a toolchain like .NET Aspire should reasonably do as an automatic part of the dev inner loop in my opinion. Or are you saying that even if you modify the HOSTS file manually, you can't use those URLs in service projects if they're launched via an AppHost project?

I'm issuing that the AppHost cannot make projects' endpoints listen to anything other than localhost like this:

var smsApi = builder.AddProject<Projects.Bargeh_SMS_API> ("smsapi").WithServiceBinding(hostBase: "bargeh.net");

Also, another workaround for this problem is just adding an option so the AppHost reads the launchSettings.json file for each project, and hosts them respecting to those, similiar to this:

var smsApi = builder.AddProject<Projects.Bargeh_SMS_API> ("smsapi").WithServiceBinding (launchProfile: "https"));
DamianEdwards commented 10 months ago

Also, another workaround for this problem is just adding an option so the AppHost reads the launchSettings.json file for each project, and hosts them respecting to those, similiar to this:

This is the default behavior today. The launchSettings.json is read for projects and used to configure the endpoints the app listens on. However, I imagine that the ingress layer (DCP) that's put in front of the projects when launched via the AppHost is what's not supporting custom domains.

@karolz-ms?

karolz-ms commented 10 months ago

@dotnetian sorry, I am not sure I fully understand the ask.

With Aspire preview 1, when you call AddProject(), you do give it a "logical name". For example, this is how Catalog service gets its logical name in the Aspire eShopLite sample: https://github.com/dotnet/aspire-samples/blob/main/samples/eShopLite/eShopLite.AppHost/Program.cs#L6

The project/service can then be referred by that name from clients: https://github.com/dotnet/aspire-samples/blob/main/samples/eShopLite/eShopLite.Frontend/Program.cs#L12C60-L12C60

The idea being here, that in your local development environment that name resolves to a combination of local loopback address and some port, but in production environments like Azure Container Apps or Kubernetes, the same logical name will resolve to real container/service IP.

So could you please elaborate what is it that is missing for you from what Aspire provides already with regards to logical host names. Thanks!

DamianEdwards commented 10 months ago

The request is for support of non-localhost host names for endpoints, not the logical names.

davidfowl commented 10 months ago

The only valid thing to bind is localhost, *, 0.0.0.0 or the machine IP. Not a host name.

DamianEdwards commented 10 months ago

Yeah, binding to localhost or 127.0.0.1, and then editing your HOSTS file to map your custom domain to 127.0.0.1 should work from a networking point of view, but the configuration of service discovery to map your app's references to "http://mycustomdomain.com" would not be automatic. I imagine you could write some hosting extensions to make that work though, by separating the configuration of the endpoint binding and service discovery environment variable values.

dotnetian commented 10 months ago

Fixed this issue by setting the launch profile:

builder.AddProject<Projects.Bargeh_Main_Wapp> ("bargehmainwapp")
    .WithReference (usersApi)
    .WithReference (smsApi)
    .WithLaunchProfile ("https");
atrauzzi commented 10 months ago

On Linux, I enable dnsmasq as part of NetworkManager and then I do something like this:

# /etc/NetworkManager/dnsmasq.d/localdev.com.conf
local=/localdev.com/
address=/localdev.com/127.0.0.1

This gives me a wildcard domain that I can use for all my local development, it just requires whatever reverse proxy I run to "catch" the specific names and forward them to the correct services.

@DamianEdwards -- Would it be possible to configure Aspire with a "tld" (using that term loosely here) that it could map ingress to services for using nginx or whatever? Names would just be $"{serviceName}.{configuredTld}"?

ryno1234 commented 9 months ago

Fixed this issue by setting the launch profile:

builder.AddProject<Projects.Bargeh_Main_Wapp> ("bargehmainwapp")
    .WithReference (usersApi)
    .WithReference (smsApi)
    .WithLaunchProfile ("https");

Not sure how this fixed your problem. I'm trying to do the same thing you are, but this doesn't fix the issue from what I'm seeing. The Apire dashboard is still showing the URL for my project being mapped to localhost:port despite my launchSettings.json for my project having a custom domain specified.

{
  "profiles": {
    "https": {
      "commandName": "Project",
      "launchBrowser": true,
      "applicationUrl": "https://client1.myapp.local/",
      "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "Development"
      }
    }
  }
}

To the team here, a concrete reason for a custom domain would be an application that operates as Software as a Service with client-specific URLs. Each client gets their own URL, i.e. client1.myapp.com. On my machine, I have custom host entries in the form of client1.myapp.local which maps to 127.0.0.1. The host name is read within the app and we serve up content specific to Client 1. I could easily change this to client2.myapp.local and get a totally different set of content.

Is this not possible right now?

One other concrete reason for this is authentication cookies stored for a specific domain. For example, we have many apps with a TLD, and one of those is our sign on app that sets cookies for a TLD (i.e. myapp.local). Those cookies aren't read when I hit localhost:5000 (etc) because the TLD is different. I need all my sites to be accessible through that TLD.

davidfowl commented 9 months ago

We should break down the problem encountered. To make host name mappings work in the browser (and every client that relies on the OS for DNS resolution), you have to add host entries on your machine (/etc/hosts on linux and in the appropriate location on windows), this is something you have to do manually. Aspire does not help here (nor does it have plans to help here). You can also use a tool like dnsmaq to make wildcard routing easier for local dev.

The next piece of the puzzle is port mapping. Aspire is avoiding port conflicts on your local machine by randomly selecting ports. Using an address like https://client1.myapp.local/ assumes that you have a server listening on :443 (and probably :80) routing requests to the appropriate application.

The last piece of the puzzle is how aspire's networking interacts with this.

Once you have this, it should be possible to make the end-to-end work.

jakenuts commented 9 months ago

I've yet to try, but would an alternate setup where each .local domain is given its own 127.0.0.N ip address help or hinder this setup? Prior to Aspire that was the only way for me to have two related web apps running in Kestrel/IIS Express without clashing over the 80/443 ports.

davidfowl commented 9 months ago

I just did it so I know it works 😄. Like I said, Aspire doesn't help you with the fact that you need a proxy on 80/443 that can do subdomain/hostname routing. You can use the proxy of your choice (like YARP, nginx, etc) to do that.

atrauzzi commented 9 months ago

It would be super useful if it bundled YARP (or whatever) and even a DNS server for an even more killer out of the box experience.

People could basically just define services and start with all the toil of getting a realistic TLS and CORS setup magic'd away. All they'd have to do is add the DNS server to their systems network config!

davidfowl commented 9 months ago

Yes agreed, we could attempt to make this easier, but it's not currently on the roadmap.

jakenuts commented 9 months ago

Thanks, this was the configuration I used to get that to work on my system (each local domain has it's own IP address, local wildcard ssl cert).

image

image