dotnet / aspire

An opinionated, cloud ready stack for building observable, production ready, distributed applications in .NET
https://learn.microsoft.com/dotnet/aspire
MIT License
3.61k stars 401 forks source link

Full support for Kubernetes Deployment #830

Open VenkateshSrini opened 9 months ago

VenkateshSrini commented 9 months ago

Hi, From the Aspire project I would like to generate the deployment and service . yaml. If my app has a console application, then if that is not skipped then I need a Kubernetes Job deployment. I also need to have a facility for specifying of that is a cron job. I need a way to mount my application appSetting.json from Config map. If there are any secrets used in application then I want the K8s secrets also to be generated

timheuer commented 9 months ago

My only nit is that generate is doing much more than generate. It's doing some partial steps of actually deploying as well (building and pushing containers).

prom3theu5 commented 9 months ago

My only nit is that generate is doing much more than generate. It's doing some partial steps of actually deploying as well (building and pushing containers).

Aye - feels like that should be either a separate command, or part of apply. Especially since it requires write access to your registry to push those containers too. Might just want to spit out some manifests - prototyping etc

User journey for apply is already two hops though, possibly three if you have to add in init too

What if we roll it into the apply command - pre cluster selection, so it pushes container images then, as they are required for apply to actually succeed after deploying manifests

timheuer commented 9 months ago

This is such good progress @prom3theu5! I'm torn to keeping the discussion here but also not losing individual discussions on features of the tool in the repo LOL -- so I logged this as well: https://github.com/prom3theu5/aspirational-manifests/issues/37

prom3theu5 commented 9 months ago

Thanks a lot πŸ˜„

timheuer commented 9 months ago

User journey for apply is already two hops though, possibly three if you have to add in init too

Right, don't want to be too petty for the individual actions if there is a fast-path. I like the idea always of give me all the knobs, but also give me an easy button. As you note, sometimes I may just want to generate manifests, other times 'just do it all' for me. In some of our own tools we do this (e.g., provision (create), deploy (push), but up (provision+push)).

prom3theu5 commented 9 months ago

I think some of the commands right now are too opinionated about how they run. For instance, everything for init, outside of injected services, aka prompts / questions - is all contained within the InitCommand

Moving some of the actions, certainly around prompts etc, into a shared command base would allow the commands to be streamlined into sort of action orchestrators

We could then easily create combination commands, as its just extra orchestration of actions

Maybe from a composition point of view, it actually makes more sense to have a collection of single action classes, that each perform one given prompt, and allow chaining them together fluently

davidfowl commented 9 months ago

@prom3theu5 put that video in the README!

davidfowl commented 9 months ago

@prom3theu5 Does it support deploying to a private registry?

davidfowl commented 9 months ago

Also it's too late now but its possible the version should have been preview πŸ˜„.

https://www.nuget.org/packages/Aspirate

prom3theu5 commented 9 months ago

put that video in the README!

Done haha.

Does it support deploying to a private registry?

It does yeah. What it doesn't directly support is generation of imagePullSecrets if they don't exist in your namespace However, you can either add them to your cluster manually, or customise the kustomize templates, and add them

Also it's too late now but its possible the version should have been preview πŸ˜„.

https://www.nuget.org/packages/Aspirate

Hahaha.. you are definitely right there... whoops. Deprecate 1.01, republish as 1.0.2-preview?

davidfowl commented 9 months ago

0.1.0-preview1? Hard to go back in time πŸ˜„

prom3theu5 commented 9 months ago

Aye no control over http-locals for people who have it already, what happens in those cases, they wouldnt be offered an update would they, even if I unlist?

I think you have to explicitly set -version anyway dont you on preview installs?

davidfowl commented 9 months ago

I think unlist and rebrand as 0.1.0, yep! Not many people have downloaded yet

prom3theu5 commented 9 months ago

Sorted - its listed as '0.1.0-preview-1' now The others I've just unlisted

davidfowl commented 9 months ago

OK I posted on X! Let's see if we can get some feedback for the MVP.

pjmagee commented 9 months ago

What does helm use?

helm upgrade and helm install with some variations like --dry-run or -install and then you pass it the chart either via a registry or local disk folder or zip.

prom3theu5 commented 9 months ago

New version has just gone out I've now converted the cli app to utilise System.CommandLine This brings up the ability to introduce things like localization etc

Also - it allowed me to implement better handling of input parameters

Commands now invoke a chained series of actions within an ActionExecutor, and all actions have a shared State within the context of a command run.

An extension on the state looks at the incoming Input options for a command pre-execution, and updates the current app state with incoming values, augmenting them / overriding them

This opens up a lot of potential, such as ease of implementation of new commands (A command is basically an orchestrator [order picker / someone following a shopping list] that runs (picks) a bunch of reusable actions in the order that they want to run (use / "eat") them in

So, lets say we want a new command that combines the actions of Generate and Apply, easy - create the command, and just chain together the existing actions.

It also means individual actions can be tested in isolation, purely operating on their singular task. Only actions share state, all services take input parameters from actions, meaning they are also isolated in the context of an execution.

With this i've now rolled out the ability for the flag --non-interactive to be passed into just the init command for now, along with the ability to supply all the required parameters to set the config file.

NonInteractive is a shared option, which I will roll out to the other commands tomorrow, along with implementing the prompt bypasses in the actions that require it.

davidfowl commented 9 months ago

If you want a hard mode challenge, attempt eshop https://github.com/dotnet/eshop

we have some gaps here that we want to address for the next preview. This will help flesh them out!

prom3theu5 commented 9 months ago

If you want a hard mode challenge, attempt eshop https://github.com/dotnet/eshop we have some gaps here that we want to address for the next preview. This will help flesh them out!

I can see just looking at the AppHost there is a lot more going on in that πŸ˜› Will pull out some issues from it, and add to project board.

Was playing with container support last night, and toying with the idea of adding an experimental option to generate docker-compose output as well. Problem for both containers and docker-compose is the containerPorts aren't added to the manifest in this release. I know thats coming soon though, and thats the missing piece for containers πŸ˜„


Just pushed the final command changes for --non-interactive support, and I added a --skip-build flag to the generate command, which skips the build and push of containers. That allows you to use this in scripts / cicd now without an interactive console

aspirate generate --non-interactive --skip-build

     _                    _           ___
    / \     ___   _ __   (_)  _ __   ( _ )
   / _ \   / __| | '_ \  | | | '__|  / _ \
  / ___ \  \__ \ | |_) | | | | |    | (_) |
 /_/   \_\ |___/ | .__/  |_| |_|     \___/
                 |_|
Automate deployment of a .NET Aspire AppHost to a Kubernetes Cluster

Non-interactive mode enabled.

Successfully loaded existing aspirate bootstrap settings from '.'.

Generating Aspire Manifest for supplied App Host:

Executing: dotnet run --project "/Users/prom3theu5/git/personal/demo/TestAspire/TestAspire.AppHost/." -- --publisher manifest --output-path
manifest.json
Building...
info: Aspire.Hosting.Publishing.ManifestPublisher[0]
      Published manifest to: /Users/prom3theu5/git/personal/demo/TestAspire/TestAspire.AppHost/manifest.json
    (βœ”) Done:  Created Aspire Manifest At Path: /Users/prom3theu5/git/personal/demo/TestAspire/TestAspire.AppHost/manifest.json

Non-Interactive Mode: Processing all components in the loaded file.

Gathering container details for each project in selected components

    (βœ”) Done:  Populated container details cache for project apiservice
    (βœ”) Done:  Populated container details cache for project webfrontend

Gathering Tasks Completed - Cache Populated.

Skipping build and push action as requested.

Generating kustomize manifests to run against your kubernetes cluster:

    (βœ”) Done:  Generating /Users/prom3theu5/git/personal/demo/TestAspire/TestAspire.AppHost/aspirate-output/cache
    (βœ”) Done:  Generating /Users/prom3theu5/git/personal/demo/TestAspire/TestAspire.AppHost/aspirate-output/apiservice
    (βœ”) Done:  Generating /Users/prom3theu5/git/personal/demo/TestAspire/TestAspire.AppHost/aspirate-output/webfrontend

 πŸš€ Execution Completed
prom3theu5 commented 9 months ago

If you want a hard mode challenge, attempt eshop https://github.com/dotnet/eshop we have some gaps here that we want to address for the next preview. This will help flesh them out!

Immediately whats shown is the need for other container backed resource types (postgres, rabbitmq, redis etc) to have expanded output in the manifest Specifically Annotations. the manifest for this doesn't include the fact that a custom image is being used for postgres. That could be handled by using the custom template option, and just using a template with the image in it - but would be better to have the ability to read that info

Also - we have no ContainerBaseImage support atm

image

Managed to get services running with a few manual edits Its all mainly due to IdentityServer

We need to really introduce a YARP instance which handles reverse proxy of the auth service, with forwarding headers enabled. I had to manually fudge the middleware in the identityApi to override the the server url before identityserver was registered to ensure OIDC discovery was generated correctly, and then also ensure the TempKey was copied into the container, and that the pod had emptyDir mounted on /app/keys

Also had to remove https webapp redirect url etc - as we dont have tls yet Oh and just changed the container image for postgres in the generated yaml, to the one from AppHost

mitchdenny commented 9 months ago

I've been out for all last week and let me just say what you've done here in such a small period of time is just awesome!

davidfowl commented 9 months ago

Immediately whats shown is the need for other container backed resource types (postgres, rabbitmq, redis etc) to have expanded output in the manifest Specifically Annotations. the manifest for this doesn't include the fact that a custom image is being used for postgres. That could be handled by using the custom template option, and just using a template with the image in it - but would be better to have the ability to read that info

This is one of the things that we need to decide on or improve in general. Today these are abstract resource types in the aspire manifest. At runtime, they are containers, but in the manifest, they end up being these "typed resources" that the deployment tools need to interpret. As an example, when deploying to azure container apps, azd treats redis.v0 as use this addon (https://learn.microsoft.com/en-us/azure/container-apps/services), but you could imagine that it could also mean use azure redis in a different "environment". In that universe the "containerness" is an implementation detail. That's why the image name isn't in the manifest. Here is the issue tracking this discussion https://github.com/dotnet/aspire/issues/616. I think we will end up with abstract resources (redis, rabbit etc), and specific resources (e.g. container, cloud).

Also - we have no ContainerBaseImage support atm

By this do you mean it isn't in the csproj? The container tools default this based on the TFM I thought. @baronfel ?

Its all mainly due to IdentityServer

We need to really introduce a YARP instance which handles reverse proxy of the auth service, with forwarding headers enabled. I had to manually fudge the middleware in the identityApi to override the the server url before identityserver was registered to ensure OIDC discovery was generated correctly, and then also ensure the TempKey was copied into the container, and that the pod had emptyDir mounted on /app/keys

Can you file an issue on github.com/dotnet/eshop? Also if you have a fork, push it so we can see the changes you made.

Also had to remove https webapp redirect url etc - as we dont have tls yet

Yep, adding suport for TLS is going to be interesting. Easier on k8s for sure!

prom3theu5 commented 9 months ago

Can you file an issue on github.com/dotnet/eshop? Also if you have a fork, push it so we can see the changes you made.

Oh I wouldn't want to push it somewhere and see people trying to use it i the wild - its was pure hacks πŸ˜‰ I was utilising middleware pre addition of identityserver to completely fudge the oidc discovery document response when certain paths were requested lol

When using a private local domain, i.e. david.local, without introducing a dns server, and passing the dns config to kube so that each container gets the resolv.conf updated, the cluster will never be able to communicate with your host pc in docker-desktop. At least - i don't think it can. Maybe using the internal docker addresses it'd be possible instead of trying to use an nginx ingress controller like I was

Its due to the fact that auth happens server side using the internal cluster address of the identity-api, but the oidc document endpoints will then be set to internal addresses - i.e. http://identity-api:8080/connect/authorize This will obviously fail when you are redirected on an auth request back to the front end, as they are addresses internal to the cluster If you change the IdentityUrl used in the web app to your reverse proxy address from your nginx ingress controller then the request will fail to get to the actual server, as its resolved internally against the hosts file in the pod. You can hack in hostAliases in the deployments, but that means you need to know the internal ip a pod is going to be assigned up front, and when i was trying that it was failing. This was failing, even with Forwarding host headers for host and proto enabled, and added as first middleware within the identity api.

In the wild, using an actual domain means that when your kube cluster starts and passes your hosts dns config to the pods, they can resolve the hostname successfully. Purely a local issue this.

Now if auth was to happen client side in wasm, and not server side, then the address mapped through the ingress resource to the backend service would actually work - as it'd be resolved using the hosts "host" file.

Truth be told I didn't spend a massive amount of time working the issue, maybe 1.5 hours max.

By this do you mean it isn't in the csproj? The container tools default this based on the TFM I thought. @baronfel ?

Oh sorry I mean in the context of aspirate - aspirate doesn't do anything with non-qualified BaseImages. The base image set in the csproj for the mobile backend for frontend yarp reverse proxy isn't fully qualified, i.e. its just mobilebffagg or something like that. The container tooling within .net complained it didn't know where to pull the image from. I just removed it, it used net 8 base, and worked fine.

This is one of the things that we need to decide on or improve in general. Today these are abstract resource types in the aspire manifest. At runtime, they are containers, but in the manifest, they end up being these "typed resources" that the deployment tools need to interpret. As an example, when deploying to azure container apps, azd treats redis.v0 as use this addon (https://learn.microsoft.com/en-us/azure/container-apps/services), but you could imagine that it could also mean use azure redis in a different "environment". In that universe the "containerness" is an implementation detail. That's why the image name isn't in the manifest. Here is the issue tracking this discussion https://github.com/dotnet/aspire/issues/616. I think we will end up with abstract resources (redis, rabbit etc), and specific resources (e.g. container, cloud).

Ahh gotcha - I understand Will be interesting to see how it pans out πŸ˜„

cysnet commented 9 months ago

@prom3theu5 @davidfowl So cool, can’t wait to try it out!

mitchdenny commented 9 months ago

For projects 8080 (for http, and ignore https for now), I would start with this since .NET 8+ containers expose this port. Container resources will have a containerPort property on the binding definition.

@davidfowl wondering whether we should introduce a tlsTerminationFor property to express the relationship between HTTP and HTTPS bindings.

mitchdenny commented 9 months ago

"containerness" is an implementation detail

I think this is key. I think each deployment tool should have configuration (if required) which tells it what to interpret these high level resources (redis.v0 etc) as. I don't think we should try and add fallback container name in the manifest and stick to the "the deployment tool should understand all resource types in manifest or fail" line.

prom3theu5 commented 9 months ago

I think this is key. I think each deployment tool should have configuration (if required) which tells it what to interpret these high level resources (redis.v0 etc) as. I don't think we should try and add fallback container name in the manifest and stick to the "the deployment tool should understand all resource types in manifest or fail" line.

Oh absolutely - in fact thats how Aspir8 handles it. It has an abstract resource type, and upon deserialization of the type from the manifest performs polymorphic deserialization from the type name, to a given defined type which all inherit from the abstract resource.

The problem when it comes to containers, you can add ports, add annotations and environmental variables - its that info that needs to be in the manifest for tools to read.

Then with Redis for example - if you are ultimately deploying to AKS, you may not using Redis in the cluster as a container - you might be using Azure redis, or a dedicated vm etc (its cheaper to get a Bsm2 and purely use redis on there lol). I think this example is where you'd need separate types in the manifest. There has to be something that tells tools what type it has to treat it as I terms of aspir8, for a scenario like this, instead of generating a manifest for deployment, and service, we'd ultimately just be generating a secret here which hs the connectin string details to your external deployment

mitchdenny commented 9 months ago

@prom3theu5 wanted to bring you into the loop on a new resource type we are looking to add to the manifest called dockerfile.v0. The scenario is that sometimes people have got a Python app, or a Node.js app or similar that they run locally, but when deployed it needs to be packaged up into a container.

We could force them to do the container build themselves and just have an AddContainer(...) call in the AppHost for deployment purposes, but we wanted the ability for folks to use AddExecutable(...) for inner loop where it is convenient (which would be wrapped by things like AddNodeApp(...) and AddNpmApp(...).

https://github.com/dotnet/aspire/pull/1059

davidfowl commented 9 months ago

I think this example is where you'd need separate types in the manifest. There has to be something that tells tools what type it has to treat it as

This is just layers of indirection. Either the developer describes this in code, or the deployment tool gives the developer a way to describe that choice:

e.g.

IResourceBuilder<IRedisResource> basketCache = builder.Environment.EnvironmentName switch
{
    "Development" => builder.AddRedisContainer("basketcache"),
    "Staging" => builder.AddAzureRedis("basketcache"),
    "Production" => builder.AddRedis("basketcache"), // look for a connection string that will be provided externally.
    _ => throw new NotSupportedException($"Environment '{builder.Environment.EnvironmentName}' is not supported.")
};

We could end up with a model like this where the choice is explicit. Or one like this:

var basketCache = builder.AddRedis("basketcache");

Where that decision for what that means is completely deferred to the deployment tool and the orchestration runtime (during development).

prom3theu5 commented 9 months ago

wanted to bring you into the loop on a new resource type we are looking to add to the manifest called dockerfile.v0

Thanks Mitch - that looks really cool. Is that the final manifest output choice in the link? If so I'll start on building in some support

Where that decision for what that means is completely deferred to the deployment tool and the orchestration runtime (during development).

This is better in my opinion - in an ideal world - the application source shouldn't "know or care" about how its going to be hosted. Its a deployment time choice.

Most of the time there has to be some choice of technology stack in the app source code that dislodges this - but thats what I love about the idea of aspire so much. You aren't tightly coupling your app code to tech stack.

prom3theu5 commented 9 months ago

Released support for Dockerfile building, which was merged yesterday into Aspire, with both the build and generate commands

Will likely need further binding testing once aspire preview 2 is released.

Projects automatically get a service and Kestrel ports added to deployment and said service, however Dockerfiles operate fully on custom ports from the manifest, with no default entries.

Also - i'd like to touch on the build command, which I dont think has bene mentioned much - It's very useful outside of actually generating k8s manifests - in fact i've worked in into my daily workflow now.

aspirate build allows you to rebuild and push containers to the registry swiftly and efficiently. This means you can make changes to your work and see them reflected in your containers almost instantly as you have the flexibility to select the items you want to build using a simple menu. This gives you control over what parts of your project you want to focus on at any given time. Its convenience really. One of my peeves as a developer is having to repeat myself. If I have to repeat an action more than twice, why not get back the minutes it takes to perform said action, and just automate it? Thats what I did here ^^.

Also if you pass in an existing aspire manifest to the build command, the build menu appears instantly allowing you to select items to build and push

aspirate build -m ./manifest.json

It'll also populate its internal state with the aspirate.json file if you've initialised it in your AppHost directory and use your tags and registry fall back values. You can ignore that step and pass those values on the cli too.

Generate has been changed too. Previously every generate you ran - overwrote the final (top level) kustomization.yml file, which contained links to the previously generated feature items.

That meant if you generated a set of all your component, each time you would have to rerun the full generation. Now you can regenerate any selection of services, and the final prompt to generate the top level file you can say no to. This will only overwrite the manifests for the services you have selected in that run, allowing you to perform iterative deployments to your cluster, overwriting / updating already deployed components. Big time saver when trying to debug issues

davidfowl commented 9 months ago

@prom3theu5 ! Excellent.

mitchdenny commented 9 months ago

Wow ... you are officially the first tool to support dockerfile.v0 ... I think I'm going to have to start using aspirat8 for my local testing :P

prom3theu5 commented 9 months ago

Haha thanks Mitch Just trying to give back a little. You guys give us so much year on year. I mean aspire is awesome. We now have this single file (manifest) in a language independent format (json) that literally describes the entirety of our solution from implementation to deployment. Doesn't get much better ^^

davidfowl commented 9 months ago

@prom3theu5 trying out aspirate for the first time and I ran into errors:

Aspirate supports setting a fall-back value for projects that have not yet set a 'ContainerRegistry' in their csproj
file.
Would you like to set a fall-back value for the container registry? [y/n] (n): n

Aspirate supports setting a fall-back value for projects that have not yet set a 'ContainerTag' in their csproj file.
Would you like to set a fall-back value for the container tag? [y/n] (n): n

Aspirate supports setting a custom directory for 'Templates' that are used when generating kustomize manifests.
Would you like to use a custom directory (selecting 'n' will default to built in templates ? [y/n] (n): n

(βœ”) Done: Configuration for aspirate has been bootstrapped successfully at
'C:\dev\git\newaspire-sample\.\aspirate.json'.
{
  "TemplatePath": null,
  "ContainerSettings": {
    "Registry": null,
    "Tag": null
  }
}
➜ aspirate generate

     _                    _           ___
    / \     ___   _ __   (_)  _ __   ( _ )
   / _ \   / __| | '_ \  | | | '__|  / _ \
  / ___ \  \__ \ | |_) | | | | |    | (_) |
 /_/   \_\ |___/ | .__/  |_| |_|     \___/
                 |_|
Automate deployment of a .NET Aspire AppHost to a Kubernetes Cluster

Successfully loaded existing aspirate bootstrap settings from '.'.

Generating Aspire Manifest for supplied App Host:

Failed to generate Aspire Manifest at:
davidfowl commented 9 months ago

Ah you have to run it in the apphost folder. That would be a nice error message to add πŸ˜„

davidfowl commented 9 months ago

Next issue:

Required MSBuild property 'ContainerRegistry' not set in project
'..\newaspire-sample.ApiService\newaspire-sample.ApiService.csproj'. Cannot continue.
(1): Aspirate will now exit.

Seems like the registry is needed even though it said it wasn't?

davidfowl commented 9 months ago

I will start a thread on that repo

prom3theu5 commented 9 months ago

Hi @davidfowl

You dont have to run it in the AppHost, but its best to if you don't want to use the command line args to control where it reads files from.

The procedure is

  1. Navigate to AppHost Directory
  2. run aspirate init and say (Y) to the Fallback container registry
  3. run aspirate generate this will allow you to select which services to build and push as containers, then generate manifests for etc

During 3 you should see in cyan that the fallback value has been passed to the dotnet publish command Manifests will be output to ./aspirate-output (customizable with args)

Latest version is 0.1.10-preview

I figured it may be an issue with windows, But i've jut tested it in parallels

PS C:\git> dotnet new aspire-starter -o Test
The template ".NET Aspire Starter Application" was created successfully.
This template contains technologies from parties other than Microsoft, see https://aka.ms/aspire/1.0-third-party-notices for details.

Processing post-creation actions...
Restoring C:\git\Test\Test.sln:
  Determining projects to restore...
  Restored C:\git\Test\Test.ServiceDefaults\Test.ServiceDefaults.csproj (in 301 ms).
  Restored C:\git\Test\Test.Web\Test.Web.csproj (in 301 ms).
  Restored C:\git\Test\Test.AppHost\Test.AppHost.csproj (in 301 ms).
  Restored C:\git\Test\Test.ApiService\Test.ApiService.csproj (in 301 ms).
Restore succeeded.
Restoring C:\git\Test\Test.AppHost\Test.AppHost.csproj:
  Determining projects to restore...
  All projects are up-to-date for restore.
Restore succeeded.
Restoring C:\git\Test\Test.ServiceDefaults\Test.ServiceDefaults.csproj:
  Determining projects to restore...
  All projects are up-to-date for restore.
Restore succeeded.
Restoring C:\git\Test\Test.ApiService\Test.ApiService.csproj:
  Determining projects to restore...
  All projects are up-to-date for restore.
Restore succeeded.
Restoring C:\git\Test\Test.Web\Test.Web.csproj:
  Determining projects to restore...
  All projects are up-to-date for restore.
Restore succeeded.

PS C:\git> cd .\Test\Test.AppHost\
PS C:\git\Test\Test.AppHost> aspirate init

     _                    _           ___
    / \     ___   _ __   (_)  _ __   ( _ )
   / _ \   / __| | '_ \  | | | '__|  / _ \
  / ___ \  \__ \ | |_) | | | | |    | (_) |
 /_/   \_\ |___/ | .__/  |_| |_|     \___/
                 |_|
Automate deployment of a .NET Aspire AppHost to a Kubernetes Cluster

Aspirate supports setting a fall-back value for projects that have not yet set a 'ContainerRegistry' in their csproj
file.
Would you like to set a fall-back value for the container registry? [y/n] (n): y
Please enter the container registry to use as a fall-back value: localhost:5001

(?) Done: Set 'Container fallback registry' to 'localhost:5001'.

Aspirate supports setting a fall-back value for projects that have not yet set a 'ContainerTag' in their csproj file.
Would you like to set a fall-back value for the container tag? [y/n] (n): n

Aspirate supports setting a custom directory for 'Templates' that are used when generating kustomize manifests.
Would you like to use a custom directory (selecting 'n' will default to built in templates ? [y/n] (n): n

(?) Done: Configuration for aspirate has been bootstrapped successfully at 'C:\git\Test\Test.AppHost\.\aspirate.json'.

 ?? Execution Completed
PS C:\git\Test\Test.AppHost> aspirate generate

     _                    _           ___
    / \     ___   _ __   (_)  _ __   ( _ )
   / _ \   / __| | '_ \  | | | '__|  / _ \
  / ___ \  \__ \ | |_) | | | | |    | (_) |
 /_/   \_\ |___/ | .__/  |_| |_|     \___/
                 |_|
Automate deployment of a .NET Aspire AppHost to a Kubernetes Cluster

Successfully loaded existing aspirate bootstrap settings from '.'.

Generating Aspire Manifest for supplied App Host:

        (?) Done:  Created Aspire Manifest At Path: C:\git\Test\Test.AppHost\manifest.json

Gathering container details for each project in selected components

        (?) Done:  Populated container details cache for project apiservice
        (?) Done:  Populated container details cache for project webfrontend

Gathering Tasks Completed - Cache Populated.

Building all project resources, and pushing containers:

Executing: dotnet publish "C:\git\Test\Test.AppHost\..\Test.ApiService\Test.ApiService.csproj"
-p:PublishProfile="DefaultContainer" -p:PublishSingleFile="true" -p:PublishTrimmed="false" --self-contained "true" --os
"linux" --arch "x64" -p:ContainerRegistry="localhost:5001" -p:ContainerRepository="apiservice"
-p:ContainerImageTag="latest"
MSBuild version 17.8.3+195e7f5a3 for .NET
  Determining projects to restore...
  Restored C:\git\Test\Test.ServiceDefaults\Test.ServiceDefaults.csproj (in 272 ms).
  Restored C:\git\Test\Test.ApiService\Test.ApiService.csproj (in 272 ms).
  Test.ServiceDefaults -> C:\git\Test\Test.ServiceDefaults\bin\Release\net8.0\Test.ServiceDefaults.dll
  Test.ApiService -> C:\git\Test\Test.ApiService\bin\Release\net8.0\linux-x64\Test.ApiService.dll
  Test.ApiService -> C:\git\Test\Test.ApiService\bin\Release\net8.0\linux-x64\publish\
  Building image 'apiservice' with tags 'latest' on top of base image 'mcr.microsoft.com/dotnet/runtime-deps:8.0'.
  Uploading layer 'sha256:1f7ce2fa46ab3942feabee654933948821303a5a821789dddab2d8c3df59e227' to 'localhost:5001'.
  Uploading layer 'sha256:227fbee7f8333fe9242a4dff10ac036f7038a975be6b3bf7c132de76da623bb4' to 'localhost:5001'.
  Uploading layer 'sha256:eaf0575a2a24037fde65a105ab55d031a3bb7f9f9cd0e4d31f90825e746b67bc' to 'localhost:5001'.
  Uploading layer 'sha256:9dfad474d1d8dd245c9b94fc9034573971c48a627a67a7b13f3118261bb0d626' to 'localhost:5001'.
  Finished uploading layer 'sha256:eaf0575a2a24037fde65a105ab55d031a3bb7f9f9cd0e4d31f90825e746b67bc' to
'localhost:5001'.
  Finished uploading layer 'sha256:227fbee7f8333fe9242a4dff10ac036f7038a975be6b3bf7c132de76da623bb4' to
'localhost:5001'.
  Finished uploading layer 'sha256:1f7ce2fa46ab3942feabee654933948821303a5a821789dddab2d8c3df59e227' to
'localhost:5001'.
  Finished uploading layer 'sha256:9dfad474d1d8dd245c9b94fc9034573971c48a627a67a7b13f3118261bb0d626' to
'localhost:5001'.
  Uploading config to registry at blob 'sha256:2a6a83f5968285ae2c94361dc7a805a477885383e56ce1384c8fde3024cf081d',
  Uploaded config to registry.
  Uploading tag 'latest' to 'localhost:5001'.
  Uploaded tag 'latest' to 'localhost:5001'.
  Pushed image 'apiservice:latest' to registry 'localhost:5001'.
        (?) Done:  Building and Pushing container for project apiservice

Executing: dotnet publish "C:\git\Test\Test.AppHost\..\Test.Web\Test.Web.csproj" -p:PublishProfile="DefaultContainer"
-p:PublishSingleFile="true" -p:PublishTrimmed="false" --self-contained "true" --os "linux" --arch "x64"
-p:ContainerRegistry="localhost:5001" -p:ContainerRepository="webfrontend" -p:ContainerImageTag="latest"
MSBuild version 17.8.3+195e7f5a3 for .NET
  Determining projects to restore...
  Restored C:\git\Test\Test.Web\Test.Web.csproj (in 256 ms).
  1 of 2 projects are up-to-date for restore.
  Test.ServiceDefaults -> C:\git\Test\Test.ServiceDefaults\bin\Release\net8.0\Test.ServiceDefaults.dll
  Test.Web -> C:\git\Test\Test.Web\bin\Release\net8.0\linux-x64\Test.Web.dll
  Test.Web -> C:\git\Test\Test.Web\bin\Release\net8.0\linux-x64\publish\
  Building image 'webfrontend' with tags 'latest' on top of base image 'mcr.microsoft.com/dotnet/runtime-deps:8.0'.
  Uploading layer 'sha256:1f7ce2fa46ab3942feabee654933948821303a5a821789dddab2d8c3df59e227' to 'localhost:5001'.
  Uploading layer 'sha256:227fbee7f8333fe9242a4dff10ac036f7038a975be6b3bf7c132de76da623bb4' to 'localhost:5001'.
  Uploading layer 'sha256:eaf0575a2a24037fde65a105ab55d031a3bb7f9f9cd0e4d31f90825e746b67bc' to 'localhost:5001'.
  Uploading layer 'sha256:e31c0404b9b6add954c4ca536e179442ddfe4edaf83d20546daf499ea5a7c918' to 'localhost:5001'.
  Finished uploading layer 'sha256:eaf0575a2a24037fde65a105ab55d031a3bb7f9f9cd0e4d31f90825e746b67bc' to
'localhost:5001'.
  Finished uploading layer 'sha256:227fbee7f8333fe9242a4dff10ac036f7038a975be6b3bf7c132de76da623bb4' to
'localhost:5001'.
  Finished uploading layer 'sha256:1f7ce2fa46ab3942feabee654933948821303a5a821789dddab2d8c3df59e227' to
'localhost:5001'.
  Finished uploading layer 'sha256:e31c0404b9b6add954c4ca536e179442ddfe4edaf83d20546daf499ea5a7c918' to
'localhost:5001'.
  Uploading config to registry at blob 'sha256:2ad8cae70953e0de7505f1960ed891b2dbe251ce53f020f534da6224d54dafa9',
  Uploaded config to registry.
  Uploading tag 'latest' to 'localhost:5001'.
  Uploaded tag 'latest' to 'localhost:5001'.
  Pushed image 'webfrontend:latest' to registry 'localhost:5001'.
        (?) Done:  Building and Pushing container for project webfrontend

Building and push completed for all selected project components.

No Dockerfile components selected. Skipping build and publish action.

Generating kustomize manifests to run against your kubernetes cluster:

        (?) Done:  Generating C:\git\Test\Test.AppHost\aspirate-output\apiservice
        (?) Done:  Generating C:\git\Test\Test.AppHost\aspirate-output\webfrontend

Would you like to generate the top level kustomize manifest to run against your kubernetes cluster? [y/n] (y): y

        (?) Done:  Generating ./aspirate-output/kustomization.yml

 ?? Execution Completed
PS C:\git\Test\Test.AppHost>

you can confirm if the fall back registry has been set so you dont need the msbuild prop if you cat the aspirate.json file

 ?? Execution Completed
PS C:\git\Test\Test.AppHost> cat aspirate.json
{
  "TemplatePath": null,
  "ContainerSettings": {
    "Registry": "localhost:5001",
    "Tag": null
  }
}

I should make container registry default to Y i think in the init command πŸ˜› Noticed you said no ^^

davidfowl commented 9 months ago

Why can't I get it to use my local docker registry when I say no?

prom3theu5 commented 9 months ago

When you say no it just doesn't set any fallback value

The fallback value is only there in case no msbuild prop is set.

I can't really default it to localhost:5001 as it's entirely up to the user what ports they bind their local registry on etc.

The reason it needs to be a registry instead of the local docker context is k8s needs to pull it from a registry it won't use the local docker context. Even if you are running k8s under docker desktop, unless you spin up a registry when you go to deploy to k8s it won't find any images / will fail to pull.

In my testing, even setting pull image to 'never' in the deployments it still failed to find the images

davidfowl commented 9 months ago

The reason it needs to be a registry instead of the local docker context is k8s needs to pull it from a registry it won't use the local docker context

Are you sure about this? I'm not sure that's the case.

prom3theu5 commented 9 months ago

It seemed to be on my Mac

Then again it could also be because I wasn't assigning a fallback tag which results in latest being used

davidfowl commented 9 months ago

@prom3theu5 try deploying a new application without any customization. I see eShopLite has a ton of custom properties for containers, but the SDK defaults everything so there's no need to set those properties in order to publish a container:

Building image 'myfrontend' with tags 'latest' on top of base image 'mcr.microsoft.com/dotnet/aspnet:8.0'.
  Pushed image 'myfrontend:latest' to local registry via 'docker'.
   <PropertyGroup>
     <TargetFramework>net8.0</TargetFramework>
-    <ContainerRegistry>localhost:5001</ContainerRegistry>
-    <ContainerRepository>my-frontend</ContainerRepository>
-    <ContainerBaseImage>mcr.microsoft.com/dotnet/aspnet:8.0</ContainerBaseImage>
davidfowl commented 9 months ago

Let me see if I can get something deployed without a registry, if not I'll run a local one.

prom3theu5 commented 9 months ago

Thanks @davidfowl

Aye - I've just pushed a new preview release 0.1.11-preview With this I've made container registry optional.

In fact - you can skip the entire init command and just use generate now if you wish on a new starter template. In order to make this work on Kube locally under docker-desktop, I've introduced a prompt post build and push, pre generation of manifests.

It will ask which image pull policy you'd like to use, with the default being IfNotPresent. This is what has to be selected for local docker.

If you select Always - kubernetes will try to check the digest of the image instead of just the tag, and because there is no registry this will fail

davidfowl commented 9 months ago

Latest version works like a charm on local Kubernetes? Is there a way to make it save all of the information to avoid prompts?

davidfowl commented 9 months ago

Also, we should discuss what it would take to wire up a default ingress rule to expose one of the services.

PS: This is very cool!

prom3theu5 commented 9 months ago

Latest version works like a charm on local Kubernetes? Is there a way to make it save all of the information to avoid prompts?

Thanks David. Glad to hear it.

You can pass everything on command line that's required as well as --non-interactive for all commands and it'll auto run without prompts.

I like to run 'aspirate build' a lot and pass -m with the manifest file path. That won't regenerate the manifest and instead will just show you the component menu you'd like to rebuild Great when debugging stuff etc

In the next version with input management and container.v0 support I'm replacing the simple config file with the entire state saved between sessions so subsequent runs will have all the previous runs info available in the aspirate.json config file.

Suffering at the moment though lol Little girls had a stomach bug and I've ended up with it. What makes them ill for a day nearly wipes me out lol.

prom3theu5 commented 9 months ago

Also, we should discuss what it would take to wire up a default ingress rule to expose one of the services.

PS: This is very cool!

That's a good idea. Once we have container support in I could add a menu to select which services you you'd like to expose, gets tricky when the service has multiple ports and you only want to select one. That's a lot of menu and a lot of user journey πŸ˜›

Maybe a public service flag in manifest?