ARM Template incremental mode behavior

[SamirFarhat]

Hi, What this the expected behavior of deploying an ARM Template in incremental mode with SFM. My test was the following: 1- Deployed an App with 2 Services 2- Deployed the same template with 1 Service in incremental mode (default)

The result is that the removed service from the template was removed from the deployment. Is this what is expected ? (I know that the Service is not an ARM resource

[mattrowmsft]

That setting relates to resources under the same resource group. Incremental would do PUT on any resources in your template and ignore all others. Complete(or whatever the other option is called) will do PUT on resources in template and DELETE on all other resources in resource group.

[SamirFarhat]

In my example, i had 2 services within my SFM app. Deploying an ARM template with only Service 1 has resulted on the deletion of Service 2. Is this what you are expecting ?

[aloneguid]

I like this behaviour as it aligns with what full SF does.

[guibirow]

In this case I think it is because a service is a component of the application resource, you deploy an application not a service.

On the other hand, applications, network and gateway are independent resources.

This is my understanding, I might be wrong!

[aloneguid]

This is also my understanding. SFM application actually maps to a usual SF app, as I can see when deploying to local cluster and browsing in SFX.

[mattrowmsft]

@guibirow your understanding is correct. The application is the ARM tracked resource so the entire definition of the application is replaced during deployments.

[SamirFarhat]

Let us discuss why this behavior is important for me: We are building a new Application on Top of SFM with a target release version next summer. The Application follows a Micro-services architecture, and hence contains tens of services. Each service (or a group of Services) are handled by a Team which is represented by a CICD pipeline. My question comes here : How can let these teams release only their respective services without being able to touch other services ? If at the end of the CD steps, there is an ARM template that deploys the new container images versions, any deviation from the last ARM template (used y everyone) version will cause the whole app to break.

So i see two options:

• The CD have to be handled by a transversal team that controls the deployment templates life-cycles
• Using an SFM App by team with a shared Network ??

Any idea guys ?

[aloneguid]

I'd personally go for shared network approach and leave the teams with their own application. I think there is a bit of misunderstanding in terminology. An app is supposed to be a self-contained set of microservices, owned by one entity. If a team needs to achieve it's goal by splitting their application (what is often called a microservice in business terms) they are free to do so - other teams shouldn't have an influence in their decisions. Sharing application description resources is a bad idea in long term -now you're locking people into specific tech they might not want to use in future or not ready to yet. Or they might want to use different incompatible versions of SFM in future.

Also I wouldn't have a dedicated DevOps team with SFM, it just never works. Creating hierarchy on top of something simple just pissed people off. Two-pizza teams are much more effective IMHO.

[guibirow]

I agree with @aloneguid, separate applications is the way to go. I've been using SF for a while, I've used single application with multiple services and multiple applications with minimal service dependency, the former is much more flexible and reliable.

Applications should be named service group or something that does not mislead the users in thinking that an application should contain all the services.

[SamirFarhat]

It's nice to see your thoughts guys, this is the best path to take i think. i wonder what @mattrowmsft can add to this. Have you planned how to use CICD pipelines with SFM?

[mattrowmsft]

I agree with sentiment of an application per team. Application is meant to be a deployment unit and I have always found a team owning the deployment end to end leads to the most productive teams.

I think the friction here is that we haven't enabled multiple applications on a network properly yet, which is forcing multiple teams to "co-own" the application. We are working hard on fixing this limitation.

[SamirFarhat]

When do you think this will be possible, for Apps to share the same network, and to see each other using appnsme.servicename? Q1,Q2?

[mattrowmsft]

The next refresh is focused on MSI + diagnostics (although it is different people working on features). I would guess 3-5 months from now.

[SamirFarhat]

Nice. What do you mean be MSI please ?

[ross-p-smith]

Managed Service Identity. https://azure.microsoft.com/en-us/blog/keep-credentials-out-of-code-introducing-azure-ad-managed-service-identity/

[SamirFarhat]

I thought @mattrowmsft won't use the old naming for this service, which is now Managed Identity for Azure Resources :wink:

[mattrowmsft]

I didn't even know about this rename (probably those working on feature did). I think I have finally gotten SF Mesh to stick in my head as opposed to our original seabreeze codename :-).

[SamirFarhat]

We can close this issue, as we do know now that the Application is the unit of deployment. Any chance that Services become an ARM Resource?

[mattrowmsft]

Once multiple apps can have connectivity easily, is there any advantage to services being arm resource as opposed to using 1 service per app?

[fauxcoding]

"How can let these teams release only their respective services without being able to touch other services?"

I've been exploring this at work and I was under the impression that we could use Linked Arm Templates.

https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-linked-templates

Essentially each code repository for a service would have its own subset of the SF Mesh template allowing teams to control a particular service.

A single arm template would be used to deploy the application but it would link each service to a separate arm template.

I haven't tried this out but I was expecting this to be the case.

[SamirFarhat]

@mattrowmsft My problem is the naming. For me an App, is the entire Application. A Service is the 'micro service'. So technically, using multiple-apps will solve the 'governance' question, as each team can control its related App. But the concept and naming will be misleading. I want that we/you work on spreading awareness about this subtilty, in order that SFM adopters know how to design/implement their container-based microservices application.

[SamirFarhat]

@heymega Using linked templates can quickly become very messy, in addition to that, this will not solve our problem. An SFM app contains services. A SFM ARM te.ate for example deploys an App. The App is an ARM resource. The problem comes when two teams owns different services within the same app. Team1 owns Service 1 Team2 owns Service 2 Let's say Service 1 uses a container image1 tag 10. Let's say Service 2 uses a container image2 tag 10. When Team1 makes a new deployment, they need first to update their template with anything made by Team2, otherwise they may overwrite what Team2 has updated. For example, if Team2 service is now using image2 tag 11, Team1 may overwrite this configuration whenver they make a new deployment using their template. This was a short example to show the governance challenge.

[fauxcoding]

They wouldn't overwrite it if they're using a linked arm templates though, surely.

Let's say there is a single arm template for the application with two child templates which are linked. Team 1 and team 2 own their own child template.

Team 1 can't change team 2's child template and vice versa.

Both teams deploy the application template but it'll use each team's specific child template.

I'd imagine the template link points to the Master branch in each team's git repository for the live template.

[SamirFarhat]

The linked template is a template tjat deploys what? An SFM app or an SFM Service ?

[fauxcoding]

The linked template deploys a SFM service.

There is a master template that deploys the application and the linked child services.

As long as you ensure the deployment mode is idempotent (which is default I believe) then this would work.

Now I haven't tried this out in practise yet but I've used linked templates before and we are looking to do this with SF Mesh.

This is how I envisaged it working so I hope it works!

We are still waiting for the SFM to reach GA (which I believe is Jan/Feb time) and this is one of the things on our checklist. Once we've got round to trying it out I can let you know and send you the POC files.

[SamirFarhat]

@heymega in that case'

• who will manage the parent template?
• what if a tem decided to add a new service? They need to ask to update the parent template

Using Linked Templates may be z workaround, but it does fit well in a single responsibility model when teams are independent and can work solely in DevOps/CICD mode

[fauxcoding]

We have DevOps engineers who we plan on managing the parent template but I guess it comes down to your processes and team. I actually like the idea that onboarding a new service requires some sort of approval/person/team to sort out.

I agree thats teams working in silos is ideal but if they're using the same SFM Application there is already a slight coupling.

For example. Let's say we have an e-commerce application with an order service and a shipping service. Even though we have two separate teams working on each service/bounded context they still have some coupling to the application. If that coupling is too much for you, then you can have separate applications, order application, shipping application but then you lose some of the benefits of a single application (DNS for example).

Since the arm template is just a JSON document at the end of the day, you can always come up with something custom built around your processes.

I'd be interested to know how other teams plan on doing this though as we are still early with SFM and are constantly reviewing these processes.