microsoft / service-fabric

Service Fabric is a distributed systems platform for packaging, deploying, and managing stateless and stateful distributed applications and containers at large scale.
https://docs.microsoft.com/en-us/azure/service-fabric/
MIT License
3.03k stars 401 forks source link

Open source Service fabric tools #905

Open aL3891 opened 7 years ago

aL3891 commented 7 years ago

Hello!

Are there any plans to open source the service fabric sdk and build tools? By that I mean the msbuild targets/tasks and the powershell module used for deployment/management. I've started to make some modifications to FabricActutil, however i'd also like to make some changes to the other target files as well as add powershell cmdlets.

Some things i've been looking at/working on

Please do consider it.

mikkelhegn commented 7 years ago

Yes - we are :-)

mikkelhegn commented 7 years ago

Someone pointed me to your symlinks idea - if this is for local debugging - we have that in the tooling today: https://blogs.msdn.microsoft.com/azureservicefabric/2017/04/17/speed-up-service-fabric-development-with-the-new-refresh-application-debug-mode-2/

aL3891 commented 7 years ago

That's great! :) any hints on when that might be? ;)

I know about the symlink support for local debugging, that is great but this is slightly different.

The current tooling creates symlinks directly in the node store to your local debug folder, witch again, is great for debugging. What we've been experimenting with though is creating symlinks to the package folder during build. So when building the sfproj project, instead of copying all the files and manifests from each service to the pkg folder, we create symlinks instead.

This solves two significant build time sinks for us, first, it reduces the amount of file copying that needs to happen, witch is about 450 mb for us. but secondly it allows us to build our entire solution in one go using msbuild, instead of running the package target on each individual sfproject.

The problem with packaging them one by one is that msbuild does not detect that all the dependant services (and their dependencies) are already built, so it rebuilds them again, this also means that we cannot build all our apps in parallel, although I've actually worked around that by using different output folders. the output folder workaround creates even more io though as multiple copies of our dependencies are being built. I detailed this in microsoft/service-fabric-issues#257

By "building" the sfproj using symlinks to create the pkg folder these issues are resolved, I can build the entire solution excluding the sf projects once, msbuild will detect all dependencies and if they're already up to date, then I run our script to create symlinks (if they don't already exsist) and we're good to go.

This also makes the process of creating a diff package more efficient since I don't need to copy all the service files only to then delete them, I just delete the symlink to the output folder of the service instead.

I've a proof of concept with creating the pkg folder using symlinks and it works, but now i'm working on a script to do it for any sfproject. it would be nice to integrate into the tooling though, perhaps as a different package target or a property that can be set in the sfproject file

mikkelhegn commented 7 years ago

@aL3891 Thanks for your feedback. I'll file the above specific scenario as an improvement to our tooling. Let's see if we get to this feature or open sourcing first... :-)

Saphirim commented 6 years ago

Any update on when we might see the CmdLet-Source revealed? Would be especially interested in the routine Test-ServiceFabricConfiguration to assure compliant environments before the installation of a on premise cluster.