search

microsoft / service-fabric-services-and-actors-dotnet

Reliable Services and Reliable Actors are Service Fabric application frameworks for building highly-scalable distributed cloud applications.
Other
269 stars 113 forks source link

Future/roadmap for Service fabric app models #206

Open aL3891 opened 5 years ago

aL3891 commented 5 years ago

On a recent video on channel 9 i left a addmittedly rather salty comment basically noting that most if not all talks, blogs and general information about service fabric is in regards to mesh, containers or some combination there of, and that there had been very few features added to the regular app models, Reliable services and Reliable actors, for quite some time.

Microsoft pushed heavily for these app models, yet they are completley unsupported on mesh with no public road map for when that will change. We were involved quite early in the SF public lifetime and we were assured that reliable collections and actors was the future and something that microsoft would invest significantly in for a log time (hence my saltyness) Im now of the impression is that these app models, while supported in terms of security and stability, will not be getting any significant feature updates.

Am i wrong? Have i missed a build session or blog post where the future for these app models have been laid out? One major gripe we've had for years is the backup api where you have to provide a physical path for data to be written/read to instead of a stream or some other lower level api. Another is being able to programatically access and edit backup data (for the purposes of restoring data to a diffrent number of partitions) as well as explore data in the cluster. Another one is lower level acess to replicated storage. Stand alone reliable collections are another one. Restoring to single node dev clusters is another..

These are things that the team has said that they're been working on for years, and while i have no doubt that's true, or that the team would like to deliver them, they have not been delivered.

So what is the future of the Reliable* models? Are there any planned feature updates? If not that's fine but then what are the migration paths? For our part we're talking the lack of information as a bad sign and our long term planning has us moving from reliable services initially, with reliable actors following some time after that.

Since we're so heavily invested in these app models (especially actors) however this will take alot of time and effort that we much rather spend elsewhere, but sadly, as much as i want to be wrong, we've seen this pattern before and with no information, we're left guessing.

shacal commented 5 years ago

I think Service Fabric app models should open up lower lever access (for speed and less overhead). Im not so intrested on highlevel model, as I usually find that they are too generic with stuff you dont wanna have in production environment. Also Im on the road to sqeeze as compact as possible as many services as possible, it helps to have low level access.

What I also think, is to have production quality, proven patterns and samples. Now I see samples as "here you go, this is one way"... but it does not actually work in larger scale :D

So, treat Service Fabric App model as Kernel, and make sure community will add/create models on top of that :P

ChackDan commented 5 years ago

@msfussell - Is there something you can speak publicly about the work you are doing on the programming model front and may be even solicit some early feedback under NDA ?

darraghjones commented 5 years ago

The silence is deafening....and only goes toward proving @aL3891 concerns.

aL3891 commented 5 years ago

its been two months, still no response @msfussell + team?

amanbha commented 5 years ago

@aL3891 Thanks for asking whats on your mind, I want to address the concerns you raised by stressing that Service Fabric is a supported product with new features added to it. I also want to assure that all production issues raised by users are supported by Microsoft support team and Service Fabric product team. All new feature requests are still honored and are triaged with other work items which the team has to deliver. I would like to distinguish between Programming Models and Application Models. Application Model is how you define your applciation and how it would use other resources like network, services, secrets etc, Programming Model is what users will use to write the services code, reliable collections and reliable actors being examples of programming models. Service Fabric Mesh (name is still TBD) is new serverless offering and has different application model on how you define your applications to run in serverless environment. The fact that it would be running on Service Fabric is an implementation detail and the underlying Service Fabric concepts are not exposed to user as user doesn't sees the cluster underneath. For writing services in serverless environment users can bring container running any code of their choice to be hosted, they define how these services compose an application and uses other resources using the application model. For serverless environment we have on our roadmap a cloud/edge distributed programming model providing state management, Actors, pubsub capabilities in a language agnostic (I cannot delve into more details as it still evolving and will most likely go through multiple change).

Being said all that, I want to close by again mentioning that Service Fabric and its native programming models are still supported and very well alive with features actively worked on.

juho-hanhimaki commented 5 years ago

@amanbha that is good to hear. I think it would be great if development and design was done more on open and there would be public roadmaps and design documentation and discussions. Move more towards the .NET Core / ASP.NET Core development process.

With Service Fabric it seems like there are a lot of undocumented features and since development and design is pretty much done hidden from the community it's really hard to get any idea what is going on. When bugs are fixed, often the issues are still left open and the release notes might not even mention the particular fix. It can understandably be hard to keep track of all issues but I think this is again problem of the development process.

I've heard of NDAs even on this thread and it just doesn't feel like today's Microsoft. After all Service Fabric is developer tool / platform (especially for us who use your programming models). I think you should try to take note how things are done with .NET Core / ASP.NET Core which has been very successful. I assume .NET Core / ASP.NET Core teams have all imaginable resources at their use. Same might not be true for Service Fabric so maybe its challenging.

Some examples:

I follow all the SF repos I know of to try to get even some sense of the development being done, but it feels like there is either little being done or stuff is done behind closed doors. Time from time a new repo appears in some comment in some issue thread and nobody has ever heard of it before. For example this potentially valuable repo appeared in comments of some issue (long before the 6.5 release blog post) https://github.com/microsoft/Service-Fabric-AppDRTool. How are we supposed to know something like that is in design or development and to get involved / provide feedback in any capacity.

I really hope the experience can be improved in the future and the communication and community will be more integral part of the development process.

ChackDan commented 5 years ago

This is fair feedback Juho. We can certainly do a better job with the roadmap and doing more work in the open. Thankyou for the examples as well, these help drive home the points you make. Adding my Dev manager as well, so that he hears this first hand. He and I can take this into account when we staff improvements for the next semester, writeup the release notes, and comments in the PRs.

From: Juho Hanhimäki notifications@github.com Sent: Monday, July 22, 2019 1:36 PM To: microsoft/service-fabric-services-and-actors-dotnet service-fabric-services-and-actors-dotnet@noreply.github.com Cc: Chacko Daniel chackdan@microsoft.com; Comment comment@noreply.github.com Subject: Re: [microsoft/service-fabric-services-and-actors-dotnet] Future/roadmap for Service fabric app models (#206)

@amanbhahttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Famanbha&data=02%7C01%7CChackDan%40microsoft.com%7C6b79ae29b6c84c07de0508d70ee432cd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636994245572703373&sdata=Qa7mzYUFD6Heu5B652wKPiaN75U4pcf9f7oiU8YBykg%3D&reserved=0 that is good to hear. I think it would be great if development and design was done more on open and there would be public roadmaps and design documentation and discussions. Move more towards the .NET Core / ASP.NET Core development process.

With Service Fabric it seems like there are a lot of undocumented features and since development and design is pretty much done hidden from the community it's really hard to get any idea what is going on. When bugs are fixed, often the issues are still left open and the release notes might not even mention the particular fix. It can understandably be hard to keep track of all issues but I think this is again problem of the development process.

I've heard of NDAs even on this thread and it just doesn't feel like today's Microsoft. After all Service Fabric is developer tool / platform (especially for us who use your programming models). I think you should try to take note how things are done with .NET Core / ASP.NET Core which has been very successful. I assume .NET Core / ASP.NET Core teams have all imaginable resources at their use. Same might not be true for Service Fabric so maybe its challenging.

Some examples:

I follow all the SF repos I know of to try to get even some sense of the development being done, but it feels like there is either nothing being done or it is done behind closed doors. Time from time a new repo appears in some comment in some issue thread and nobody has ever heard of it before. For example this potentially valuable repo appeared in comments of some issue (long before the 6.5 release blog post) https://github.com/microsoft/Service-Fabric-AppDRToolhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2FService-Fabric-AppDRTool&data=02%7C01%7CChackDan%40microsoft.com%7C6b79ae29b6c84c07de0508d70ee432cd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636994245572723365&sdata=RGsWebFEz42KOn0Y45%2B24PLfT8qmGX91sJnujSRzlSs%3D&reserved=0. How are we supposed to know something like that is in design or development and to get involved / provide feedback in any capacity.

I really hope the experience can be improved in the future and the communication and community will be more integral part of the development process.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fservice-fabric-services-and-actors-dotnet%2Fissues%2F206%3Femail_source%3Dnotifications%26email_token%3DACVL3JU32F2WXTSDNPXGAR3QAYK2XA5CNFSM4HR76ITKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2RCS3Q%23issuecomment-513943918&data=02%7C01%7CChackDan%40microsoft.com%7C6b79ae29b6c84c07de0508d70ee432cd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636994245572733368&sdata=vAx9OwcUpiZXgJPlB6Ka0cLpTRwhWMJawTz9auqxizI%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FACVL3JV32KTSXQ4HPWX6EUTQAYK2XANCNFSM4HR76ITA&data=02%7C01%7CChackDan%40microsoft.com%7C6b79ae29b6c84c07de0508d70ee432cd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636994245572733368&sdata=2yFYOF5MrCRmSYrgKndLxfY3tuEqCsUxdrozXTdAj4E%3D&reserved=0.

amanbha commented 5 years ago

@juho-hanhimaki I think your feedback around release notes is valid and important, we try to include the major changes in the release notes (not all bug fixes go into the release notes), but if you feel its still not comprehensive, as @ChackDan mentioned above, we will takt this feedback into account. Reg. closing the issues on github, I will circle back with the team to close the issues and mark them with appropriate release when fixed.

aL3891 commented 5 years ago

I want to close by again mentioning that Service Fabric and its native programming models are still supported and very well alive with features actively worked on.

I agree that is nice to hear but i also what to stress that to me, as an outsider, there is very little actual evidence of this.. i dont doubt that the team is working hard, it just seems to be on other things (like mesh)

What features of the native programming models are being worked on? what features have been recently released? i look though the full release notes in every release and there is not much in there that pertains to the native programming models.

Mesh is completley incompatible with native the native service fabric programming models and the most guidence we've heard from the team is some rather vague talks of "something to discuss in the coming months". it always seems to be "in the comming months" each time someone asks though.. can you really blame us for feeling left in the dust?

Heck, even the open sourcing it self of servicefabric seems to have grinded to a halt, to my knowledge you still can not build service fabric on windows, and the source for the tooling is nowhere to be found.

To me it really feels like Service fabric is not really onboard the open source train, service fabric is not being developed in the open, the team arent tracking development issues publicly, https://github.com/microsoft/service-fabric has 64 commits and still doesnt have the 6.5 souces.. just as another example there is an almost 1 month old issue asking what's in version 6.4.670.9590 with zero replies..

It is so incredibly frustrating as a user who has tried to sell service fabric as a solution to collegues and clients, argued the case agasint other solutions like kubernetes, that reliable services and service fabric in general is a long term project that is going to see active development for a long time to see this lack of transparancy from the team.

Just tell us what's happening. Show what you're actually doing instead of giving vague statements. we want to help, we want to make service fabric sucessful, again just look at what the community has done for dotnet/asp.net net.

amanbha commented 5 years ago

@al3891 Actors and Services layer including remoting is built in open in this very repo. All the issues are opened and discussed here. Similarly SF aspnetcore integration is also done in open, and you are currently part of the feature being discussed for development. Granted, there is scope for improvement in release notes to be more comprehensive and rest assured that feedback is taken. Recently the team has posted the long term release schedule for SF: https://github.com/microsoft/service-fabric/blob/master/README.md#service-fabric-release-schedule Hopefully this reasserts the fact that SF is very well and alive. Reg. Mesh, its (name is still TBD) a new serverless offering and has different application model. The fact that it would be running on Service Fabric is an implementation detail and the underlying Service Fabric concepts are not exposed to user as user doesn't sees the cluster underneath. For serverless environment, we have on our roadmap a cloud/edge distributed programming model providing state management, Actors, pubsub capabilities in a language agnostic fashion, it's still in early stages, but once the details are finalized it will be an open sourced project completely developed in open on GitHub. Reg. SF runtime open source plan: @chackdan and @dkkapur, please take this as an important community feedback and share more details about the plan in the SF repo.

juho-hanhimaki commented 5 years ago

If this repo is an example of development in the open it feels like a mess to me. There are no public plans, design, work items, milestones. It isn't clear what is the direction of this repo or if there is any at all. There is no feasible way for community to participate since the only thing that comes in to this repo is finished code without any context (documentation, design, release schedule...). Even the code itself isn't that useful as releases aren't clearly tagged (what code is in what released version?).

Take dotnet as example:

@aL3891 has tried to multiple times to ask what are the things that are being worked on. If the development was done in open we could all see what are the items that are in the next milestones/releases and what are further down in the backlog. People could even provide feedback BEFORE the whole thing has been coded.

Are my expectations of how open-source development should work wrong? Is this closed approach to open source code taken by choice of the SF team? The code is open but not the development. There is a difference.

I am not avid or experienced contributor in open source myself so certainly there are people with more experience in these matters. Still the more open nature of dotnet development has been helpful to me multiple times as the information is available in the internet and not hidden on some Microsoft guy's laptop. Of course the dotnet project has also benefited hugely from the open source as there are hundreds of contributors https://github.com/dotnet/core/blob/master/release-notes/2.0/2.0.0-contributor.md.

aL3891 commented 5 years ago

Actors and Services layer including remoting is built in open in this very repo. All the issues are opened and discussed here.

So am I understanding correctly that all the ongoing feature work is being done in this repo? All the planned feature work is documented here as issues/work items? That would make the on going/planned features.. what? There are 17 open issues in this repo right now and none of them are features. There are no feature branches, no open PRs. looking at the closed PRs 2019, witch fit in a single page by the way, there are only bugfixes. The asp.net integration repo has even less.

I'm sorry but to me it is not any clearer what features if any are coming or what the future actually is for the native programming/app models.

Recently the team has posted the long term release schedule for SF

While I do appreciate any information given, I was hoping for a little more detail than "version 7 will follow version 6" :) Such as what, even in the broadest of terms, is in these releases? i assume those dates are not grabbed out of thin air?

Actors, pubsub capabilities in a language agnostic fashion, it's still in early stages

And so they have been for close to a year with no visible progress.., not a brief outline of the plans, nothing.. Is the plan even to make it available to SF native apps?

I really can not think of a reason for keeping this such a close guarded secret, especially for something that is supposed to be developed in the open.

Contrast with how project bedrock/span/pipelines where developed, david fowler published it to his own repo and got folks like ben addams involved, the project then moved to corefxlab and is now part of coreclr. The community was involved from first inception ałl the way to shipping product. Blazor is another great example of this.

I completely agree with Juho, having the code public is good but in no way does that alone equate to being developed in the open. I'd also like to echo his question if this is a deliberete decision or due to some other circumstances or what.

amanbha commented 5 years ago

@al3891 To give you the full diverse and broader view: Service Fabric team works on various components: Runtime (including reliable collections and data stack), Actors & services Programming models, a managed service(to create SF clusters in Azure), provding supoport to our first and third party existing customers & onboarding new customers. Programming model components is mere one aspect of it which is being developed in this repo and aspnet core repo and every issue/work item is being done here however low the number of changes may be. The Actor models and aspnet core integration are initial feature complete and we focus more on long term supportability and customer reported issues and requests. Lot of issues are reported through Azure support for service running in production, we root cause those issues and fix them in this repo. As far as this repo goes, there are branches indicating the releases. Code changes in this repo comes through PRs. I believe that if you want to contribute to the repo, it would be appreicated, infact I would encourage you to be more involved as I really see strong sense of commitment in you for the project, so a sincere heartfelt thanks for it. Managed service is not developed in open. Runtime development is not fully in open, that's why I added relevant folks to update it and add a roadmap. But the fact that, the team has published future release plan shows the commitment to the product.

Reg. the new programming model, its still very much a proof of concept, I am not sure why you are saying its close to a year. Once its out of that stage and we have a minimum viable product we will open source it, I am assuring your for it as I did it in my previous response.. If you want to get involved early on with it, please join the Azure Advisors.

aL3891 commented 5 years ago

Service Fabric team works on various components

Sure and I get that. Again, I'm sure the team is working hard, but from my perspective it just seems like the programming models are not very high on the list of priorities.

My assumption was that you were doing development and work item tracking internally and then only pushing releases here, but that is not the case then?

I know about the branches in this repo, I'm just bringing that and PRs up as a counterpoint when you say that the programming models have active feature work being done, the activity in this repo paints a different picture from what you're describing.

The Actor models and aspnet core integration are initial feature complete and we focus more on long term supportability and customer reported issues and feature requests

Please do correct me if you feel like I'm misrepresenting your point but to to me that really sounds like the team is basically done with the programming models and will fix issues that are discovered, but not do active feature work themselves but leave that to the community. Would you say that's a fair interpretation?

The problem with this is that as a community member I have no idea what the direction of the project is, what areas is it meaningful to invest in. If the programming models are not going to see feature investments from the team, why should community members make that investment instead of putting that effort elsewhere where there is more active development.

Further, I have no idea if the changes I make are going to be obsoleted by some other change that the team is doing internally. For example, are serverless and native going to converge or drift further apart? Will concepts from serverless come over to native or the other way around? With this lack of transparency it's very very difficult for the community to be motivated to do additional feature work.

Service fabric is very broad, so while I'm sure there is a big commitment to it as a whole it's far less clear what direction actually is going.

Going back to the dotnet team again, they've come flat out and said they will not bring workflow foundation and web forms over to core. Now with the community knowing that, alternatives and forks have sprung up. But that was only after they where open with the community what the plan was.

I would much much rather have the SF team officially declare the programming models done in terms of features and active development, only to receive critical fixes from Microsoft, rather than being kept in the dark. Again like the dotnet team has done with full framework 4.8

Reg. the new programming model, its still very much a proof of concept, I am not sure why you are saying its close to a year

I say that because when we first heard about mesh, about a year ago (I think), our very first piece of feedback was "well what about the reliable* programming models, our entire solution are built on those on your recommendation" and the awnser we got was something to the effect of "we're working on a solution, we'll talk about it soon" and that's the same awnser we and others have gotten since then everytime the question is asked.

.. If you want to get involved early on with it, please join the Azure Advisors

I'm sorry but I'm not familiar with that, how do I join it?

NickDarvey commented 5 years ago

Not to just add kindling, but... ...to see other Microsoft teams not continuing with their Service Fabric integration (e.g. https://github.com/dotnet/orleans/pull/5073) doesn't do well for my faith in the future of the programming models. I understand that it is "up to them" to resource it but that does little to allay.

First-class statefulness was what brought me to SF over K8s. It'd be a shame to see that stagnate.

JustinKaffenberger commented 5 years ago

First off, I despise the concept that a given repository needs a certain 'rate of contribution' to be considered healthy; I do think it's possible for things to reach a 'feature-complete' state, only needing security patches. However, Service Fabric is a platform that underpins enormous projects, so to not have visibility into the processes and procedures that are in place to keep Service Fabric healthy and relevant makes it a really tough sell to stakeholders (not to mention my teams which have to invest time in to learn).

Quite literally, all the Service Fabric team needs to do is be more transparent, and I'd be happy. The programming/deployment model is fantastic and really makes a micro-service-oriented architecture a feasible goal for traditionally monolithic, SOA teams.

This is the bedrock that we want to support our larger, cloud-native ambitions with; we want to make sure we're not building on sand.

amanbha commented 5 years ago

@aL3891

My assumption was that you were doing development and work item tracking internally and then only pushing releases here, but that is not the case then?

For SF runtime (which includes Reliable Collections as well) that is the case.

Please do correct me if you feel like I'm misrepresenting your point but to to me that really sounds like the team is basically done with the programming models and will fix issues that are discovered, but not do active feature work themselves but leave that to the community. Would you say that's a fair interpretation?

To clarify, the plan currently is that team will take on work items including bug fixes and feature work as requested/reported by users. Community contributions are always welcome.

For example, are serverless and native going to converge or drift further apart?

I believe that the fact that it was called "Service fabric Mesh" is what is causing the confusion, serverless offerring is a managed service, users don't know that its running on SF, the fact that its running on SF is an implementation detail and underlying SF cluster is not available to user. So native and serverless are completely different things and any programing models for serverless cannot take dependency on SF apis or the underlying implementation. The native programming models are for SF and are supported for use by both external and internal customers. I would like to stress again, like I have done multiple times in this conversation that SF runtime & native model is still supported, both first party and third party large scale applications are built and run on it. The fact that the team published a long term release plan asserts that. I have also provided feedback to the team to be more open about the direction of runtime on SF github repo.

I'm sorry but I'm not familiar with that, how do I join it?

I reached out to team which handles this program, and to get the process started, it would need your corporate email info, could you please share it with me with me and I will connect you with the team which handles it.

@NickDarvey First class statefulness is integral to SF runtime, system services require statefulness. Statefulness is exposed to user services using Reliable Collections and is not stagnating.

aL3891 commented 5 years ago

@JustinKaffenberger

I do think it's possible for things to reach a 'feature-complete' state, only needing security patches.

I absolutley agree, though i'm not sure the programming models are at that point yet. More than anything though what the teams plans are is what i'm after. If the team are not going to put more features in, i'm fine with that, as long as i know that's actually the plan

@amanbha

For SF runtime (which includes Reliable Collections as well) that is the case.

And the other SF components like asp.net integration and programming models? (it sounds like they're diffrent based on your phrasing)

To clarify, the plan currently is that team will take on work items including bug fixes and feature work as requested/reported by users. Community contributions are always welcome.

In other words, no new features initiated by the team? The future direction of the programming models is basically up to the community? As i've said, getting community support is very dificullt without some kind of direction, people are generally unwilling to spend alot of time implementing something if they dont know it aligns with the project and has a reasonable change of getting accepted/used.

I believe that the fact that it was called "Service fabric Mesh" is what is causing the confusion

Perhaps, but that is what microsoft is calling it everywhere its talked about and it has been branded as the future of servie fabric.. you can't blame people for not understanding what is not communicated.

I would like to stress again, like I have done multiple times in this conversation that SF runtime & native model is still supported

You have to separate the notion if being supported and being under active development. .net framework 4.8 is supported, it will recive critical security fixes, but it is not under active development, it will recive no new features. That is what the question is. Are the app/programming models going to recive new features, like the ones the team has promised/the community has requested for a long time? (backup explorer/backup api/low level replication access/calling a specific stateless instance to name a few)

The roadmap you're refering to is better than nothing but not by much, it could just as well refer to the serverless things the team seems to put most of their current focus on. Simply listing the version numbers of upcoming releases tells the community basically nothing. It also says nothing about the roadmap for the other aspects of Service fabric. As i've said, i'm not doubting the runtime will continue to exsist, the question is where its going.

If it is infact under active development, it would to much to reassure the community to show what you're actually doing, what the plans are.

Could you please share it with me with me and I will connect you with the team which handles it

They/you can contact me at allan at funrock.com

alexmarshall132 commented 5 years ago

Not to just add kindling, but... ...to see other Microsoft teams not continuing with their Service Fabric integration (e.g. dotnet/orleans#5073) doesn't do well for my faith in the future of the programming models. I understand that it is "up to them" to resource it but that does little to allay.

First-class statefulness was what brought me to SF over K8s. It'd be a shame to see that stagnate.

I have the same concerns and I'd very much like to know what Microsoft's roadmap is here. The Reliable* Programming Models are fantastic and they allow my team to be very productive. I would very much like to see them carried forward into Service Fabric Mesh.

dkkapur commented 5 years ago

Hi folks, thanks for the feedback on this thread. We need to do a better job in terms of providing visibility into the work we are doing. @athinanthny @masnider and I are looking into how we can improve the content being shared on Azure Updates, this repo, and other places for this. If you have any suggestions, please let us know.

Here’s a quick summary on things we are working on now that hopefully addresses some of your questions –

aL3891 commented 5 years ago

Thank you for the update @dkkapur, it's very much appriciated, looking forward to trying those things out, the new node agent sound and CRM stuff sound especially interesting.

As for suggestings i really recommend looking at Arcade, the shared reusable tooling the dotnet teams are using for their repos. it contains tools for building, creating nuget packages and syncing git repos. also, the winforms team have documented their process of moving here (though i cant read it since im external, i assume you can though)

i'd also really recommend that you try and find ways that the community can help you and try to turn them into issues on each repo. You basically have a bunch of free labor here, we can and will help you but people need direction and a place to start. these doesnt have to be very fleshed out either, just bullet points that can be discussed. it could also be process things that you'd like to automate away, like creating builds and packages that you'd mentioned takes alot of time. these are also not really specifc to the product so they should be easy to share and also serve as documentation for new members of the SF team

I think build tools would be a great start here, that's something that doesnt require a ton of domain knowledge about service fabric internals and something that both you and the community will benefit from alot.

dkkapur commented 5 years ago

Agree with your points strongly, thanks for that (and the link to the doc does work for me, thanks). Going to follow up with some of our teammates here over the next days and report back with some semblance of a plan that we can start to execute on.

oising commented 5 years ago

@dkkapur Actor model on serverless? Interesting! Do tell more.

ChackDan commented 5 years ago

@msfussell is leading the effort to get the 'Actor model on Serverless' . He is currently jumping through all the legal and political hoops, before he and @amanbha can invite contributions and comments from all of you. My hope is that it happens early Q4CY2019.

aL3891 commented 5 years ago

Sounds great @dkkapur and @ChackDan i'd love to help with these things :)

stephenvisser commented 5 years ago

I'm assuming Durable Entities is what is meant by 'Actor model on Serverless'?

Worth considering if you're looking into Service Fabric primarily for the Reliable Actors functionality like me. Based on this, it seems like GA might be this year still...

KatteKwaad commented 5 years ago

I find myself still a little in the dark regarding which direction to take our clients even after reading this and trying to find other sources of info. @dkkapur , @amanbha, your advice here would be great:

To me, it comes down to a question of whether you should containerize or not. We have some green-fields clients who are entering the world of Micro services and almost exclusively run on MS tech. They are investigating Kubernetes et al and I informed them on the Service Fabric programming models that would not require them to invest in container tech. They will most likely utilise a mixture of actors, stateful/less services.

If a server-less offering such as Mesh appears and they have the option of not managing a cluster, that would be something they would most definitely want to use. I have mentioned Mesh to them but cannot make sound decision yet. Problem is: if I advise them down the path of Actors, Stateless/Statefull Services and tell them afterwards that they now need to containerise, it would not go down well at all.

If you can shed some light on this, it would probably answer some concerns others have too as I am getting mixed signals around this topic.

olitomlinson commented 5 years ago

I'm assuming Durable Entities is what is meant by 'Actor model on Serverless'?

Or @stephenvisser they are working on a Managed Akka solution? I'm completely speculating....

olitomlinson commented 5 years ago

@KatteKwaad "To containerise or not to containerise, that is the question!" - Your concerns are something that I've spent a lot of time going over, and to be honest, I don't think I can ever be fully confident that I've made the right decision because all I can ever do is make decisions on what I know today, not what I might know to be true tomorrow, next week etc.

I've settled on trying to understand what matters the most to the client/customer and get them to understand and agree the approach I will offer - being honest about the challenges and uncertainty along the way.

Is this a brand new thing for the customer? In an unproven/volatile market? Do they need results and traction fast? These kind of questions put an emphasis on understanding the customers agility and time to market needs. If the answer is Yes, then I start prioritising technology that aligns to those needs, such as PaaS/Serverless.

Maybe what you've been asked to build is a v2 something that already has a stable market proposition, you just need to do it faster or better or at a larger scale etc. If that's the case then you can start to explore the possibility of lifting and shifting the v1 stuff into Containers and focusing on removing scaling or reliability constraints or whatever the problem is. Anecdotally, My org is currently going through this option and they will likely settle on k8 because that's what seems to fit the needs of the business with the least friction and the highest certainty.

On the other hand, I'm working on another project for my org, which is focusing on using Service Fabric Cluster. Simply because I need co-located durable state with a strong multi-tenancy story, which k8 can't offer.

I completely sympathise with you. Its incredibly difficult. But if in doubt, keep going back to the customer, get under their skin and find out exactly what success looks like to them, and then focus your plan on achieving that success with as much certainty as possible.

One thing I always try to do is ensure the customer understands that software doesn't just come to an end - its a series of ever moving targets and all we can do is be honest about what targets we need to hit today and what can wait.

Apologies if I'm rambling or telling you stuff you're already well versed in. I guess what I'm trying to say is, you are not alone in this struggle!

shacal commented 5 years ago

” On the other hand, I'm working on another project for my org, which is focusing on using Service Fabric Cluster. Simply because I need co-located durable state with a strong multi-tenancy story, which k8 can't offer.”

Sole reason Service Fabric is for hardcore, websites can be run with k8 all day long. With hard core, you need to get hands dirty, there is no ready made solution to your unique problem... Service Fabric allows you to build it, with working shell and proven track record.

darraghjones commented 5 years ago

How does Dapr fit in with the future of Service Fabric?

https://cloudblogs.microsoft.com/opensource/2019/10/16/announcing-dapr-open-source-project-build-microservice-applications/

Specifically, this bit sounds rather familiar:

Virtual actors – A pattern for stateless and stateful objects that make concurrency simple with method and state encapsulation. Dapr provides many capabilities in its virtual actor runtime including concurrency, state, life-cycle management for actor activation/deactivation and timers and reminders to wake up actors.

olitomlinson commented 5 years ago

@darraghjones I echo your question. Hopefully the Team can offer us some insight soon.

With regards to Actors, I got this from Mark Fussell on Twitter “The Dapr .NET client libraries were literally migrated from ReliableActors”, so yes it seems reasonable to assume that some of the concepts from SF/Mesh have evolved into Dapr.

Which leaves the question of how does Dapr and SF coexist? Are they complimentary? Mutually exclusive etc

amanbha commented 5 years ago

@olitomlinson & @darraghjones Dapr is a programming model runtime and SF is a hosting platform. Dapr can run on multiple platforms with some platform specific integration and on SF too. Dapr's building blocks like pubsub, triggering, state management can be used in SF. SF Stateful services provided by SF stores the state on cluster nodes, where as Dapr takes a pluggable mode in which state is stored in a configured state provider (redis, cosmos etc). Reg. Actors, Actor framework within Dapr is built on learnings from SF reliable actors, it allows the actors to be written in multiple languages and can run on multiple hosting platforms, in fact the Dapr's C# sdk is heavily influenced by SF Reliable Actors If you are on SF cluster and using C#, SF Reliable actors uses SF platform capabilities to the fullest including in cluster state replication

darraghjones commented 5 years ago

@msfussell - Is there something you can speak publicly about the work you are doing on the programming model front and may be even solicit some early feedback under NDA ?

@msfussell is leading the effort to get the 'Actor model on Serverless' . He is currently jumping through all the legal and political hoops, before he and @amanbha can invite contributions and comments from all of you. My hope is that it happens early Q4CY2019.

Am I correct in now assuming these two comments were in reference to Dapr?

msfussell commented 5 years ago

Th Dapr project is about bringing actors to another platforms including serverless ones such as SF Mesh. Dapr is public now so feel free to talk about this. You will see that Dapr C# actors are very similarto SF Reliable Actors. You can get engaged wit the community here https://github.com/dapr/dapr

MedAnd commented 5 years ago

Any chance @msfussell you can comment on OAM for Service Fabric... rudr is for k8s but Atlas surely must be using an OAM version targeting Service Fabric?

peteraritchie commented 4 years ago

W.r.t to DAPR, this seems related: https://cloudblogs.microsoft.com/opensource/2019/10/16/announcing-dapr-open-source-project-build-microservice-applications/

kosperera commented 2 years ago

I see this thread has not been active for nearly 2+ years. I see the release notes have improved and has some mentions about upcoming features, but I'm trying to find a roadmap for Service Fabric and its ecosystem, somewhat like the Azure Functions team does: