Closed orchardbot closed 9 years ago
@bleroy commented:
I'd put media in core as well, as well as the Utilities.
There's a contradiction in that indexing and search are in the core list, but also in the search list. I would just put all of search but Lucene into core.
Scripting (at least the core Scripting module) should go into Core as well.
We already know that what's in the core distro is stuff we think is useful to most. I'd modify the rule so that now all modules satisfying that condition go into the Core module unless they are likely to be replaced or have more than one implementation: Lucene, MediaServices, Scripting.Dlr, etc.
@csurieux commented:
When you say Core is it Orchard.Core or Orchard.Framework ? Is this distinction still relevant ?
IntranetFactory commented:
I think this idea needs some adjustments - merging some small projects into larger ones might be nice to have, but bloating Core will break our solution development in several ways:
For our own development we moved each module into it's own repository. This makes team development and progress tracking so much easier. I'm wondering why all Orchard modules are currently in one "monster" repository? So when starting to clean up the project structure I'd like to suggest to move the optional modules (like Azure, Scripting, Media etc) into their own repositories. Seeing the changes per module or at least functional area instead of one large stream of changes from everywhere would make tracking at lot easier. If the responsibilities for the different repos become delegated as well then I think it should also become easier to contribute and to manage pull requests.
@Piedone commented:
Merging multiple modules into single projects is great. I don't find bloating the core a good idea either. Actually, I'd rather see all modules - regardless of being "Core" or not - in the Modules folder, the only distinction being in the Category set to Core (as it is with Orchard.jQuery: it's not in the Core folder but nevertheless it's a Core module and can't be disabled).
I also agree with the concerns nr 1 and 2 of IntranetFactory.
@sfmskywalker commented:
@Bertrand - Thanks, I removed Indexing and Search from Core for now until we reach a conclusion regarding IntranetFactory's and Zoltan's concerns.
@Christian - When we say Core, we refer to Orchard.Core. Orchard.Framework doesn't provide modules. Rather, it provides a framework that sustains modules.
@IntranetFactory - I don't think it's bloating core if we only include modules that are useful to the majority of Orchard sites - even if they don't enable all features. I think that the advantage of having less assemblies outweighs the disadvantage of having a larger assembly.
Interesting. I actually find it much nicer to have all modules be part of a single repository, because I get all changes using a single pull. Question: does your team pull from each repository manually, or did you automate that somehow?
@Zoltan - So would you prefer to have just a handful of projects inside of the Modules folder, where each project hosts multiple modules? Each project would then represent a category of modules (as outlined in this issue), hosting one or more modules.
@Piedone commented:
@Sipke regarding your comments on my remarks: yes, exactly.
I'd happily see an Orchard.Core project inside the Modules folder and other ones each hosting multiple modules. I'm all for reducing the number of projects/assemblies (although for me personally it's a minor inconvenience to have 90+ projects in a solution instead of let's say, 20) and I'd generally welcome the option to have multiple modules per project.
I am, however, against maintaining a different class of modules (currently Orchard.Core: though this is the status quo I always disliked it). I don't see any practical difference in having some "Core" modules in a different location than the Modules folder but I see that this causes massive confusion for almost all newcomers I met. I don't see any value in such a distinction either: as the example of Orchard.jQuery tells (what massively adds to the confusion of "what does core mean?") there is no need to have Core (i.e. always-on) modules not in the Modules folder.
Now I see even less value in moving some modules to a different location when they won't even be always-on modules. What would be the point? We maintain and support all modules that are in the Orchard solution anyway.
Having all modules in the Modules folder is a crystal clear approach. Having multiple modules in a single project is a great convenience feature. Having some special, always-on modules is well, an exception that I think we have to live with (although I also dislike that there are references to Core modules hard-coded in several places) but there's no need to add more complexity.
IntranetFactory commented:
but I think the pros outweigh the cons here, which is startup time. Did anybody some testing that startup time will be impacted? I'm wondering how loading one large assembly containing a bunch of code not being used by us (e.g. Workflow) or only used quite infrequently in most installations (e.g. Import/Export) can be faster then NOT loading that module at all.
So what's the main reason for the idea to start the restructure? What should be achieved?
Improve server/application pool startup time? Is the number of assemblies impacting startup time? Maybe then having a build step to merge/optimize assemblies might be a better idea then bloating core.
IntranetFactory commented:
but I think the pros outweigh the cons here, which is startup time.
Did anybody some testing that startup time will be impacted? I'm wondering how loading one large assembly containing a bunch of code not being used by us (e.g. Workflow) or only used quite infrequently in most installations (e.g. Import/Export) can be faster then NOT loading that module at all.
Is the number of assemblies impacting startup time? Maybe then having a build step to merge/optimize assemblies might be a better idea then bloating core.
@csurieux commented:
If something has to be done in the area of modules, I hope we will consider before the existing bugs as this one https://orchard.codeplex.com/workitem/18610
@sfmskywalker commented:
@Zoltan - I wouldn't mind moving Orchard.Core into the Modules folder either - it would merely clarify that it is a project containing modules. Zoltan, how would you restructure the list of modules?
@IntranetFactory - See the answer: http://stackoverflow.com/questions/13418108/how-does-the-number-of-classes-in-an-assembly-impact-performance. In summary, the biggest expense is finding the DLL - the more DLLs, the longer it takes to load the application. So yes, the number of assemblies is definitely impacting startup time.
@Christian - There's a gazillion things to be done - let's take'm one at a time.
Vite commented:
There are modules that must be transferred to the nucleus by definition. I think it depends largely on the concept of the application architecture. At this stage, Sipke working on module Orchard.Layouts, and this may need to restructure the basis of the application. For example, the concept of the zone for Orchard.Widgets module loses its original meaning, and this is normal.
I think to solve this problem you need to open a new thread, and vote.
+1 in favor of the proposal Sipke
@Piedone commented:
@Sipke: I agree with the structure apart from too many modules being in Orchard.Core. In my opinion Core should only contain modules that are either always enabled or have implementations for very basic services. I.e. those modules that you'll surely need to start up your site if you don't want to implement everything yourself. So this includes every module now in Core and e.g. Users, Roles, Autoroute, Alias. I wouldn't include recipes, or if that is included, then also include Packaging (Recipes has an implicit dependency on Packaging).
All that said I understand the concerns of IntranetFactory: if you want to customize what to include in your solution (and this is a fair use-case; we could even have official solution versions) then it's a lot better to have separate projects.
IntranetFactory commented:
So if the only performance improvement is startup time then I don't understand why that's a problem which justifies giving up the option to get a lean configuration based on your needs.
Wouldn't adding ILMerge to the build process allow to get everything in one DLL or into a smaller number of assemblies to improve startup time even a lot more without messing with core?
From my understanding Core should only include what's absolutely needed by everybody to run a production site. The current solution configuration comes close to that goal. There are some points which can be improved to cleanup the architecture (e.g. we think jquery doesn't belong to core)
Bloating core is from our point of view a step in absolutely the wrong direction! Why should e.g. workflow or content permissions be included in core and deployed when they are not used in production? More items being deployed just means losing flexibility, adding more complexity, having more moving parts, more code to test and maintain in every production deployment.
From our point of view the proposed changes will fundamentally change the direction of Orchard from being a powerful and flexible platform for any kind of web applications to just another bloated, all-you-can-eat CMS.
@sfmskywalker commented:
OK - I kind of like the idea of Orchard.Core ONLY containing features that are required for any Orchard installation, so maybe we need to package things up differently. That way, you could have an Orchard installation without any module besides the ones that come from Orchard.Core. We would have an additional project called something like Orchard.CoreDistribution that contains most of the current modules. You can still get rid of the modules you don't want by simply deleting them.
Does this sound at all fair?
ILMerge sounds interesting, but not sure if that works with Orchard's dynamic compilation feature and module discovery. I suppose it could be investigated by someone. Personally I'd prefer a solution with as little projects as possible.
@StanleyGoldman commented:
The other major benefit would be to Visual Studio itself.
Less waiting for Visual Studio to finish loading projects when you open the solution. Less time spent building. Maybe Visual Studio will use less than 600 MBs of Ram.
A lot of people like to use programs like Resharper, the number of projects with ReSharper is crippling. Memory usage goes up to 1GB regularly.
@sfmskywalker commented:
Yep, and there's that.
IntranetFactory commented:
Having a minimal core and maybe a cms foundation package adding the most commonly used other modules seems to be a better idea. Maybe that foundation should be splitted also in to modules being used in production or not.
I don't think that VS becomes a lot faster as long as the total number of codes lines you have in the solution doesn't change.
The best improvement in VS we got so far is to switch completely to SSD drives.
Being able to streamline what you need to have in your development solution is the main reason we would prefer to see Orchard being broken down into multiple repositories and to keep the module structure fine grained. This would allow to have just a minimal core and the modules you are currently developing in your solution. Modules and code you don't use can then easily excluded. All modules will then only be required at build time. I'm pretty sure that this would speed VS load times and memory a lot. At least this approach works quite well with our small team of 5 developers. We currently do also a lot of development with Google Polymer. That project has a large community and large number of contributors and use also the approach of having each component in a repo of its own. So it works also well for large teams.
Having multiple repos makes it so much easier to track progress of the development of each task as you get a clean history of just the changes for a specific module. Seeing only an aggregate history of all changes by all developers makes agile development a lot harder then it could be.
AimOrchard commented:
I'm not a fan of the multiple repos idea, but I can confirm: the biggest time (life?) saver was getting my SSD at work.
I advice anyone who works on Orchard to use a SSD drive to put their whole solution on! You'll be glad you did!
@csurieux commented:
Yes, VS should be delivered with an SSD :), great that prices are lower now. I must admit that I share most of the args of @IntranetFactory.
Concerning loading speed + memory used, should'nt we speak of autofac and the number of modules to parse and related structures it manages in memory, may be also switch to last 4.5.2 improvment could help with more async processes.
I would had appreciate to have a higher level of granularity, as for exemple for multitenant which I never use and slow down Orchard: I would appreciate to be able to fire an Orchard without multitenants.
This discussion spreads over many scopes and concerning the building of assemblies and VS, I think the usage of NuGets in place of /lib would be a great thing to have, beeingable to switch up/down .net+related versions (even if there is a price to pay in compatibility).
@Piedone commented:
I think the biggest argument here against consolidating multiple built-in modules into single projects is the loss of flexibility in terms of solution (and thus: build and deployment) customization where you can cherry-pick what you want (since deleting folders from projects is a lot more upgrade-endangered modding than excluding projects from a solution). Flexibility is one of the core values of Orchard instead of being opinionated about what you're supposed to need.
I think we gathered a lot of theories about what would change in terms of performance but I think before moving any more forward we should make some measurements too.
Any way I'd be glad to at least see the ability to use such compacted modules for custom modules.
@StanleyGoldman commented:
An SSD drive was the biggest improvement to using Orchard. Although, running Orchard on an SSD will drastically reduce that SSDs lifespan. Better you and I than a few Western Digital drives I'm sure.
It's not conjecture... VS becomes much faster if half of these projects weren't combined. ReSharper would become useable again.
All of my developers wouldn't complain about how slow everything is in Visual Studio.
I'm really sad that everyone trashed what was about to be done for this ticket.
@StanleyGoldman commented:
VS becomes much faster if half of these projects were combined.
@Piedone commented:
Nothing is trashed. This is an ongoing discussion where we gather the pros and cons. Actually I think this is one of the best discussions of this type, and such invasive changes should be made like this.
@Piedone commented:
And to add: it's great to weigh everything because Orchard is neither Sipke's, Bertrand's, Christian's or mine project, it's ours. A change might be super great for me but might break somebody else's workflow, like it is now with IntranetFactory's example.
@jeffolmstead commented:
I have been following this thread and am loving it. I agree that it is important that everyone share their opinions and how it might help or hurt what they are doing. At the end of the day, we all benefit from having Orchard remain strong as a product and not become fragmented.
Having read through all of the items above, it appears to me there are the following considerations surfacing:
Whether that is useful or not is up to you, but sometimes consolidating the list of thoughts helps to focus in on the options (pros and cons). Feel free to copy / expand the list if I missed anything.
@sfmskywalker commented:
Thanks jao28, for summarizing, that's helpful.
So considering everything discussed so far (besides repository management), how about the following:
Despite the fact that it might be a little harder to get rid of modules you don't want to deploy from a single project, I don't see a disadvantage of that weighing up against the advantages of combining modules into fewer projects - there's no disadvantage of code sitting inside of an assembly that's not used - it won't execute, so it won't impact performance. You don't pay for code inside of an assembly when that code isn't used.
And on the other hand, at least 2 major benefits are to be gained when combining modules into fewer projects as mentioned before: faster startup and faster Visual Studio with ReSharper.
So here's my updated proposal (note I went ahead and added and distributed all modules besides the ones already in Orchard.Core since they ain't going nowhere):
Orchard.Core
modules to be added:
Orchard.Foundation
\ Future module*
Azure
Caching
Media
Scripting
Search
Utility
Not combined modules (each in their own project)
That would bring down the number of projects that host modules down to 9.
How do we feel about this?
@StanleyGoldman commented:
Fine here
IntranetFactory commented:
I'm sorry, but the only difference I can see is that the problem is now called Orchard.Foundation instead of Orchard.Core
It's so sad and frustrating to see that one of the core reasons we'd chosen and invested in Orchard should be nuked: flexibility and modularity. Why have modules when you can't select them? What's next: removing AutoFac because it's also adding overhead and slow on slow machines?
there's no disadvantage of code sitting inside of an assembly that's not used
That's simply not true. After stuffing unwanted modules down my throat
So for us it's essential to be able to build a system with just the modules we need.
So when this "restructure" isn't stopped, we are essentially forced out of the Orchard future. Why? Because a wonky "ReShaper" doesn't work for somebody? Because Orchard should not be a platform any longer? Being just yet another CMS is enough?
AimOrchard commented:
It is not because they are combined, that such combined modules cannot be split up in features.
IntranetFactory commented:
But if each module isn't a project any longer, I can't exclude modules I don't want. I can't ensure that they are not enabled accidentialy. So from my point of view foundation isn't modular any longer.
Freedom and flexibilty comes with a price. Nobody has facts how much startup becomes faster, but nevertheless the flexibilty should be nuked.
From our point of view a slightly longer startup time (if at all) is nothing we would trade for not being able to get a flexible system anylonger.
If you don't have unwanted code in VS, VS is already fast enough. In the future you can't exclude unwanted modules any longer. So for us this change will also make VS a lot slower
and no: deleting unwanted folders is not a solution. it's like telling that you can already copy everything into one project if you want to have it combined
that's so frustrating
@csurieux commented:
Why don't have several solutions for code: lean and common, (as a startup before adding others) ?
@Piedone commented:
@IntranetFactory: BTW you can prevent a module from being used in a number of ways but the most simple one is to hide it from the admin by editing that tenant's shell settings and not include that module (in the same way you can restrict which themes are available for a tenant, which can also be the Default tenant).
Although I, personally, would like to see fewer projects for the sake of structure I don't buy the "it will be faster" argument before seeing some measurements :-).
IntranetFactory commented:
@Piedone We don't use tenant's - so we can't use shell settings to just hide modules. And even if they're hidden - the change will force us to lot a large number of unused modules, so our build time will at least double and VS becomes slower and requires more memory.
So this will lead to the exact opposite of the alleged reasons for melding down orchard - this task does not only improve nothing, it makes everything slower.
I also don't see a reason why somebody should care about the possibility to maybe save a few seconds when rebooting a server?
And I think it's a big, big maybe if there is any relevant gain at all. I've just checked to request a basic home page after a restart. I'd had over 800 accesses to the file system for non existing files. I know that this is by architecture and required to find the right version of every file in the right location. Getting rid of these requests will probably speed up the system more then combining 30+ small assemblies into one large foundation assembly (especially given the fact that not all of them will usually be loaded, e.g. we usually just use 6-8 of them). Are you planning to remove the possibility to customize views in themes next? This will also improve start up time.
So maybe a new Orchard.Core should contain all web platform features probably required by everyone and a new "CMS Foundation" combining all content management specific features:
New Orchard.Core Alias, Autoroute, Caching, Modules, Recipes, Scripting, Setup, Warmup ImportExport, Roles, Users, Themes, ContentTypes, Deployment, AuditTrail, Email, Migrations, JobsQueue, SecureSocketsLayer
New Content Management Foundation ContentPicker, CustomForms, ArchiveLater, Tags, Tokens, ContentPermissions, Fields, Forms,jQuery Layouts*, Lists, Localization, Markdown, Pages, Projections, PublishLater, Taxonomies, Templates, TinyMce, Widgets
New Social module: AntiSpam, Comments
Don't merge: MultiTenancy, TaskLease, Workflows
AimOrchard commented:
Building the solution is heavy for 2 reasons : many files AND many output folders!
It copies around a lot of duplicate files to each project' output folders.
The less projects you have, the less files need to be copied there. If I have to guess, it'll make it FASTER for you to build (certainly if you are not using a SSD drive to build on)!
@StanleyGoldman commented:
Not to mention, your SSD will live longer.
@bleroy commented:
Let's base this on evidence, as suggested by IntranetFactory. We should definitely have performance numbers proving whether this is advantageous or not. IntranetFactory could provide their list of unwanted features so we can benchmark that as well as an additional data point.
The things we want to measure are, I think:
What did I forget?
To be clear, we are not throwing out modularity or flexibility. After years of observing Orchard usage, we have a pretty good idea of what modules are almost universally used, and what pain points are.
The number of projects in the solution is clearly a pain point that needs to be addressed. There are large Orchard users that have up to 150 projects in their solutions. The problem can be partially addressed by the customer, by having a more rational organization based on features within a smaller number of projects, but they can't do anything about the growing number of core modules (that they do need and use).
The reason why we have features within modules was that we knew from the start that we wanted to be able to do what we are proposing here. Our solution was to separate the unit of compilation from the unit of functionality. They are clearly two very different concepts, and there was no good reason to make them the same.
What we are proposing here is to use that capability of Orchard to reduce the number of compilation units (projects) while keeping the number of units of functionality the same.
While I understand the concern that this may make it more difficult to build a leaner Orchard for a specific application, I think we may be able to get creative about this. If the features are correctly organized in folders, it should be easy to exclude those folders, right from Visual Studio. This way, they wouldn't get included in the build process, and would not affect performance or the functionality of replacement modules. They also won't be available to be enabled by mistake by users. They are also very easy to add back. One downside would be that merges of the project file may be a little trickier.
The missing file hits issue that IntranetFactory mentions is an interesting one, but I think it's a completely separate problem that should be the subject of another ticket. Please file it if it doesn't already exist.
@jasper-d commented:
I like the idea of getting some facts (numbers) before making any definitive decisions, especially since I share IntranetFactory's concerns.
In addition to that, I think that putting multiple modules in one project comes with the risk of adding hard dependencies between these modules. Not on purpose but by mistake. Right now one have to add a reference to project A if module B depends on it. Once a few dozens of modules are part of core a dependency would be just one using
away.
I also don't see the advantage in terms of structure. Right now pretty much every piece of functionaly lives in its own project. This is a clear boundary and if I need to know where and how something is implemented, what the relevant interfaces are, and how it can be replaced I can concentrate on that single project. That makes understanding Orchard easier than dealing with a huge pile of code that is only structured using some folders.
Loading the solution is probably faster if it consists of less modules, but the same effect can be achived by excluding projects that are not used. This way flexibility is maintained and nobody is forced to test, maintain and update stuff that isn't used (now we are supposed to do the same thing with folders...). And if one have 150+ projects in an Orchard solution it might be a better approach to consolidate the 80+ custom projects before forcing everyone to go this way.
As I understand there are no reliable numbers which modules are used frequently and which ones are barely used at all. Orchard (thankfully) does not collect such data, so all we have anecdotal evidence. I don't like the idea of making such changes with gut instinct.
I understand the good intention, but I really see the risk of a meltdown here.
@BenedekFarkas commented:
May be a stupid question, but merging projects would mean that the resulting projects will have an Areas folder (by MVC convention)? This would mean that namespaces wouldn't change (as much as I can tell now). Btw Sipke said yesterday that he managed to set up a source with merged projects, so he will produce test results soon.
erik_oppedijk commented:
What about ASP.NET v.Next?
no assembly's needed anymore, whats the impact on that (memory/load times)?
@csurieux commented:
Orchard will use it in 2019
MrGenius commented:
I have the same opinion of @JasperD. I really like the current modular structure of Orchard. Its easier to include and remove modules from the solution based on the need and it gives better readability.
I use Orchard as a base framework for one of my projects. We do not have all the modules/projects in our Orchard solution. We have stripped down Orchard by keeping what is only relevant to our project. This also makes our life easier when we migrate Orchard code from one version to another.I have migrated Orchard many times module by module based on the issue fixes/needs than migrating the entire code base in one stretch from one version of Orchard to another.
Also I find it easier with current modular structure to customize some of the projects by suppressing them or replacing them with custom implementation. I know its possible, even after consolidating them all together. But I do not prefer replacing or suppressing core modules.
But, please think about the readability/flexibility comes from small self contained modules. No one wants to sit and crack one big project.
Yeorwned commented:
I also agree with JasperD and MrGenius, and would prefer for this change to not happen. A lot of my deployments don't always include all of the functionality listed in the proposed changes and those modules are left out altogether. The Orchard team has done such a great job of making things modular. Is it really worth taking a step back just to lower the project count?
@sfmskywalker closed and commented:
After much testing, we found that no performance gains were being observed (and in some cases even decreased performance).
@sfmskywalker created: https://orchard.codeplex.com/workitem/20800
The goal of this ticket is to compose a list of Orchard core modules that should be moved into Orchard.Core and then execute it.
Orchard.Core
Alias ArchiveLater Autoroute ContentPermissions ContentTypes Fields (and consolidate InputField with TextField) Forms ImportExport JobsQueue Modules Pages PublishLater Recipes Roles SecureSocketsLayer Setup TaskLease Templates Themes Tokens Users Warmup
Workflows
This would cut down the number of projects in the solution by 23. Similar to packaging up core modules into Orchard.Core, we could think of packaging up other modules, for example:
Azure
Azure
Azure.MediaServices
Caching
Caching OutputCache
SysCache
Markup
Markdown
TinyMce
Media
ImageEditor Media MediaLibrary MediaPicker
MediaProcessing
Scripting
Scripting Scripting.CSharp
Scripting.Dlr
Search
Indexing Lucene
Search
Utility
CodeGeneration DesignerTools Migrations Packaging
Upgrade
This would cut down the number with another 16 projects.