Drafting a new Resource module layout & build

[gaelcolas]

Pull Request (PR) description

This is an early preview on a new template I'm working on, introducing some different concepts than what's been used historically.

It is still work in progress but submitting this draft for starting the discussion and getting early feedback. The reasons for changing are multiple, and I'll explain in-line in a review, but the TLDR is:

• Be closer to PowerShell community (best?) practices in creating modules (lower friction between PowerShell module and DSC module
• Enabling a model more compatible with secure corporate environment (private galleries, package repository, no internet access), so that it's easier for corporate to develop internally, and share publicly
• Automating some tasks to lower the learning curve, and reproduce regardless of the CI tool (facilitating the move to other CI tool, such as AzureDevops)
• Enabling easier Integration testing with Test-Kitchen (yet to come)

/cc @PlagueHO @johlju @mhendric

---

This change is 

[johlju]

To use this we have to refactor the test framework and change the release process. Is you suggestion that all existing modules in the DSC Resource Kit should change to this model?

[PlagueHO]

Hey guys, sorry I haven't had a chance to engage on this, but I think we also need to consider auto-documentation resources in this. IMHO we should he encouraging the use of this model because it reduces the amount of work required to maintain and reduces risk of documentation not being the source of truth. This would go well with PlatyPs. I also haven't seen any mention of helper modules specific to the resource. The folder doesnt appear in the template.

[PlagueHO]

Also, shouldn't Requiredmodules folder be in the .gitignore?

[PlagueHO]

One final thought on this we need to carefully consider: we are introducing significant amounts of technical debt to every repository when ever we change the template/structure. Maintainers have a choice:

1. Move to the new template (we generally dont have enough time as it is)
2. Remain on the old template and and diverge.

IMHO we are going to end up having large divergence in approach to these which is actually going to make things more difficult for maintaineemrs and contributors in the long run.

I'm not saying don't do this or that it is not a good idea, but we haven't even got our major modules (including xPSDesiredStateConfiguration, xActiveDirectory,xWebAbmimistration etc) all using an existing template standard.

I believe like any change it should be made incrementally. We also would need to provide a massive amount of effort into the other repos to try and get them up to speed.

But again, I support and agree wity improvements in this area, I just know without the right level of investment and support from all maintainers we will just have another divergent standard that will require additional code in DSCResource.Tests to maintain.

Sorry to be mr. Negative today- it's just that I've been through several of the major structural changes to the templates and it usually takes me a solid year to catch up on the tech debt it generates. Many other maintainers dont bother.

[johlju]

I agree with @PlagueHO on the technical debt part, that was my thought with if this should be used in all repos. I’m not seeing that happening in the short term, since there are no contributors (well, in 2,5 years I have seen 1, no counting maintainers) interested in moving a resource module to a different model. That’s leaves the maintainers that are already busy reviewing PRs and fixing already existing technical debt on their free time. Personally I’m not seeing me moving those resources I maintain over to this any time soon. I have still not yet moved to the auto-documentation, but that is one priority to simplify documentation for contributors and myself.

In the short term, I see the need for a way to integration test on multiple nodes (having and AD to test clustering, and setting up a AD on multiple nodes, SQL Server multiple modes, etc.). For example test-kitchen, or Azure DevOps (automatically setting up a resource group while testing). As a maintainer I want a way for integration test the xActiveDirectory resources when a contributor sends in a PR. I don’t mind running a test suite in azure DevOps or Tast-Kitchen if it is automatic.

A user that have the need to use private repositories etc. probably have the knowledge of manually download the prerequisite from there approved local resources. I’m only seeing the need for documentation what is needed where.

You did a very nice job about this, but not sure this is gonna be used in the DSC resource Kit unless someone can put in some serious time implementing this in the repos.

I don’t think this template will help get contributors to the DSC resource Kit. It has always for me been more of a style guideline of how the project structure should look like. If this template changes to much at once, it cannot be used like that anymore.

[johlju]

Btw I agreed with all that @PlagueHO said in his comment, not just the technical debt part :smiley:

[johlju]

I agree with @PlagueHO on breaking down the changes so we can add them individually, so we can see the benefits and the assumptions, like “module X is released in gallery”, or “function/module Y must have implemented Z”.

I didn’t mean to sound negative in my previous comments, I’m all for change to the better, and the work here is intended to be for the better. I’m just afraid changing everything at once will never work, as the changes will never make it outside the template. Slow pace (=smaller pieces) is better, and also easier for the community as a whole to grasp it (for me too) and be part of the discussion.