RFC: Thoughts on moving to yarn

mtrutledge commented 5 years ago

Currently after the generator creates the templates and files it runs npm install. Should we change this over to yarn?

This would give us the ability to restructure some of the module directories and use yarn workspaces so we would not have to duplicate the node_modules if the sub-projects used the same modules.

Thoughts?

david-poindexter commented 5 years ago

"Yes" would be my vote, well at least for now. :)

SkyeHoefling commented 5 years ago

I would be interested in seeing if there is any buy-in to move to Cake or no 3rd party build scripts. I would prefer to see yo-dnn take the direction of build script agnostic so devs can choose what they want to use. If that really isn't an option I would recommend taking a look at a pure powershell script or cake which are more front-end agnostic than using gulp vs yarn.

The challenge I have ran into a few times is the MVC template comes with gulp which can be difficult to rip out if I want to use a different build system.

I'm not sure if this is a direction you want to take or not but I can elaborate more on how we could potentially solve some of the building problems with Cake as I am doing it for a few projects right now.

mtrutledge commented 5 years ago

Cake could be used for the build system for C# however, how would users build their angular, vuejs, react, webcomponents modules/apps? Does cake have any support for building those? The modules are so much more than just C# projects now. Maybe this changes with Blazor, but I have a feeling we will always support a HTML module type because of the flexibility. But if Cake can build the front end components then that could definitely be an option.

mtrutledge commented 5 years ago

I have been working on some things where webpack is just used with node (no gulp) and converting the C# projects to support the Roslyn compiler so to build the C# side of things I just call dotnet build using the dotnet command line utility (passing parameters to specify building for net461). In this scenario gulp is removed but npm or yarn is still required to build the front end.

mtrutledge commented 5 years ago

Looking a bit more into cake they do have some tasks to run yarn commands. @ahoefling would you envision it working like this:

Cake runs nuget restore for sln/csproj
Cake runs yarn/npm to build frontend modules if applicable
Cake runs dotnet or msbuild for sln/csproj files
Cake runs package module step

Or some other way?

SkyeHoefling commented 5 years ago

@mtrutledge thanks for all the responses we have a couple things going on here and different directions you can take. I'll start with the simple answer

Cake isn't going to solve every problem we have

Right Tool for the Right Job

MVC Modules

I think you should absolutely use Cake for a MVC Module and anything that is really pure C#. I know MVC modules have frontend code, but let the developer decide on what they want to use.

SPA - Angular, React, Vue.js, Etc.

These modules should use the default build system that are required IMO. I don't think Cake is necessarily the right decision here. We would run into the same problem that we are forcing developers to use the build system we think is best and not the one that is best for them.

We can use Cake to run gulp, yarn, npm, etc. for the SPA modules but I am not convinced it is the right answer.

Dotnet CLI Build Commands

You brought up a really good point of using the dotnet CLI to run your builds. All of my new projects now are built using the dotnet CLI. I start with a .NET Standard project template then I target net45 or the framework version I am interested. This allows me to use dotnet build to compile DNN Modules or other .NET Framework projects.

Build System Agnostic

The challenge here is there are many build actions that need to help to compile a project. We aren't just compiling C# anymore, there is javascript, css, etc. You can draw a line in the sand and say all yo-dnn projects are going to use yarn, gulp or cake.

Another option would be giving the user an option of what build system they want to use. So we would need to support a few different templates and they can then choose. I think this would be the right direction, but the problem is this requires a lot of work.

The Challenge

I want to outline the challenge with this discussion as I see it. Every developer has their own thoughts an opinions on the best way to build projects. I think limiting this library to focus just on one build technology unintentionally alienates other developers that would use this utility. Here is a list of ways I have seen people build Dnn Modules

MSBuild (look at the Dnn Core Modules or a Christoc Module Template)
Cake
Gulp
Yarn
npm
I'm sure I missed some

mtrutledge commented 5 years ago

I do have to be opinionated in some way to make this work at a reasonable level. Otherwise, like you said, it is a lot of work. I would like the build to be consistent across project types. Especially since a user can add multiple project types to their solution. For instance, I have a project that has webforms, mvc, and react modules in it (the webforms are slowly being replaced with reactjs modules). I used the generator-dnn project to create the mvc and react modules, then did some slight folder restructuring so I could use yarn workspaces and lerna so the node_modules were not duplicated across modules. My build process for this project is defined like this in TeamCity:

Update assembly versions
run nuget restore
run yarn install
run yarn build (this just builds my react modules)
run msbuild on the sln file
package all _Install.zip files into a nuget package
deploy nuget package to my Octopus Deploy Server
Trigger deployment from Octopus to dev Evoq instance.

I define these steps in TeamCity for the build and have no issues getting error states out from webpack or grunt (I use grunt in my project instead of gulp, but am working on replacing it with webpack plugins and standard nodejs scripts).

To me it looks like cake is a task runner, just like gulp or grunt is a task runner. Integrating into a CI/CD environment it appears instead of defining the steps and tasks for the build in the CI environment, you define them in the cake file and have the CI trigger the build via cake. To continue using my project as an example, I would create a single cake file at the root of the project, (basically where the solution lives), that would define steps 2-4 from TeamCity and then tell TeamCity to run cake, correct? With a single cake file at the root of the project, it could build the same locally as it does in a CI environment, just by calling cake. You could use TeamCity, Appveyor, Azure devops, etc. The CI tool wouldnt matter as long as you can trigger cake.

valadas commented 5 years ago

Msbuild scripts also support running yarn with msbuild.community.tasks, and node.js and yarn are available as nugetbpackages. I use that in my mixed modules to build the frontend and it works good. So with that there are 0 software to install other than Visual Studio, not even node.

mtrutledge commented 5 years ago

Interesting @valadas. Not installing node would work for a project, might be something to consider for the scaffolded projects, but I still need them to install node so they can install yeoman and the generator.

valadas commented 5 years ago

Hmm true, true. But after generated, still useful in CI environments though. Those only need to have Visual Studio and fire up a release build. And we are sure that local and CI builds work exactly the same.

valadas commented 5 years ago

Also useful fir someone who did not generate the project but works on it, like open source modules

bdukes commented 5 years ago

I think that the ideal scenario means primarily interacting with only tool, which could be cake or dotnet or yarn. But under the covers, any of those tools can be running gulp or webpack or cake. For example, our workflow inside Engage uses yarn to wrap calls to gulp, so that getting setup on a project is just calling yarn (to install local dependencies) and then yarn start to kick off Gulp.

I agree with @valadas' point that it should be easy to run without having to install extra tools; however, I personally think it's reasonable to expect that Node.js is going to be available on developer machines and CI agents (or, at least, not a major barrier to get it installed). Whether you start with the JS or .NET package manager is mostly a matter of taste, IMO (i.e. if dotnet calls yarn or yarn calls dotnet).

To the original question, Engage's yeoman template checks whether yarn is installed (using which), and Yeoman's installDependencies call takes an options object which can say whether to use npm or yarn, so we just make it conditional based on what's installed on the developer's machine, which makes sense to me to a public-use tool.

mtrutledge commented 5 years ago

In version 1.0.8 I am using which to detect if yarn exists and installing based off of that flag. I have updated the readme to reflect using yarn to install instead of npm. I will also make sure that I am pushing yarn in my presentation at summit. Using yarn will allow the use of workspaces in the future. Peter Donker has done some work with cake which might also be integrated via a choice in the future.

mtrutledge / generator-dnn