search

chucknorris / roundhouse

RoundhousE is a Database Migration Utility for .NET using sql files and versioning based on source control
http://projectroundhouse.org
920 stars 247 forks source link

Revise Build Process to no longer require NAnt or .NET 3.5 #284

Open BiggerNoise opened 7 years ago

BiggerNoise commented 7 years ago

Current build is built upon UppercuT and thus NAnt. NAnt demands .NET 3.5 and has become more out of date than is reasonable to continue using.

The purpose of this issue is gain consensus on where we want to go next. @ferventcoder has mentioned wanting to migrate to Cake Build.

Personally, I am skeptical of the need for build frameworks. I prefer a plain old make/msbuild file.

Let's throw some arguments around.

ferventcoder commented 7 years ago

UppercuT is what this should use, since both are part of the chucknorris framework - my proposal is that uc moves to cake build.

ferventcoder commented 7 years ago

NAnt can be ran under .NET 4.x, one only needs to upgrade uc

erikbra commented 7 years ago

I personally agree with @BiggerNoise . I don't see what value UppercuT adds to the build process, in addition to MSBuild. MSBuild is programming in XML, which is a nightmare. But, MSBuild is actively developed, and recently released as Open Source, and for all platforms, which I think is an advantage, should we move towards .NET Core and multi-platform support (which I think would be lovely). Even SQL Server is available on Linux now.

The other sweet thing about MSBuild, is it's the same build format that Visual Studio uses, and then you can have full parity between your IDE and command-line builds.

That said, I am very open to that I don't see all the facets of this, and UppercuT does look like a beast that can do just about everything. I'm just a bit skeptical to the NAnt foundation, as that is a bit old, and even Ant, which it is based upon, is widely replaced with other tools (e.g. maven) in the Java community too.

I think it would be fruitful to do a quick sketch of the build process of RoundhousE. Which steps are involved, and what are its deliverables. Then we can see which tools can help us performs these steps in the simplest fashion.

A requirement of a new build tool/build pipeline, IMHO, would be better integration with TeamCity, i.e. that the build logs are easily available in the TeamCity UI, and preferably with good step-by-step integration with TeamCity (can be achieved with Service Messages in just about any tool.

It would also be benefitial if the tools are widely installed, as it would make it easier for new developers to join the project. There is something elegantly simple with just cloning a repo and opening it in VS, being able to build and run tests at once.

Do you have any previous experience with Cake, @ferventcoder ? Are you using it in other projects, and have good experience with it?

BiggerNoise commented 7 years ago

Absolutely agree on moving in the direction of .NET core and multi-platform support.

Very practical question: Who is going to update UppercuT? I'm willing to put nights/weekends into roundhouse because it's an important part of my day job. UppercuT is not going to play a similar role, so I am not willing to put the same kind of effort in there.

I'll sketch out what I understand of the build process later this week.

ferventcoder commented 7 years ago

Cake is xplat and supports .NET Core, plus all standards. Plus like NAnt, it can run MSBuild, which leans into why it is a more powerful choice. It has one heck of a community behind it, I just started picking it up and I can say I am extremely impressed with it's flexibility, power, and maintainability. Plus it was adopted into the .NET foundation and it has very active development and innovation. To boot, you are simply writing C#, which makes it a joy to work with.

We didn't pick MSBuild last time around, I'm not keen on picking it this time around either.

UC hasn't been upgraded in awhile here. It is a templated build, you just pick the things you want and it just works. Which also makes upgrades flawless. Not something that can be said for a custom script. There are numerous benefits to why uc, the benefits far outweigh that it currently uses NAnt on the back end.

ferventcoder commented 7 years ago

@gep13 and I will be updating UppercuT to Cake as we use it for Chocolatey.

ferventcoder commented 7 years ago

Just an FYI, NAnt has always been multi-platform and UC supported that (both .NET platforms and Linux platforms).

BiggerNoise commented 7 years ago

I'm not that familiar with Cake, so I'll have to take your word for it on its upsides. Certainly being able to write build instructions in C# would be a boon.

I don't find UpperCuT particularly easy to use. I spent an hour trying to keep two files from being copied into the output so we could get past Chocolatey moderation. Eventually, I gave up and created a P/R with what I had.

Undoubtedly, that's my ignorance at work and this was a five minute job for someone familiar with the tool, but I'm not likely to use uc anywhere else and learning which commands I can override and how fine a scalpel this tool gives me is not a task that holds a lot of reward.

ferventcoder commented 7 years ago

I don't find UpperCuT particularly easy to use.

@BiggerNoise s/UppercuT/NAnt ;)

Not arguing with you at all. Definitely want to get it over to something else while still maintaining ease of use (and adding ease of maintenance), xplat, git clone and build, etc.

erikbra commented 7 years ago

@ferventcoder Could you maybe explain a bit, or point to some documentation which explains some of the benefits of using UppercuT on top of nAnt/Cake/Fake/PSAke/Msbuild, instead of using the other build systems "raw"? I must say that I hade quite some adventures trying to update e.g. Nuget in the RoundhousE world, but things might look a lot different if using Cake underneath. I wouldn't know.

What additional options/tools/processes/abstractions does UppercuT add? (I'm genuinely interested, not trying to be critical at all).

Could we make a small writeup of the steps necessary to build RoundhousE?

anything else?

erikbra commented 7 years ago

Another thought, how about the rest of the build tooling/process, should we upgrade that related to this as well? Things I can think of:

ferventcoder commented 7 years ago

Cake answers much of those questions by not keeping anything in the repo except for scripts - take a look at https://github.com/chocolatey/ChocolateyGUI (setup.cake) - uses Cake Recipes, which are extremely similar to UC.

BiggerNoise commented 7 years ago

I could get behind keeping the packages out of the repo and not including nuget in the repo. Probably be a bit of futzing to define the path to the user's nuget.

netstandard 2.0 seems like the right place to aim for, I'm slightly fuzzy on this. If we target standard 2.0 would a machine that has 4.5 installed be able to run it? Or would all of the machines that run rh have to have the very latest of .NET installed?

From a coding point of view, I want at least .NET 4.6 and C#7.

BiggerNoise commented 7 years ago

Took a look at that setup.cake; I think I had a brief spasm of XML withdrawal. :)

ferventcoder commented 7 years ago

@erikbra

ferventcoder commented 7 years ago

From my perspective, runtime would be best at .NET 4.x, which would allow all .NET 4+ areas to run - biggest reason is more compat with lots of environments.

BiggerNoise commented 7 years ago

@erikbra And all of this build stuff happens when you run build.sh/.bat

erikbra commented 7 years ago

Thanks, @ferventcoder.

I saw the versioning step. It seems to do very much the same as what GitVersion does. We might consider using that if we don't want to do everything ourselves. It's available via Nuget too (https://www.nuget.org/packages/GitVersion/3.6.5)

The other steps I think we can achieve in any build language.

erikbra commented 7 years ago

I think @ferventcoder is right. 4.x is best. But I think we should settle for at least .net 4.6.1+. Because .net 4.6.1 implements netstandard20. netstandard20 is a specification. It is implemented by netcore20 and net461 (think HTML - implemented by both Chrome and Edge). https://docs.microsoft.com/en-us/dotnet/standard/net-standard

So, keeping the support libraries (e.g. each individual db provider) on netstandard20 would make it possible to build RoundhousE.exe both for net461 (which would work in almost any windows environment) and core20 (which is xplat, but not so likely to be installed everywhere, esp. on db servers, if people run RoundhousE directly on the server).

I'm all open to using Cake, no worries. Love learning new stuff! But @ferventcoder , you are planning on changing the backend of UppercuT from nAnt to Cake, do I understand correctly? Does Chocolatey use UppercuT too?

ferventcoder commented 7 years ago

@erikbra I think you are really going to love Cake, GitVersion is already built into it.

erikbra commented 7 years ago

I hate MSBuild, by all means. I just know it way better than I wished I did ;) (learned the hard way).

ferventcoder commented 7 years ago

@erikbra yes, Chocolatey does use UC. A newer version of it with quite a bit more than what RH is currently using - First key in is the settings file - https://github.com/chocolatey/choco/blob/master/.uppercut (.uppercut) with only diversions from the default settings.

erikbra commented 7 years ago

So, say we should try to get this working? How should we work? Should we branch the main repo, and make pull requests off that one, or maybe divide the work into several smaller tasks, and just create PRs straight off master?

Separate tasks I can see off the top of my head:

ferventcoder commented 7 years ago

stores nuget packages in ~/.nuget (one local cache for all projects)

They actually did that then? Nothing can go wrong there :/

Change project format to "SDK-style"

That's definitely something I'm looking forward to with choco and others

Upgrade nuget dependencies to latest version and remove them from git repo

👍 as long as it is handled with good integration testing

Convert UppercuT backend to Cake

What type of schedule are we talking here? Days, weeks, months or more like early 2018?

erikbra commented 7 years ago

I think @BiggerNoise has set the pace here. Tens of pull requests per day ;)

No, but seriously, it would be nice if a RoundhousE release was more of a bi-weekly thing than bi-anually. It's been two in three months now, that's amazing!

I love working on this, but knowing myself, and schedule at work, I might not be able to contribute that many hours each week. But I definitely want to help here. Let's just make the development feedback loop great, so that PRs can be made smoothly.

Who set up the build agent on the teamcity server? Do we know what's installed there? I did have some issues that it only handled C#5, e.g. Can we dictate the tools (e.g. Visual Studio 2017) on the build agents, or is this set by Jetbrains?

Do we have some environments for running integration tests? Are the integration tests actually in good shape? Can they be run on a regular basis (on localhost, anyway)? What stuff needs to be installed locally to run all integration tests?

Having a vivid and good integration test suite would be advantageous before we start this quest. Then we can make changes with much higher confidence that we don't break anything.

ferventcoder commented 7 years ago

Let's use AppVeyor. It's much, much better in terms of evaluating PRs and builds.

ferventcoder commented 7 years ago

(AppVeyor vs TeamCity is not an either or thing either, hope I didn't confuse that).

erikbra commented 7 years ago

By all means. Do you have an account on the hosted one?

ferventcoder commented 7 years ago

on TC? yes.

erikbra commented 7 years ago

I meant AppVeyor, but you can always create an account. See it's free for OSS. Do we think we can build the master branch there now, or do we need to update the toolchain first?

ferventcoder commented 7 years ago

Yeah, we could build now. Here's what I propose. We update NAnt stuff for .NET 4.0 and then go from there with the rest of this. That solves the immediate issues.

erikbra commented 7 years ago

OK. Are you in control on how to upgrade NAnt?

ferventcoder commented 7 years ago

I can do that

erikbra commented 7 years ago

Great. I'll start looking at updating the .csproj files to SDK-style projects.

erikbra commented 7 years ago

I have made an attempt on converting the projects. Hope you don't find it too intrusive. Please don't take my ramblings about the build system too harsh. I just find the build definition files very difficult to work with, compared to simple scripts. I really hope this gets better if moving to a Cake backend for UppercuT :)

BiggerNoise commented 7 years ago

@erikbra Please continue with your frank feedback and steady contributions.

I'd rather have us make a misstep here and there than return to the mostly dormant state of the last few years.

ferventcoder commented 7 years ago

What @BiggerNoise said. 👍

erikbra commented 7 years ago

Should we leave this one open until we have a more "official", working build that is independent of .NET 3.5? Or create a separate issue on migrating to Cake? Build.ps1 is not dependent on .NET Framework 3.5 now.

BiggerNoise commented 7 years ago

I am fine with leaving this open until we have a more permanent build solution.