search

dotnet / project-system

The .NET Project System for Visual Studio
MIT License
971 stars 389 forks source link

Add support for ASP.NET (non-Core) projects #2670

Open NickCraver opened 7 years ago

NickCraver commented 7 years ago

We routinely run into cases where the new project system and ASP.NET MVC 5 collide (or rather, don't) - I'd like to see support added. Here are the main use cases I'm aware of, I hope others add more to this issue:

Use Case: Migration

One of the awesome features of the new .csproj system is glob patterns (and Visual Studio not replacing them on every save). This was something we really looked forward to as a real time saver and merge reducer, but in practice we can't use the new project system because a lot of functionality for ASP.NET MVC 5 apps is missing, e.g. even hitting F5 to run it.

When we migrate a large solution to ASP.NET Core, it's a stop-the-world endeavor as are most major changes. Swapping any fundamental framework piece will need to be a big swap, so you want to reduce that window and risk as much as possible. In the case of ASP.NET to ASP.NET Core, you need to swap the project file, all references, port controllers, views, etc. This is a lot to do at once, and every change since you started that path is a royal pain.

A large part of this pain is in the project file itself, since nearly every commit adding or removing files is a merge conflict. And when moving from non-glob to blog worlds, it's a huge pain to manually sync everything into that major change branch.

But why not just stay on non-globs until after the move?

Again, merge pain. A large change like this necessitates moving files around quite a bit and that's the same merge pain between users as exists in 1 world today.

Use Case: Library Projects

We also have several projects that need to support ASP.NET and ASP.NET Core, there are hundreds of thousands of users on the old systems and we need to support them. In the ASP.NET cases specically, we need to have a ASP.NET and an ASP.NET Core sample. In all of these, the only use of the old project system is the ASP.NET (non-core) sample projects. We have to choose between:

Adding support for ASP.NET MVC 5 projects for example would help tremendously in porting to ASP.NET Core. In our case, it'd help now for library projects (needed dependencies for main websites to move), and later for those sites like Stack Overflow. We've been working on the .NET Core story for quite a while now, but the daunting task of upgrading the main applications would be made much easier if we could get on the new project system first.

For today examples, I'm using the new .csproj in MiniProfiler and Exceptional MVC 5 Sample projects. The upside is management and simplicity, the downside is users constantly asking why F5 doesn't work. At the moment I'm considering changing them to the old system since the main point of a sample is a bit missed when they can't test it.

Use Case: Immediate

Additionally, it'd just make life easier today in itself, apart from upgrade reasons, by:

Can we please consider adding support for these project types? Such support would make life easier both immediately and for upgrading in the future.

NickCraver commented 7 years ago

cc @davkean @davidfowl - we've discussed this a bit before as I beat my head on the wall making these things kind-of-work in testing the new system.

mduu commented 7 years ago

One more: The project files are the files most often result in merge-conflicts because multiple dev added or even renamed files in the projects. Project files are the bottleneck here as most work will result changes in the csproj. Even if the developers work in totally different areas of an assembly chances are high they will get merge conflicts.

xt0rted commented 7 years ago

I'd love to move my projects over to the new system but I'm forced to stay with MVC5 until all dependencies work on .net core as they do today.

Since EF code first migrations don't work with the new system the two projects that would benefit from this the most can't be updated (the website & the DAL). Because of this there's nothing to be gained if I migrate the other smaller projects.

If web projects were supported then one of my two sites could be fully updated, and the other could allow for everything but the DAL project making it worth my time.

mbp commented 7 years ago

For the immediate use case, the new project file format provides some great features such as transitive references. And for some reason, updating packages in a large solution with the old project file format takes ages. Just being able to hand-edit the csproj to update version numbers is one of the best features in it.

JesperTreetop commented 7 years ago

As someone with a lot of MVC5 projects, I have considered migrating them to the new csproj, since there's a lot to like. Since NuGet package scripts don't work, this breaks or at least makes working with Entity Framework migrations an unsupported scenario. (See xt0rted's comment.) They are present in all or almost all of the projects I had in mind, so that's a roadblock. Additionally, some NuGet packages that insert themselves into the build process would probably need to be similarly adapted, not to mention any project-type specific tooling (Microsoft or otherwise).

The features in the new project system would make my life easier - If this could magically be made to work, or if Microsoft would decide to make it a supported scenario and update EF(6) to work (or allow the community to help out), I would consider moving. As an unsupported it-seems-to-work-but-you-break-it-you-bought-it scenario, I wouldn't.

scottsauber commented 7 years ago

At DevIntersection in May it was said by a MS high up in a session that the new project system was coming to older projects by the end of the year this year, FWIW.

shaggygi commented 7 years ago

@scottsauber I seriously don't think this will happen based on this Repo Roadmap for milestone 16.

dirething commented 7 years ago

I have a number of solutions that contain projects that cannot migrate anytime soon as they rely on things like signalR in its current form, MSMQ and EF code first. It would be nice to be able to start new work where possible in the new system until I have the manpower available to rewrite the main components that rely on things the team has decided not to support in a directly upgradeable manner or does not yet support.

The library scenario would at least allow some work to begin moving in the right direction here.

NickCraver commented 7 years ago

Every time we reload the Stack Overflow project (git pull), Visual Studio 2017 crashes. Every single time. Everyone here experiences this, and every time we fix a bug, another pops up later causing the same. While not related directly to this issue, most of the time the project file didn't need to change. Simple support of globs would have prevented the vast majority of reloads in the first place.

Can we please get an idea of if this will happen? It's the most upvoted issue in this repo and causes tangible daily pain. We want to move to ASP.NET Core, etc. but that's going to take probably a year+. I'm sure there are many, many people in the same boat. Please, add support for the current projects most ASP.NET developers are dealing with and will be for some time.

davidfowl commented 7 years ago

Honestly I don't see this happening in the short term (this year). There are lots of kinks to work out with SDK projects and the new project system before we can bring them to older projects. Those kinks don't even have anything to do with ASP.NET so then there would be a bunch of work that needs to happen to port ASP.NET things to the new project system which is non trivial e.g. old razor editor, aspx editor, webforms designer??, build system (WAP projects have a hybrid msbuild + build manager build system), and any other quirks that System.Web requires. Some of those components are written in native code since the older project system was native.

All of this to say, it's not a simple flip the switch, it's the same set of people working on new and existing project systems. When the SDK projects are near flawless, I'd look at bringing it to down level projects.

NickCraver commented 7 years ago

@davidfowl I realize to support everything takes more work, but there's a lot of audience covered with far less than all of that. For example out of that entire list only the old razor editor applies to everything we have. But, I'm not sure what's even required there, as the razor editor appears to work as-is. Not listed is F5 debugging, which today results in:

Unable to run your project. The "RunCommand" property is not defined.

I am using the new project system for sample applications in our library projects because I have to. You can't mix old and new (see migration issues in the initial post here). It mostly works, but you can't F5 it because it can't launch. The editors work. The build works. Everything we need for daily almost works, if only F5 would start and attach. How much work are we talking about to make that work?

We don't have webforms, we don't have .aspx, we don't have build issues with WAP, and apps run fine in all our IIS setups (that's how we have to run them on the new project system). That's passable for sample apps (but really, what choice do we have?) but not for a team working together. I guess I disagree on what the minimum bar is here for a huge daily life improvement.

If F5 worked for spin-up debugging (sometimes attaching isn't practical), we'd have moved over already. What would it take to make this work?

davidfowl commented 7 years ago

It might be acceptable for you but I'm like 85% sure we need to have fully parity before claiming that something is a replacement. Those missing features are probably fine for your development but what about other people? Do we replace the default file new or this is just a hacky edit your csproj and hope it works solution for stackoverflow developers?

If we can't replace the file new project templates for existing ASP.NET 4.x applications then it's not ready yet. You won't believe the sorts of things people rely on Day to day in their projects. Forget those basic features I mentioned, there's also other extensions and features that interact with our project system that would need to work.

That said we could be making incremental progress but it's not a priority. It needs to work well for .net core projects and it really doesn't as yet. There's lots of performance problems that we're actively working on. To expand the scope of the project system before we worked out some of those major kinks would be a disaster

It would be awesome if our web project system was OSS so people could contribute and help make this a reality.

NickCraver commented 7 years ago

@davidfowl We're already in a very hacky state to even think about migrations, anything would be an improvement here. All MS effort appears to be going into a new system we can't realistically migrate to because the old system has been neglected. We have to use VS 2017 for current versions of C#, but to use it with old project systems we're crashing multiple times a day. We can't use VS 2015 unless we want to stick with project.json. We are dedicating our time to libraries on project system in hopes others can use them but we can't even do it ourselves.

We're stuck in a terrible position here. We can't move to the new stuff that works. Efforts to support the new stuff are quite obviously breaking the old stuff and we're suffering for it with zero benefits. I can honestly say: as things stand, we'd have been better off staying on Visual Studio 2015. Yes, I mean that. But it's too late to go back without significant work, we now use a C# version it doesn't support pervasively.

There is no incremental migration story. The solution is all or nothing at the moment due to the new project system not supporting all the projects people already have. dotnet msbuild is the same story, since it can build new things but not old projects.

And let's not forget the clock is ticking. It's already been said that ASP.NET Core 3.0 will drop .NET Framework support and move to .NET Core. This means we have a time limit on an incremental migration story that we can't even begin yet. Due to issues like this migrations to ASP.NET Core remain a logistical nightmare. My teams are asking when we can upgrade and I don't have an answer for them because of these blockers.

davidfowl commented 7 years ago

All MS effort appears to be going into a new system we can't realistically migrate to because the old system has been neglected

That's very far from the reality of the situation. I'm going to guess you're specifically talking about SDK projects and nothing more. PackageReference was actually ported to the old style csproj without requiring the use of SDK projects.

We have to use VS 2017 for current versions of C#, but to use it with old project systems we're crashing multiple times a day.

That sucks. I can't speak for what changed here that's causing the existing project system to crash multiple times a day. I assume you've given dumps to the appropriate people and they're looking at it?

We can't use VS 2015 unless we want to stick with project.json

This is confusing? Are you using both .NET Core projects and .NET Framework projects in the same solution? Is that the issue you're facing right now?

We're stuck in a terrible position here. We can't move to the new stuff that works. Efforts to support the new stuff are quite obviously breaking the old stuff and we're suffering for it with zero benefits.

Do you mean that you can't move new projects to ASP.NET Core or .NET Standard because it requires the new project system? Can you be more specific here?

There is no incremental migration story. The solution is all or nothing at the moment due to the new project system not supporting all the projects people already have. dotnet msbuild is the same story, since it can build new things but not old projects.

Yes this is true and I don't see this changing for a while. Changing all of the old things to the new project system will take years. Certain project types will be easier than others but it's not easy changing them all with full parity (see roslyn 😄 for a similar story, that took ~5 years).

And let's not forget the clock is ticking. It's already been said that ASP.NET Core 3.0 will drop .NET Framework support and move to .NET Core.

That's absolutely false, that was never said. The thread that got out of control made tons of assumptions that maybe that was one of them. Nobody from Microsoft said that ASP.NET Core 3.0 is absolutely dropping .NET Framework support.

Some questions for my education:

NickCraver commented 7 years ago

I'm going to guess you're specifically talking about SDK projects and nothing more. PackageReference was actually ported to the old style csproj without requiring the use of SDK projects.

Correct, talking about this area. <PackageReference> is a great concept, but broken in the older world. It took us a full day to get a build setup and working on and decided against attempting it with any further ASP.NET non-Core projects. We can use it in libs, that's it.

I assume you've given dumps to the appropriate people and they're looking at it?

For all but the latest yes. We've hitting yet another crash in the new preview. We're hopeful when hitting the update button that things will improve from one preview to the next but honestly it's 50/50. From a time standpoint though, we can only invest so much. We need to do our work, not fight with Visual Studio all day.

Do you mean that you can't move new projects to ASP.NET Core or .NET Standard because it requires the new project system? Can you be more specific here?

We can't move because we can't even start to move the solutions. We can't move things to libraries on the new system and migrate over in any reasonable way, because the build system won't build both. And there's a big mismatch on server build tools and what Visual Studio does here (see the fights we had with it above). Side note: those server build tools aren't labelled with a version.

Yes this is true and I don't see this changing for a while. Changing all of the old things to the new project system will take years. Certain project types will be easier than others but it's not easy changing them all with full parity

Sure, but this isn't asking for full parity on day one. By removing the biggest blockers to starting migration on the way to parity, we can parallelize a lot here and end up with ASP.NET Core far sooner.

That's absolutely false, that was never said.

I'm fairly certain this was said in a community standup around the time all the changes were happening as the current thinking. If that's changed then great, but we've got no newer information AFAIK.

What are the big things you are struggling with at the moment?

I'm assuming you mean in trying to make this work. Building (on a server, not in VS) and debug launching are the big 2.

Why is supporting is ASP.NET 4.x with an SDK project a stepping stone to ASP.NET Core support?

Because you can't mix both styles in the same solution and have the tooling work (at least today). The project files are massive today (no globs), so it's a big bang change to move everything over. It also means we can't move to anything with the dotnet build system. dotnet build and dotnet msbuild both go boom with these project types.

Why can't you use both ASP.NET Core and ASP.NET in the same solution?

See above.

Do you have .NET Standard libraries that are shared between old and new?

We want to, but again, see above. The only things we can share are completely external to the solution (e.g. MiniProfiler, StackExchange.Redis, Exceptional, Jil, protobuf, etc.). Moving code outside of the solution that belongs to it in order to make the builds work would make dev time take even longer (e.g. unnecessary packages, publishing, and build time for every minor change).

Can you list out some of the pains you have migrating (any pains really)?

The above covers it mostly. We can't use dotnet msbuild to build solutions if they contain old project types, so we can't go there. We can't use globs because VS will replace them. And crashing on every project reload is just maddening, since it takes 60-120 seconds per restart of Visual Studio for it to becomes responsive again. I'd estimate at this point in time I'm losing about 30 to 40 minutes a day on Visual studio locking up (for over a minute at a time) or restarting due to a crash. And losing project file changes when it does that.

I'm happy to work with someone and send our solution over (NDA style) if it'll help.

davkean commented 7 years ago

@NickCraver I've reached out to you privately on the crash you are experiencing. The long delays are likely caused by a mix of slow design-time builds + the fact that we block the UI while waiting for them in the legacy project system. This experience should be much better in 15.5 as we've done some performance here to reduce the number of builds we do during solution load/reload/configuration change. If you have custom targets, I'd use the steps called out in the doc to make sure that they aren't increasing your design-time build time. Performance work we're doing for .NET Standard/.NET Core scenarios will also benefit these builds for legacy scenarios.

Our intention, long term, is to have feature parity with the legacy project system, however, our biggest focus for the next couple of updates is to improve the performance and reliability of the existing scenarios we've already bitten off.

Can you expand on the the "all or nothing" argument? You can continue to call msbuild solution.sln on solutions that have a mix of both legacy format and sdk-based format. Moving over to dotnet build isn't a requirement. Is it because you don't want to update the build server?

davidfowl commented 7 years ago

I'm fairly certain this was said in a community standup around the time all the changes were happening as the current thinking. If that's changed then great, but we've got no newer information AFAIK.

Nope, it was never said and nothing changed.

Correct, talking about this area. is a great concept, but broken in the older world. It took us a full day to get a build setup and working on and decided against attempting it with any further ASP.NET non-Core projects. We can use it in libs, that's it.

It would be good to get more feedback here. Was the entire experience just error prone or were there specific issues?

We can't move because we can't even start to move the solutions. We can't move things to libraries on the new system and migrate over in any reasonable way, because the build system won't build both. And there's a big mismatch on server build tools and what Visual Studio does here (see the fights we had with it above). Side note: those server build tools aren't labelled with a version.

Like @davkean asked, why can't you even begin to move? MSBuild 15 supports both old and new project types AFAIK, you can't use dotnet build but msbuild should work. If the build tools don't have an msbuild 15 equivalent then we suck and should have that.

Sure, but this isn't asking for full parity on day one. By removing the biggest blockers to starting migration on the way to parity, we can parallelize a lot here and end up with ASP.NET Core far sooner.

The issue is asking for "support" we can't add a fraction of the features and claim support. Support means full parity. Like I asked before, knowing what specifics are absolutely blocking you from beginning the migration would be extremely useful.

Because you can't mix both styles in the same solution and have the tooling work (at least today).

This is the real crux of the issue. We need to figure out why this isn't working. Can you list out the issues with this? This absolutely needs to be fixed because this is key to people migrating existing solutions.

The project files are massive today (no globs), so it's a big bang change to move everything over. It also means we can't move to anything with the dotnet build system. dotnet build and dotnet msbuild both go boom with these project types.

This isn't new and shouldn't be a blocker. MSBuild should work like it always did with both existing and new style SDK projects.

We can't use dotnet msbuild to build solutions if they contain old project types, so we can't go there.

Can you just use the latest msbuild on windows?

We can't use globs because VS will replace them.

This isn't new and I don't see how it's a blocker. Over time more projects will likely be converted when full parity can be reached, but it's not a short term thing.

jmarolf commented 7 years ago

Correct, talking about this area. is a great concept, but broken in the older world. It took us a full day to get a build setup and working on and decided against attempting it with any further ASP.NET non-Core projects. We can use it in libs, that's it.

Adding @tmeschter. This sounds like a bug in PackageReference that we should fix. @NickCraver can you describe what the issue is in more detail?

NickCraver commented 7 years ago

@davkean @jmarolf

I've reached out to you privately on the crash you are experiencing.

Thanks! I've followed up and have many dump ready to send from hangs, stalls, and just-before crashes today.

Can you expand on the the "all or nothing" argument? You can continue to call msbuild solution.sln on solutions that have a mix of both legacy format and sdk-based format. Moving over to dotnet build isn't a requirement. Is it because you don't want to update the build server?

We have no problem updating the build servers (we control them all), but updating to the latest MS Build tools released we never got to work. We were deploying manual target directories and overrides to get web applications working. To make a single project work, we ended up copying all targets from local installs over to a directory on each build server and overriding the targets path in MSBuild. The new build tools installed fresh did not work for web applications (missing their targets).

On the mixing: we could never figure out how to build it. See https://github.com/opserver/Opserver/commit/7ccf72f84a0c19ae418ea14e98c69d11eb5613d9 for similar issues on finding MSBuild itself. dotnet build is a great advancement to all these things (e.g. tremendously simplified builds), but is completely unusable for most current scenarios.

The major blocking problem we had (I'll have to re-attempt this with another solution to fully document as we go, trying from recall here) was various breaks on the new project system with old references. All the tooling either created or expected <PackageReference>, or we ended up with 2 very different references on the machine since these 2 cannot point to the same DLLs. Our private projects cannot use <PackageReference> if they contain references to our authenticated NuGet feeds (the long standing reason we have to check in /packages). Almost all projects have such references. I admittedly last tried mixing <PackageReference> and old style path references in a solution 3 previews ago, but I haven't heard of any changes here, or even whether this is an expected scenario. NuGet tooling doesn't seem to expect this scenario I'm guessing.

In short, there are several areas where bridging old and new worlds is painful or just something we couldn't make work. The authenticated feeds issues is hard for me to demonstrate in open source, unfortunately.

NickCraver commented 7 years ago

@davidfowl Thanks for the replies, follow-ups:

Nope, it was never said and nothing changed.

Fair enough, if so that's my mistake, I swore this was said off-hand but maybe it wasn't by the team. My fault.

If the build tools don't have an msbuild 15 equivalent then we suck and should have that.

I'm honestly not sure what the contain. 15 to 15.3 had breaking changes you needed a newer version for, and installing the latest didn't work. I'll try a single new project in the SO solution soon as I'm able (likely next week, we're busy as hell atm) and better document step by step what breaks. See above response for some of the issues I can recall right now.

The issue is asking for "support" we can't add a fraction of the features and claim support. Support means full parity. Like I asked before, knowing what specifics are absolutely blocking you from beginning the migration would be extremely useful.

Again see above. Note: we're not asking for support (the official term), only bits that would unblock many. We built Stack Overflow on an MVC beta, test SQL in CTP in production, etc. no strangers to pre-release bits on the way to supported.

This is the real crux of the issue. We need to figure out why this isn't working. Can you list out the issues with this? This absolutely needs to be fixed because this is key to people migrating existing solutions.

Yep, above. I'll do a full doc run on an SO branch with the current preview and build server tooling to see what we hit today (AFAIK nothing has changed, we were on the latest build server tooling on the last attempt), and write up what we're hitting. I don't want to mis-file a dozen issues though, how about I file a big one and we break it out into areas as you guys and gals see fit? If so: which repo?

Can you just use the latest msbuild on windows?

I think so, but these aren't versioned like they need to be IMO. I'm both a developer and a sysadmin, and the sysadmins here are frustrated at the tooling downloads. For example go here and scroll to the bottom: https://www.visualstudio.com/downloads/ It's an unversioned installer (that needs an update immediately after downloading by the way, not an awesome experience). Since it's unversioned, we have to make some assumptions about which version is installed or blindly update it (current approach) which led to some mismatches of 15.0 vs 15.3 msbuild installs. Just for fun, it's not in the registry anymore either, so we have to install a whole other project just to find it (vswhere), and I guess check that into git or PATH it on every server (is this sounding crazy yet to do a simple build? it is to me).

In the old world we could at least find where MSBuild is in the registry (in a PowerShell build script) and move along. Now even that's regressed. This isn't an issue if you're using dotnet build, but is if you need actual full msbuild. I consider this a regression, you now need extra tooling just to find MSBuild.

But yes, we're free to use the latest version of MSBuild. Actually figuring out how to do so, which version is installed or where it is are all different questions we don't always know the answer to.

This isn't new and I don't see how it's a blocker. Over time more projects will likely be converted when full parity can be reached, but it's not a short term thing.

That's unfortunate. When we heard about the new project system it was the thing we were all excited about, because it results in the most pain. From an outside perspective (and maybe mine's way off) I don't believe that new project system being only for new projects/types was communicated well at all. It was a major let down to realize this was the case and now will be for quite some time.

Still, I'll help debug here where I can but admittedly have practical time limits on fighting tooling vs. getting any work done. Most of this is done on my personal time at the expense of our OSS getting any love.

tmeschter commented 7 years ago

@NickCraver

  1. I'd like to hear more about the issues you've had with <PackageReference> on the old project system as I implemented that. :-)
  2. I'm also interested in any crashes that occur in VS 2017 when using the old project system. In general I would expect 2017 to be more reliable in this area, and regressions are a high priority.

Feel free to ping me directly at tom.meschter@microsoft.com.

jmarolf commented 7 years ago

@NickCraver regarding the other issues:

  1. MSBuild tools does not include the targets to build web projects
  2. Unable to locate MSBuild on build server.

The first issue sounds like a bug, but I'm confused. It looks like you can install the web targets: image @NickCraver does this not work? I understand automated installation of this installer leaves some things to be desired but the following command should work

vs_BuildTools.exe  --layout c:\vs2017buildtools --includeOptional --lang en-US

Giving you a folder (C:\vs2017buildtools) with vs_BuildTools.exe that you can copy and install on any machine. while guaranteeing a known version Is this not what you are doing?

The second issue is a general problem with everything in vs being harder to find because you can now install vs anywhere. However, you can control where things get installed on your build server. If you always install the build tools to C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools msbuild will always be under C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\15.0\Bin

NickCraver commented 6 years ago

@jmarolf Here's what we ended up with to get building and not have gigs of extras to maintain and update (lengthening patch times). Tooling offline creation:

.\vs_BuildTools.exe  --layout \\path --includeOptional --lang en-US
  --add  Microsoft.VisualStudio.Workload.NetCoreBuildTools 
  --add  Microsoft.VisualStudio.Workload.WebBuildTools
  --add Microsoft.VisualStudio.Workload.MSBuildTools
  --add Microsoft.Net.Component.4.6.TargetingPack

Server install/update:

.\vs_BuildTools.exe --includeOptional 
  --add  Microsoft.VisualStudio.Workload.NetCoreBuildTools
  --add  Microsoft.VisualStudio.Workload.WebBuildTools
  --add Microsoft.VisualStudio.Workload.MSBuildTools
  --add Microsoft.Net.Component.4.6.TargetingPack
  --noWeb 
  --passive

The second issue is a general problem with everything in vs being harder to find because you can now install vs anywhere. However, you can control where things get installed on your build server.

Sure, I agree. But in open source we don't control the build server. Thousands of people control thousands of build servers.

This week we've had several major issues with Stack Overflow (including 2 outages) rooted in bad .csproj merges that could have been globs for the past decade if the tooling stopped expanding this. Is there any hope of not expanding globs at least being supported on the old system? We're far from the only ones hitting this. Or is the stance this is so far into the future it'll never happen and I should just close the issue?

Pilchie commented 6 years ago

We don't currently have work to support not expanding globs in the old project system on our radar. Instead, we'd like to eventually be able to open old style projects with the new project system without also needing them to be converted to SDK projects.

NickCraver commented 6 years ago

@Pilchie As an illustration of why this matters: Our main project (StackOverflow.csproj) using only globs goes from 13,307 lines down to 1,537 lines. Of those 1,537:

With <PackageReference> (which we can't use due to NuGet server issues yet - unrelated to this issue), that 363 goes down to 122 lines.

So any, with globs something we experience merge pain with multiple times a week goes to nothing. I looked through our git history and I believe we would have had a merge conflict only once in the past 6 years (one simultaneous lib update race) in our main .csproj. That's a huge impact to our daily lives and to the stability of our code, application, and sanity.

Please, reconsider some measures to improve life for all of the developers who are dealing with ASP.NET non-Core and will be for the foreseeable future. There are millions of us and eventually is a long time.

This is by far the highest voted issue in this repo, so I'm curious: how is work and feedback prioritized? The User Voice forums for Visual Studio are, and I don't think I'm being extreme here, a wasteland. They're not maintained, there's a lot of duplication, and User Voice has no mechanisms for keeping up with issues. While problems are getting love in the new developer community site, missing things (like this) are closed as non-problems and the customer avenues for making suggestions are pretty grim. It's not specific to this issue, but as this is by far our biggest desire to improve every dev here's daily life, I'm asking.

Note that Visual Studio is crashing on .csproj changes/reloads on a git pull more often than ever. I'm seeing many of my teammates experience this on hangouts several times a day. So this issue (or even just globs) would have a very significant impact on developer frustration (by not prompting for reloads). We're seeing new project types get support and love while those of us in the old world are just crashing. It's very demotivating to hear "eventually" on improvements while this trend is continuing.

Please, please prioritize this.

mduu commented 6 years ago

If I have to vote for one single thing, I like to get rid of all the individual file-references in our MVC5 project files (and class-libraries). They result in merge-conflicts in every other PR.

Note: we can't move to .Net Core for the next years as our hoster/customer (federal gov.) does not have any plans to support .Net Core yet. That will take years then.

davkean commented 6 years ago

@NickCraver The crash when the csproj reloads we should look at. You've mentioned this before but I'm yet to see feedback reports from your team about it - can you get them to individually report issues (you don't need to catch the crash, just report that it occurs) and we'll go through the watson reports to see what the underlying cause is?

Understand the pain here here with merging, but as @Pilchie mentioned we have no intention of supporting globs in the old project system. The work there is much harder than the new project system due to it being native (and we have double-wrap all the new globbing-based APIs from MSBuild) and unlike the new project system it has its own in-memory model of the project separate to the MSBuild model. We would rather spend that time and resources on adding features to the new project system that will enable you to move your existing projects over to it.

NickCraver commented 6 years ago

I've followed up with @davkean - the crashes we were all loading are gone in 15.5, which we're very, very thankful for. It doesn't solve the PR pain though. If we're not supporting globs because that's too much work, what about other solutions which are easier and still greatly lessen or eliminate the daily pain?

Here's my suggestion from 4.5 years ago on user voice to sort the .csproj includes alphabetically to at least avoid the merge conflicts: https://visualstudio.uservoice.com/forums/121579-visual-studio-ide/suggestions/4130464-organize-project-xml-files-to-reduce-3-way-merge-c

Pilchie commented 6 years ago

Unfortunately, that's problematic too :( The files are passed to the compiler in the order they appear in the file, and that does have some semantic effects around partial types in multiple file (though they are fairly esoteric in C# and VB). The compiler interprets partial types by basically stitching them together in order. And then:

  1. Field initializers are run in the order they appear, which means that changing the order can cause initialization deadlocks or incorrect results.
  2. (Much less of a concern) If multiple parts contain xml doc comments, they get merged in the order they appear.

With that said, @KirillOsenkov did write a tool to sort project files if these things aren't concerns for you. One could imagine writing a build task that either sorts before compiling (not my preference, as I don't like build tools that change the source tree), or detects and errors if they files aren't sorted...

KirillOsenkov commented 6 years ago

Yes, check out: https://github.com/KirillOsenkov/CodeCleanupTools/tree/master/SortProjectItems

From experience, if you just run the tool manually once every one or two months, it's sufficient to maintain the projects in a clean, mergeable state.

If you do have a bad merge conflict of .csproj files, sort both sides first, then compare.

axelheer commented 6 years ago

I might point out that "Using F5" is indeed possible (as described here)

willnogueira commented 6 years ago

It would be great to have this added. The ASP.NET sdk csproj (net461) works after compiling and deploying, but Visual Studio can't parse cshtml files during development as it expects ASP.NET core views.

ChristopherHaws commented 6 years ago

+1 for adding glob support at the very least. It would also be nice to be able to edit without reloading like the sdk based projects as well as using ProjectReference without needing to specify the project guid.

LTsLlama commented 6 years ago

Good idea @ChristopherHaws, I have not checked if this is not already happening, so I'm trusting you here. We are not using this yet, because this issue is a blocker. I think this would actually be an entirely new idea, but please add link-back so we can vote for it. Also, *blob Cheers, Douw

theo-albers commented 6 years ago

In our business we create software for different customers, thus we have a nice collection of code repositories with .NET technologies. On newer projects we have successfully used .NET Core and the SDK-project file format. I have successfully used the new project file in older ASP.NET MVC 5 solutions, specifically for new Library projects. On a day to day basis we switch between customer projects to support production systems and for developing new applications / websites / mobile apps. Luckily we can stay in VS2017, thanks to the backwards compatibility approach of VS2017!

Now mix and match of project systems in a single solution is certainly possible, but not desirable since it results in a weird developer experience. For some projects, you can quickly edit the project file, for others you have to unload, before you can edit. Try to explain to young developers why that is. But mix and match is the only way forward for ISV's with existing code: you can't destabilize the entire solution, so you pick what you want to address / change in a sprint. Wouldn't it be nice if there was some migration path for legacy project file? And yes David Fowler I hear you: you have to maintain two technologies.

In my days as MFC-dev we had to integrate WinForms / WPF in the C++ code base. It was not nice, but for the customer it worked. Isn't it possible to have some low hanging fruit in supporting SDK-features in the old csproj system? I mean technically you can call out to your new .NET libraries, right? Or is the concern build performance stopping you going that route?

Instead of saying "it's not on our roadmap", could you state if there is any low hanging fruit? Would it be possible to bring in legacy stuff via a project import in de SDK-project or the other way around? We would be happy to sacrifice a little bit of solution load performance for solutions we have marked as "hybrid".

Igorbek commented 6 years ago

An alternative could be a community-driven SDK. Custom project SDKs now seem to be supported: https://github.com/MicrosoftDocs/visualstudio-docs/blob/master/docs/msbuild/how-to-use-project-sdk.md Somebody can come up with a prototype and invite interested people here.

peter-dolkens commented 6 years ago

+1 in support of @NickCraver - We're having merge pains almost daily, often taking the greater part of a day to resolve, and our largest projects are only 10% of the size of yours!

Adding ProjectReference support to old the old project format does nothing to help, as that just adds packages.config (fairly easy to merge currently) to the pain of csproj merges.

davkean commented 6 years ago

By ProjectReference, I'm assuming you mean PackageReference?

We have added support for PackageReference to the old project system, it's easier to merge than packages.config because you no longer need to specific the entire graph, just the top level packages.

seangwright commented 6 years ago

We have moved all of our projects (many of which are .NET 4.6.1 class libs and Web Forms apps) to either the Sdk style csproj (for class libs) or at least PackageReference (for Web Forms apps).

The Nuget package resolution/restore story is sooo much better. We multi-home projects in different solutions, so no longer having to set a repositoryPath in our Nuget.config is also great.

There are still sharp corners - but those were there with the old systems so it's become a matter of learning where the new ones are.

YakhontovYaroslav commented 6 years ago

We have managed to build old ASP.NET projects using new SDK csproj with this csproj snippet:

<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>net472</TargetFramework>
    <LangVersion>latest</LangVersion>
    <OutputType>Library</OutputType>
    <AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
    <GenerateBindingRedirectsOutputType>true</GenerateBindingRedirectsOutputType>
  </PropertyGroup>

  <PropertyGroup>
    <DocumentationFile>bin\$(AssemblyName).xml</DocumentationFile>
    <OutDir>bin\</OutDir>
    <WebProjectOutputDir>./</WebProjectOutputDir>
  </PropertyGroup>

  <ItemGroup>
    <Reference Include="Microsoft.CSharp" />
    <Reference Include="System.ComponentModel.DataAnnotations" />
    <Reference Include="System.Data.DataSetExtensions" />
    <Reference Include="System.Net.Http" />
    <Reference Include="System.Web.Extensions" />
    <Reference Include="System.Web" />
    <Reference Include="System.Configuration" />
    <Reference Include="System.Web.Services" />
    <Reference Include="System.EnterpriseServices" />
    <Reference Include="System.Web.DynamicData" />
    <Reference Include="System.Web.Entity" />
    <Reference Include="System.Web.ApplicationServices" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="Antlr" Version="3.5.0.2" />
    <PackageReference Include="AspNet.ScriptManager.bootstrap" Version="3.3.7" />
    <PackageReference Include="AspNet.ScriptManager.jQuery" Version="3.3.1" />
    <PackageReference Include="Microsoft.AspNet.FriendlyUrls" Version="1.0.2" />
    <PackageReference Include="Microsoft.AspNet.Mvc" Version="5.2.4" />
    <PackageReference Include="Microsoft.AspNet.ScriptManager.MSAjax" Version="5.0.0" />
    <PackageReference Include="Microsoft.AspNet.ScriptManager.WebForms" Version="5.0.0" />
    <PackageReference Include="Microsoft.AspNet.Web.Optimization.WebForms" Version="1.1.3" />
    <PackageReference Include="Microsoft.AspNet.WebApi" Version="5.2.4" />
    <PackageReference Include="Microsoft.AspNet.WebApi.Core" Version="5.2.4" />
    <PackageReference Include="Microsoft.CodeDom.Providers.DotNetCompilerPlatform" Version="2.0.0" />
    <PackageReference Include="Modernizr" Version="2.8.3" />
    <PackageReference Include="Newtonsoft.Json" Version="11.0.1" />
    <PackageReference Include="WebGrease" Version="1.6.0" />
    <PackageReference Include="OctoPack" Version="3.6.3" />
    ...
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..." />
    ...
  </ItemGroup>

  <ItemGroup>
    <None Remove="**" />
    <Content Remove="**" />

    <None Include="node_modules" Visible="false" />
    <Content Include="**" Exclude="obj/**;bin/**;node_modules/**;**/*.cs;*.csproj*" />

    <Compile Update="**\*.aspx.cs;**\*.asax.cs;**\*.ascx.cs;**\*.Master.cs" DependentUpon="%(Filename)" SubType="ASPXCodeBehind" />
    <Compile Update="**\*.asmx.cs;" DependentUpon="%(Filename)" SubType="Component" />
    <Compile Update="**\*.ashx.cs;" DependentUpon="%(Filename)" />
  </ItemGroup>

  <PropertyGroup>
    <VisualStudioVersion Condition="'$(VisualStudioVersion)' == ''">15.0</VisualStudioVersion>
    <VSToolsPath Condition="'$(VSToolsPath)' == ''">$(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion)</VSToolsPath>
  </PropertyGroup>
  <Import Project="$(VSToolsPath)\WebApplications\Microsoft.WebApplication.targets" />

  <Target Name="UpdateWebConfigBindingRedirects" AfterTargets="CopyFilesToOutputDirectory">
    <Copy SourceFiles="$(OutDir)$(AssemblyName).dll.config" DestinationFiles="Web.config" />
  </Target>
</Project>

It handles building (with binding redirect generation) as well as packaging project for deploy. One downside of this approach is not being able to run project from VS by F5, You must configure IIS site and attach VS debugger to IIS in order to debug it, but this was no problem for us, since we already used this approach at dev time. Publish command will not work as expected too, but there is OctoPack witch can generate deploy bundle.

CI build commands (do not merge them into one):

msbuild /t:Restore
msbuild /t:Build /p:RunOctoPack=true

then generated nupkg (or zip file) can be deployed to IIS (remember to do web.config transforms by deploy tool).

Hope this helps people struggling with legacy projects. cc @NickCraver

Kralizek commented 6 years ago

OMG this is great! I'll test it ASAP.

Do you have any problems with razor squibbling?

YakhontovYaroslav commented 6 years ago

@Kralizek other than VS no longer offering legacy item types in Add Item menu no other issues so far - our project originates from earlier 200x so editing experience was already far from pleasant with shitpiles of JS in the middle of the Pages etc. Razor (and WebForms) syntax highlighting as well as autocomplete seems to work well where it was worked before)

Kralizek commented 6 years ago

@YakhontovYaroslav this part

  <Target Name="UpdateWebConfigBindingRedirects" AfterTargets="CopyFilesToOutputDirectory">
    <Copy SourceFiles="$(OutDir)$(AssemblyName).dll.config" DestinationFiles="Web.config" />
  </Target>

is giving me troubles because it overrides the whole file like settings and some webserver stuff as well. Any idea how to improve it?

YakhontovYaroslav commented 6 years ago

@Kralizek I don`t have any problems with overwriting Web.config since $(AssemblyName).dll.config in /bin will contain exact copy (with custom appsettings etc) of web.config, but with binding redirects. If you have any trouble you can write your own merge task instead of plain copy. Example:

  <Target Name="UpdateWebConfigBindingRedirects" AfterTargets="CopyFilesToOutputDirectory">
    <MergeBindingRedirects BindingRedirectsSource="$(OutDir)$(AssemblyName).dll.config" WebConfig="Web.config" />
  </Target>
  <UsingTask TaskName="MergeBindingRedirects" TaskFactory="CodeTaskFactory" AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.Core.dll">
    <ParameterGroup>
      <BindingRedirectsSource ParameterType="System.String" Required="true" />
      <WebConfig ParameterType="System.String" Required="true" />
    </ParameterGroup>
    <Task>
      <Reference Include="System.Core" />
      <Reference Include="System.Xml" />
      <Using Namespace="System" />
      <Using Namespace="System.Xml" />
      <Code Type="Fragment" Language="cs">
        <![CDATA[
          ...
        ]]>
      </Code>
    </Task>
  </UsingTask>

Just write some C# code to merge only binding redirects node and you are done. You can even move it into some NuGet package and just reference it, instead of including this same code into every csproj.

Btw, we found that dotnet new templates are good alternative to now not working Add new item menu.

And one more thing to notice: template above don`t include some special item types (for example .edmx files), but you can update\include them same way as i have included other files.

MattJeanes commented 6 years ago

I've spend a good amount of time today trying to get something working but to no avail. I looked around the source code for Microsoft.NET.Sdk.Web to see how their sdk is built and tried to build my own and adapt small parts to get it working as per @Igorbek's comment above

Ultimately, adding this line into the csproj will turn it into an AspNetCore site and will attempt to boot it using the IISIntegration stuff and automatically adds the <aspNetCore> bit into the web.config:

<ItemGroup>
  <ProjectCapability Include="DotNetCoreWeb" />
</ItemGroup>

You also need <RunCommand>/<RunArguments> elements as it will fail to start using the IISExpress command option in the launchSettings.json which is seemingly for running things like dotnet run to actually boot up the backend site but that doesn't really apply here as the IIS site is all we need - doesn't appear to be a way to just say 'use that for debugging' and don't start anything else or modify the web.config.

Trying to force remove the <aspNetCore> bit from the web config using something like <remove name="aspNetCore" /> seems like it would work, but VS throws a fit as it can't communicate with it and just gives a DEBUG request is not valid. error.

I also tried an extremely complicated way of using a "commandName": "Executable" in the launchSettings.json, the results of which you can find here and here. It kind of worked but then I realised it's complete rubbish as changing the iisSettings didn't update the applicationhost.config file unless you built the project with it set to "commandName": "IISExpress" and debugging didn't work at all so I gave up.

@NickCraver's method used in StackExchange.Exceptional does work with debugging so may be good enough for some people, probably can be adapted to use a custom applicationhost.config too for more control. This seems to be the best way to get it working at the moment, though disadvantages are that it doesn't launch the browser, bindings e.g. ports/ssl can't be configured through VS interface anymore and it launches a separate console window.

My conclusion is that it is not currently possible to run ASP.NET projects smoothly with the new csproj format as you could with the old csproj format without changes being made to Visual Studio's project system to support it, but I'd really love to be proven wrong here.

Don't want to sound like a broken record in this thread but I'm in total agreement with @NickCraver and everyone else here - we're not getting rid of our aspx site anytime soon, and while the PackageReference support is really great (honestly, thank you!) we'd really really love to use the new csproj format as well.

GrahamTheCoder commented 6 years ago

ASP NET non-core works so long as RunComand property is defined and a few other things like this:

<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <OutputType>Library</OutputType>
    <OutputPath>bin\</OutputPath>
    <AppendTargetFrameworkToOutputPath>false</AppendTargetFrameworkToOutputPath>
    <RunCommand>$(MSBuildExtensionsPath64)\..\IIS Express\iisexpress</RunCommand>
    <RunArguments>/path:&quot;$(MSBuildProjectDirectory)&quot; /port:18082</RunArguments>
  </PropertyGroup>
  ...rest of file...
</Project>

Tweaks:

I'm not sure why this isn't the Visual Studio default template for ASP NET - maybe I should be PRing it?

willl commented 5 years ago

@YakhontovYaroslav You can use msbuild /restore in place of msbuild /t:restore and msbuild /t:build if I understand what you're trying to achieve.

You'll need to have msbuild 15.5 installed for it to work. See: https://github.com/Microsoft/msbuild/issues/2455#issuecomment-362425073

304NotModified commented 5 years ago

@davidfowl any updates on this? This issue is by far the most up-voted one: https://github.com/dotnet/project-system/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc

davkean commented 5 years ago

Thanks for the feedback and votes. Our number 1 priority for the new project system in the VS 2019 timeframe is support client scenarios. In particular, Windows Forms and WPF designers and related features, for both .NET Framework and .NET Core. As we make progress on those commitments and start thinking about what we want to bite off for the future, we will factor in the above feedback.

CZEMacLeod commented 5 years ago

@davkean In your list of priorities for VS2019 feature parity is the first thing listed, yet ASP.NET (non-Core) (e.g. this issue) isn't even tagged as a parity legacy issue. While I understand that many desktop developers are 'in-the-cold' until .netcore 3 and WF/WPF/XAML come to the table, it would be nice to address the (minor) needs of those working on maintaining, and developing with ASP.NET. Given that there are already a good number of 'workarounds' in place even in this thread, it seems to me that the minimal amount of VS IDE support required for launch settings, and razor intellisense support most of which you already must have code for, just not lit up under the new project system, seems disappointing.