Source Index and pack .pdb files into regular .nupkg

[pranavkm]

Wherever we ship DLLs that are used in customer apps, we need to include the portable PDB along side it.

We need to include portable PDBs in:

• [ ] The shared framework
• [ ] The NUPKGs

And we need to do this in:

• [ ] 2.1.0-preview1
• [ ] dev

---

Original text:

From https://github.com/aspnet/KRuntime/issues/380

The plan would be to create a single nupkg rather than a .nupkg and .symbols.nupkg as part of our build and use SourceLink to support having pdbs sit in the package.

[davidfowl]

So I think the steps would be to take the output of the build and generate a source linked pdb using the pdb from the .symbols package. When we unzip and rezip the package to tweak extra stuff, we should include the pdb source linked pdb in there pointing to the github sha.

[AndriySvyryd]

Note that including the pdbs in the .nupkg where VS can find them turns the corresponding dlls into “My Code”.

By default VS has Just My Code and break on user-unhandled exceptions enabled. This will cause the debugger to break on exceptions that cross "My Code" boundaries.

We found this out because we were including the symbols in the System.Data.SqlClient.nupkg, see aspnet/EntityFramework#987

[ctaggart]

I just published a working version of SourceLink.exe. It should do what you need. Details: https://github.com/ctaggart/SourceLink/issues/53#issuecomment-64313467

[ctaggart]

Hoping to provide better examples of using SourceLink.exe, I submitted an issue that shows how to index Sake.pdb and Sake.Engine.pdb https://github.com/sakeproject/sake/issues/17.

Is there an API that I can access to get a list of files to compile in a project.json? If yes, I may be able to incorporate in SourceLink.exe. If not, use these two args instead:

        --pdb <string>: pdb file to add the index to, supports multiple and globs
        --file [-f] <string>: source file to put in the index, supports multiple and globs

For example --pdb the\path\to\the.pdb -f *\*.cs.

[ctaggart]

I just announced a stable SourceLink.exe and added documentation on how to use it. http://blog.ctaggart.com/2015/03/announcing-sourcelinkexe.html http://ctaggart.github.io/SourceLink/exe.html

Also be sure to check out the demo that demonstrates taking advantage of the source index at design time. http://ctaggart.github.io/SourceLink/index.html#Demo-Go-To-Source-during-Design-Time

I would love to make progress on this ticket.

/cc @pranavkm @davidfowl

[ctaggart]

At the Microsoft MVP Summit, I spoke with several key individuals today and we've come up with a great plan for source indexing. The primary push right now is to get debugging to work cross-platform. The portable pdb format recently became 1.0 and Roslyn csc.exe can create them with a debug switch shipping with Visual Studio 2015 Update 1. The pdb's are much much more compact and cross platform (6 to 10 times). It would be great to included them in the nupkg files.

Open issues:

• I am working with @tmat to get the source index text file into the portable pdb file via Roslyn csc.exe
• the dnx build tool would use that Roslyn support, cc @anurse
• @yishaigalatzer and the NuGet team is going to look at using satellite nupkg files like they did for native packages for CoreCLR
• the Roslyn team will have a design discussion about identifying generated source files in a couple of weeks. (may be *.g.cs like Xamarin). The generated ones could be embedded in the pdb.

Simply including the portable pdb files with the nuget packages is a great first step.

[pranavkm]

cc @rynowak for symbro.

[ctaggart]

We can change the title of this to source link instead of source index. I rewrote SourceLink as a dotnet tool for version 2. It supports Portable PDB files and its new source link support.

I've added pull requests that should get you guys started for both aspnet.mvc https://github.com/aspnet/Mvc/pull/5860 and EntityFramework https://github.com/aspnet/EntityFramework/pull/7726.

It was been agreed upon to include Portable PDB files in the nupkgs for source link support. Check out the fresh new documentation here: https://github.com/ctaggart/SourceLink

cc @davidfowl

[ctaggart]

Just to get @DamianEdwards @Eilon @shanselman & others up to speed:

• Visual Studio 2017 is shipping with "Enable source link support"
• I worked weekends and evenings this last month to rewrite SourceLink for version 2 as a dotnet command line tool to support this.
• We've agreed to ship the Portable PDB files in the nupkg files for source link support.
• I've provided examples for adding it to both asp.net mvc & EntityFramework core.

It should be easy to add. It only needs to run on the build server. Happy to help whoever does the work. I'm also wanting to onboard the SourceLink project to the .NET Foundation & have submitted to do so.

[tmat]

Let me also point out that VS 2017 supports Embedded Portable PDBs. This feature (/debug:embedded) instructs the compiler to embed a compressed Portable PDB into the dll/exe itself and the debugger finds it there. I'd recommend using that for release builds for all libraries where the extra size overhead doesn't matter. Source link feature is also available for Embedded PDBs.

[ctaggart]

/debug:embedded sounds like a great option that would make it even easier for projects to adopt source linking. I'll add support in SourceLink soon.

[Eilon]

We don't currently ship PDBs anywhere at all - I think that's probably required for this suggestion to be useful, is that correct?

[ctaggart]

@Eilon That is correct. Embedding in the dll sounds like a good option. It is simple to test and to switch to.

The objections about shipping pdb files before were:

• Windows PDB files are too big and require more network
• Windows PDB files take up too much space on the client computer

With Portable PDB files and the new dotnet/nuget tooling, these issues have been addressed. The files are much smaller & the dotnet/nuget tooling shares the downloads in ~/.nuget. It should be simple to test this change to see how much bigger the nupkg files get. Simply switch to embedded PDB files and try it with <DebugType>embedded</DebugType>.

I think the increased file size is justified to enable source link support, but please discuss with the decision makers. @yishaigalatzer is who I've talked to before about this.

[analogrelay]

Do we know the approximate increase in file size? Even if it's fairly significant, I'd probably still advocate for embedding PDBs because having to manage PDBs is JUST THE WORST. :)

I suppose I could just try it, but then I'd have to do work and _{mumble mumble I'm lazy mumble}

[analogrelay]

A simple test with Microsoft.AspNetCore.Authentication showed an increase from 62 KB to 73 KB for embedding the PDB. An increase of 17% in size. This seems completely reasonable to me given the benefits :).

[dougbu]

Is @AndriySvyryd's comment above still correct?

including the pdbs in the .nupkg where VS can find them turns the corresponding dlls into “My Code”

If yes, how do we help users focus on what they're developing when that's their preference?

[tmat]

@gregg-miskelly For opinion on JMC.

I wonder if it would make sense to distinguish Debug (My Code) vs Release (not My Code) bits.

[gregg-miskelly]

As long as dll/exe is optimized compiled, and 'Tools->Options->Debugging->Suppress JIT Optimizations on Module Loads' is in its default (unchecked) state -- no, since the code is JIT optimized it would still be considered non-user code.

[dougbu]

aspnet/EntityFramework#987 happened at the end of October 2014. I know we had a short (let's hope) period where we accidentally shipped non-optimized bits. Anyone remember if the time frames overlap?

If the System.Data.SqlClient bits were optimized, we need to figure out why VS treated them as My Code. Or @gregg-miskelly, are you describing a different behaviour than we would have seen way back then?

[gregg-miskelly]

@dougbu versions of VS before 2015 had Suppress JIT optimizations on by default.

[dougbu]

Excellent 🥇

[Eilon]

@anurse does your experiment just embed regular PDBs in the DLL, or does it include source links and/or embedded source?

[ctaggart]

@anurse @tmat SourceLink 2.0.0-b396 was just uploaded with support for embedded debug type. Please try it out. https://github.com/ctaggart/SourceLink/pull/152#issuecomment-283417583

[analogrelay]

I haven't had a chance to try embedded source/source links at all yet.

[Eilon]

@anurse ah ok, so your experiment was just with embedding a "regular" PDB into the DLL? That's still cool, though, a lot smaller than I would have guessed.

[tmat]

@eilon Had to be Portable PDB. The compiler doesn't support emebedding Windows PDBs.

[Eilon]

@tmat yup, understood. To me a portable PDB is now a "regular" PDB 😄

[clairernovotny]

@tmat, @anurse, @Eilon, @ctaggart and others: Embedded PDB's are a really, really, really bad idea....sorry. It's bad for mobile devices where the physical image size and memory is tight. We don't want pdb's to be loaded into memory because they're embedded in the dll. This is a complete non-starter.

It's not just mobile, but other constrained devices like IoT and embedded (think Tizen)

[ctaggart]

@onovotny What do you think of my previous dotnet pack --include-portable-pdbs proposal? The current --include-symbols & --include-source are options in support of "enable source server support". It would be great to have --include-portable-pdbs in support of "enable source link support".

[clairernovotny]

I'm okay with anything that makes it easy to include source indexed symbols in a place we all agree is convenient and easy to use. They just have to be a different file than the main dll to minimize the memory taken by loading the dll from storage to ram :)

The only other thing is that whatever it is has to also work with msbuild /t:pack because we cannot use dotnet pack to build most project types (things other that .NET Standard, .NET Core or .NET Desktop). If we build other project types using multi-targeting (or even single targeted but using the SDK-style projects), it has to use desktop msbuild as that's what the tasks in those targets are built for.

[roji]

@onovotny, another possibility is simply to allow stripping the embedded debugging information (plus sources) for scenarios where binary size is very important (e.g. mobile). This is similar to the Unix model, where you use strip to reduce the size of binaries by removing the debugging info. This could be a simple step in the build for mobile, etc.

There is something very appealing in the assembly containing everything needed to fully debug it - with external PDBs and sources it's frequently a real pain to make everything work together. If embedding the PDB and sources becomes the default (which I'd say it probably should), it means that over time all assemblies will become easily debuggable, without requiring individual developers to create and publish symbol source packages, etc.

[roji]

BTW and unrelated - if the debugging information is stored as an embedded resource in the assembly, does that actually have an effect on memory footprint? I mean, do embedded resources in general get loaded implicitly as opposed to user code explicitly requesting them?

(regardless of memory footprint, there's still the image size on disk, of course)

[tmat]

@roji It's not a resources, it's debug table data. I believe the memory is not committed until it's accessed.

[roji]

@tmat from https://github.com/dotnet/roslyn/issues/12390 it seemed that the embedded resource implementation option was chosen, although I may be misunderstanding things. Regardless, it's not a fundamental issue.

[clairernovotny]

@roji, if stripping out the non-essential bits, like embedded PDB's, can be part of the toolchain, and present by default on the Xamarin/Mono toolchain, then my objections go away.

My goal is that the developer of a solution that's within space constrained limits isn't encumbered by libraries that are too "fat" because library authors thought they being helpful by providing symbols.

The problem is that as an embedded resource, altering it would affect the validity of the strong name and break authenticode signatures on the output libraries. That means the deployed libraries are more easily tamperable, which is also bad.

[ctaggart]

The raised concerns were:

1) memory footprint It sounds like memory is is not committed until the debug table data is accessed.

2) image size on disk when targeting scenarios like mobile Stripping out the debug tables to save 10-20% on disk space sounds like a good option for the toolchains like Xamarin/Mono.

Embedding the debug data with source link support enabled will work out of the box with Visual Studio 2017. It is a good solution that aspnet can turn on today and .NET developers lives would improve as a result. I expect the code bases to improve as well when it is easier to troubleshoot exceptions and there are more eyes on the code.

cc @migueldeicaza

[roji]

@onovotny I understand and that makes sense.

It seems reasonable to say that all assemblies contain all debugging and source information inside by default - this simplifies the debugging experience in the majority of cases - but to definitely have a strip option in place for Xamarin.

The strong name issue is indeed a problem. It may be good to confirm that the Xamarin toolchains actually check/enforce the signature - I'm simply saying that because I have no idea whether the strong name is actually checked on iOS/Android etc.

[clairernovotny]

Xamarin does not enforce strong name signing at all. Authenticode is a separate issue that while isn't validated by default is still a concern. If an assembly is authenticode signed, as all Microsoft ones and many third party are, then it's wrong to break that chain.

[tmat]

@onovotny Just realized, if you want the smallest footprint possible you'd probably want to use a linker that tree-shakes all the unused code and merges everything into one assembly. I assume the linker also produces a separate PDB.

[davidfowl]

Right, why wouldn't the linker take care of this?

[clairernovotny]

The linker's on iOS/Android only link Xamarin iOS/Android libraries by default: https://developer.xamarin.com/guides/android/advanced_topics/linking/

I suspect it's because linking all assemblies, Sdk and User assemblies, would break too much by default.

.NET Native is similar -- by default, it doesn't link user assemblies either. From the default template: https://github.com/onovotny/Zeroconf/blob/51dd7238a9782bb1b4ddb7162f6f169fa6b61343/ZeroconfTest.UWP/Properties/Default.rd.xml#L24

.NET Native may do other things where the embedded PDB could get stripped out though, and possibly similar for the Xamarin linkers.

I have heard issues with ASP.NET Core applications too -- objections from people like @bartdesmet around library sizes in dense microservice applications where they don't want any extra bytes loaded in memory. There's no linker there for ASP.NET Core (yet).

Don't get me wrong here, I'm in favor of anything that makes it easier to debug with source. Just not at the expense of perf :)

[davidfowl]

I personally think the benefit outweighs the things you mention

[ctaggart]

I released SourceLink 2.0.0 today.

http://blog.ctaggart.com/2017/03/enable-source-link-support-announcing.html https://twitter.com/cmr0n/status/838516874258296833

The MVC pull request has been updated to use it. The builds & tests pass. https://github.com/aspnet/Mvc/pull/5905

[tmat]

@onovotny I don't think one approach fits all scenarios. Both approaches have trade-offs. It's up to the producer of each package to consider them and make the decision. It's up to consumers to choose packages that meet their requirements on features and size for their scenarios.

[roji]

@tmat it doesn't seem right to say that the producer of each package should make the decision if their package is meant to be used in a reduced-footprint context or not - as a package producer I have no idea who uses my package or for what, and shouldn't be concerned with that.

It seems more reasonable to find ways in which capabilities which benefit one scenario (e.g. embedded PDBs and sources) don't adversely affect another scenario (mobile).

[davidfowl]

It seems more reasonable to find ways in which capabilities which benefit one scenario (e.g. embedded PDBs and sources) don't adversely affect another scenario (mobile).

It's more the fact that mobile scenarios already have a very complex toolchain that already do things to reduce the impact on the device. It doesn't seem like a stretch to support stripping out more information.

/cc @migueldeicaza @mhutch

[tmat]

@roji

as a package producer I have no idea who uses my package or for what

If your users don't ask you to make the library smaller then it's probably just fine.

[roji]

I think we're all sort of saying the same thing - ideally the Xamarin (and .NET Native) toolchains would do all the necessary stripping, giving us the best of both worlds - embedded debugging information without an impact on final image size where that counts.

[ctaggart]

I opened up a Xamarin User Voice suggestion: option to strip embedded debug information. I think it is the appropriate place to discuss the mobile optimizations further.

[ctaggart]

I think the finish line is close. #553 https://github.com/aspnet/BuildTools/pull/423 merged which enables source linking. The pdb files just need to be added to the nuget packages.