search

NuGet / Home

Repo for NuGet Client issues
Other
1.5k stars 253 forks source link

Temp/NuGetScratch approached 40GB #4040

Closed scottbommarito closed 7 years ago

scottbommarito commented 7 years ago

My Temp/NuGetScratch folder was using about 40GB of my drive until I deleted it when I ran out of disk space this morning. I am under the impression that the folder is used by the client as a temporary storage for packages while performing operations. The folder should not be using anywhere near as much space and should clean itself regularly.

emgarten commented 7 years ago

@scottbommarito what was in the folder? Do you have any repro steps that show it leaving things behind?

daveaglick commented 7 years ago

FWIW, I've also seen %localappdata%\Temp\NuGetScratch get really big. Mine is currently at about 3 GB.

dcomartin commented 7 years ago

image

emgarten commented 7 years ago

Any repro steps? My scratch folder is empty, but I know from the code that it would be easy to miss the clean up step. However that is scenario specific.

MaximRouiller commented 7 years ago

My %localappdata%\Temp\NuGetScratch is sitting at 4.45 GB

Last modification today. image

Each folder is between 130MB and 250MB.

MaximRouiller commented 7 years ago

I'm ready to help you debug. Not running NuGet from the command line. Only from VS2015.

Here's a dump of About Visual Studio:

Microsoft Visual Studio Enterprise 2015
Version 14.0.25431.01 Update 3
Microsoft .NET Framework
Version 4.6.01038

Installed Version: Enterprise

Architecture and Modeling Tools   00322-80000-00000-AA913
Microsoft Architecture and Modeling Tools

UML® and Unified Modeling Language™ are trademarks or registered trademarks of the Object Management Group, Inc. in the United States and other countries.

Microsoft Visual Studio Tools for Applications 2015   00322-80000-00000-AA913
Microsoft Visual Studio Tools for Applications 2015

Visual Basic 2015   00322-80000-00000-AA913
Microsoft Visual Basic 2015

Visual C# 2015   00322-80000-00000-AA913
Microsoft Visual C# 2015

Visual C++ 2015   00322-80000-00000-AA913
Microsoft Visual C++ 2015

ASP.NET and Web Tools 2015.1   14.1.21111.0
ASP.NET and Web Tools 2015.1

ASP.NET Web Frameworks and Tools 2012.2   4.1.41102.0
For additional information, visit http://go.microsoft.com/fwlink/?LinkID=309563

ASP.NET Web Frameworks and Tools 2013   5.2.40314.0
For additional information, visit http://www.asp.net/

Azure App Service Tools v2.9.6   14.0.21111.0
Azure App Service Tools v2.9.6

Azure Data Lake Node   1.0
This package contains the Data Lake integration nodes for Server Explorer.

Azure Data Lake Tools for Visual Studio   2.2.2100.0
Microsoft Azure Data Lake Tools for Visual Studio

Browser Reload on Save   1.0.18
A Visual Studio extension for ASP.NET projects that leverages Browser Link to to reload all connected browsers when files are saved in Visual Studio.

Bundler & Minifier   2.2.307
Adds support for bundling and minifying JavaScript, CSS and HTML files in any project.

Common Azure Tools   1.8
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.

Fabric.ApplicationInsights   1.0
Allows user to instrument their Service Fabric projects for Application Insights.

Fabric.DiagnosticEvents   1.0
Fabric Diagnostic Events

File Icons   2.1
Adds icons for files that are not recognized by Solution Explorer

File Nesting   2.5.65
Automatically nest files based on file name and enables developers to nest and unnest any file manually

Image Optimizer   3.6.103
Uses industry standard tools to optimize any JPEG, PNG and Gifs - including animated Gifs. Can do both lossy and lossless optimization.

Image Sprites   1.4.47
Boost your website's performance by creating image sprites to reduce the amount of HTTP requests needed.

JavaScript Language Service   2.0
JavaScript Language Service

JavaScript Project System   2.0
JavaScript Project System

JetBrains ReSharper Ultimate 2016.2    Build 106.0.20160818.163633
JetBrains ReSharper Ultimate package for Microsoft Visual Studio. For more information about ReSharper Ultimate, visit http://www.jetbrains.com/resharper. Copyright © 2016 JetBrains, Inc.

Markdown Editor   1.10.186
A full featured Markdown editor with live preview and syntax highlighting. Supports GitHub flavored Markdown.

Microsoft .NET Core Tools (Preview 2)   14.1.21111.0
Microsoft .NET Core Tools (Preview 2)

Microsoft Azure Hive Query Language Service   2.2.2100.0
Language service for Hive query

Microsoft Azure Mobile Services Tools   1.4
Microsoft Azure Mobile Services Tools

Microsoft Azure Tools   2.9
Microsoft Azure Tools for Microsoft Visual Studio 2015 - v2.9.41104.6

NuGet Package Manager   3.5.0
NuGet Package Manager in Visual Studio. For more information about NuGet, visit http://docs.nuget.org/.

Open Command Line   2.1.179
Opens a command line at the root of the project. Support for all consoles such as CMD, PowerShell, Bash etc. Provides syntax highlighting, Intellisense and execution of .cmd and .bat files.

Package Installer   2.0.94
Makes it easier, faster and more convenient than ever to install Bower, npm, Yarn, JSPM, TSD, Typings and NuGet packages to any project

PreEmptive Analytics Visualizer   1.2
Microsoft Visual Studio extension to visualize aggregated summaries from the PreEmptive Analytics product.

SQL Server Analysis Services   13.0.1700.281
Microsoft SQL Server Analysis Services Designer 
Version 13.0.1700.281

SQL Server Data Tools   14.0.60923.0
Microsoft SQL Server Data Tools

SQL Server Integration Services   
Microsoft SQL Server Integration Services Designer
Version 13.0.1601.5

SQL Server Reporting Services   13.0.1700.281
Microsoft SQL Server Reporting Services Designers 
Version 13.0.1700.281

Syntax Highlighting Pack   2.1.89
Adds syntax highlighting and snippet support for a wide variety of programming languages such as Clojure, Go, Jade, Lua, Swift, Ruby and many more...

ToolWindowHostedEditor   1.0
Hosting json editor into a tool window

TypeScript   1.8.36.0
TypeScript tools for Visual Studio

Web Compiler   1.11.326
Compiler for LESS, Sass and CoffeeScript files

Web Essentials 2015.3   3.0.235
Adds many useful features to Visual Studio for web developers. Requires Visual Studio 2015
MaximRouiller commented 7 years ago

68 packages in the latest folder. They correspond to the list of packages found in the Manage Packages for Solution for the current project I'm working on.

rrelyea commented 7 years ago

My folder is currently empty too. Yes, would love repro steps.

MaximRouiller commented 7 years ago

5 days later. I'm at 4.57/4.61 GB size.

@rrelyea Is there anything I can do to help you gather data on this?

emgarten commented 7 years ago

@MaximRouiller can you clear the folder and then perform simple nuget actions until you see things being left behind in the folder again? Also, are you able to tell the files are?

MaximRouiller commented 7 years ago

Steps taken:

Notes:

When opening the client's solution, solution loading was slow. As in, REALLY slow. I'm sure if I keep reloading the solutions, more folders are going to pop.

MaximRouiller commented 7 years ago

Awaiting further instructions. 😄

MaximRouiller commented 7 years ago

Ahh and manually closing the solution does nothing. Opening the solution directly from the quick menu in Windows 10? Same result. 3rd folder now.

Unloading all projects, reloading them, recompiled solution. Still 3 folders. Nothing new.

emgarten commented 7 years ago

@MaximRouiller what type of project is this? XProj? CSProj + project.json, packages.config?

MaximRouiller commented 7 years ago

csproj. Good ol' OWIN.

First few lines of the UI csproj:

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <Import Project="$(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion)\TypeScript\Microsoft.TypeScript.Default.props" Condition="Exists('$(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion)\TypeScript\Microsoft.TypeScript.Default.props')" />
  <Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props" Condition="Exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props')" />
  <PropertyGroup>
    <Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
    <Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
    <ProductVersion>
    </ProductVersion>
    <SchemaVersion>2.0</SchemaVersion>
    <ProjectGuid>{47D62A3E-6269-4030-BE50-075F4D37A5CE}</ProjectGuid>
    <ProjectTypeGuids>{349c5851-65df-11da-9384-00065b846f21};{fae04ec0-301f-11d3-bf4b-00c04f79efbc}</ProjectTypeGuids>
    <TargetFrameworkVersion>v4.6.1</TargetFrameworkVersion>
.....
emgarten commented 7 years ago

csproj with packages.config?

MaximRouiller commented 7 years ago

This specific project doesn't no. It uses Task Runner Explorer and has package.json for UI stuff. Other projects (like an API)? Yes. csproj + packages.config

Here's the API csproj first few lines:

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <Import Project="..\packages\Microsoft.CodeDom.Providers.DotNetCompilerPlatform.1.0.2\build\Microsoft.CodeDom.Providers.DotNetCompilerPlatform.props" Condition="Exists('..\packages\Microsoft.CodeDom.Providers.DotNetCompilerPlatform.1.0.2\build\Microsoft.CodeDom.Providers.DotNetCompilerPlatform.props')" />
  <Import Project="..\packages\Microsoft.Net.Compilers.1.3.2\build\Microsoft.Net.Compilers.props" Condition="Exists('..\packages\Microsoft.Net.Compilers.1.3.2\build\Microsoft.Net.Compilers.props')" />
  <Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props" Condition="Exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props')" />
  <PropertyGroup>
    <Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
    <Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
    <ProductVersion>
    </ProductVersion>
    <SchemaVersion>2.0</SchemaVersion>
    <ProjectGuid>{C4AC5BDD-3A85-4B29-BC6C-021E4CE92052}</ProjectGuid>
    <ProjectTypeGuids>{349c5851-65df-11da-9384-00065b846f21};{fae04ec0-301f-11d3-bf4b-00c04f79efbc}</ProjectTypeGuids>
    <OutputType>Library</OutputType>
    <AppDesignerFolder>Properties</AppDesignerFolder>
    <TargetFrameworkVersion>v4.6.1</TargetFrameworkVersion>
MaximRouiller commented 7 years ago

Also disabled most extensions. Only stuff remaining is MS extensions and ReSharper.

I now have 4 folders.

emgarten commented 7 years ago

Would you open the package management console and paste the four digit version number? It looks like NuGet doesn't show the 4th digit in the about info you pasted earlier.

Try disabling ReSharper also if you can. I don't have it installed and am starting to wonder if that is why I can't repro this.

MaximRouiller commented 7 years ago

NuGet Version: 3.5.0.1484 Date Installed: 2016-07-20

Re-enabled most extensions. Restarted Visual Studio (Yeah! 5th folder!). Suspending Resharper. Closing Visual Studio. Re-open Visual Studio. Open up the client's solution.

5 folders. Let me close everything down again and reopen.

MaximRouiller commented 7 years ago

Folder opening now is super fast.

I blame ReSharper. How do we go about this?

MaximRouiller commented 7 years ago

They don't seem to have the issue logged on their end.

https://youtrack.jetbrains.com/issues/RSRP?q=NuGetScratch

emgarten commented 7 years ago

Interesting!

You could try one more thing, get the latest 3.5.0 RTM from https://dist.nuget.org/index.html (It looks like you have a slightly older version of the 3.5.0 beta). With ReSharper running see if it still happens, just to confirm.

ReSharper probably isn't writing to NuGetScratch on their own, they are probably calling an API that isn't normally used and NuGet is not cleaning up.

I think this will need to be debugged on the NuGet side to figure out when/where it writes all those files out. NuGet doesn't do much on solution open, so it is likely that ReSharper is querying NuGet for package info and somehow this happens. Usually the scratch folder is only used when downloading packages from online with caching off, so the whole thing is odd.

MaximRouiller commented 7 years ago

Closing Visual Studio. Downloading latest RTM from provided URL. Running VSIX. Installing. Starting Visual Studio. Resuming ReSharper.

Installed version of NuGet: 3.5.0.1996

Restarting Visual Studio. Opening client's solution. Lock folder appearing. 6th folder (a0dbc1a0-cc98-45f3-8b47-7c630da6551f). Closing Solution. 7 folder still. Restarting Visual Studio. Opening client's solution. 7th folder (83ba8700-2165-4c0e-ba2c-9db3d0d4bfac)!

Yeah. Latest version doesn't seem to fix the issue.

During the time I took to check on this issue, I now have 8 folders (including the lock folder). image

Each folder average 140Mb for a big total of about 1Gb.

Anything else I can do?

emgarten commented 7 years ago

Thanks for the help @MaximRouiller! I think that narrows it down enough for now.

Looking through the NuGet code I don't see NuGetScratch for anything beyond the lock folder. Will do some more digging to see where this might be coming from.

MaximRouiller commented 7 years ago

@emgarten The lock folder didn't reappear until I updated the extension. If that helps.

Every time a new folder appeared, you can see that the "Last Modified" date of the lock match exactly the new GUID. Might matter. 😄

If you can keep us posted as to what was the root cause, I would appreciate. Maybe they are using an undocumented API? Who knows.

daveaglick commented 7 years ago

I got nerd-sniped on this issue and am trying to gather some additional info for you. I started by running procmon while loading a solution and caught ReSharper creating the files:

2016-12-06_14h20_28 2016-12-06_14h17_49 2016-12-06_14h18_03 2016-12-06_14h18_15

Looks like the call is coming from the JetBrains.Profiler.Editor.Windows.ViewModel.Impl.dll assembly. I'm going to try getting a managed stack trace using PerfView next...

emgarten commented 7 years ago

@daveaglick thanks! that helps confirm my theory a bit.

It looks like this is coming from ReSharper's use of NuGet.Core.dll. This assembly is legacy and no longer used in the NuGet extension.

When NuGet.Core.dll is used to open a nupkg it will be extracted to NuGetScratch/{guid}/ https://github.com/NuGet/NuGet2/blob/e89c4b2fc6dc2fc752d2fceecd2a2175b0f4ab69/src/Core/Packages/OptimizedZipPackage.cs#L29-L30

Previously this was being cleaned up by the NuGet extension which called PurgeCache https://github.com/NuGet/NuGet2/blob/e89c4b2fc6dc2fc752d2fceecd2a2175b0f4ab69/src/Core/Packages/OptimizedZipPackage.cs#L315 when VS was closed.

NuGet no longer cleans up this folder when VS is closed. It has since moved to cleaning up temp folders immediately after the operation instead of having a global temp folder that is cleaned up on close (this was buggy before since other instances of VS could still be using the files).

It looks like ReSharper was relying on the NuGet VSIX to clean up everything, and that no longer happens since NuGet has moved off of NuGet.Core.dll.

Possible workarounds:

  1. Revert to NuGet 3.4.x
  2. An extension could be written to clean this folder up on close
MaximRouiller commented 7 years ago

Nice work @daveaglick ! 👍

daveaglick commented 7 years ago

Pretty sure I can confirm that theory now: 2016-12-06_14h52_29

If I'm reading this dump correctly, NuGet.PackageExtensions.GetFiles(IPackage,string) from NuGet.Core.dll looks to be the guilty party.

daveaglick commented 7 years ago

Went ahead and opened an issue with JetBrains: https://youtrack.jetbrains.com/issue/RSRP-462102

MaximRouiller commented 7 years ago

Issue was updated. Basically, they need to get rid of Obsolete DLL. Bug already fixed in their preview track. New release should come mid-December.

MaximRouiller commented 7 years ago

Running latest ReSharper update.

Version: 2016.3

%userprofile%\AppData\Local\Temp\NuGetScratch ain't growing anymore. Project loading is also faster.

Please update your ReSharper to the latest update.

MaximRouiller commented 7 years ago

Also... time to do one final cleanup. 😉

kiewic commented 7 years ago

Hi, I have the same issue running an application that uses Nuget.Core.dll to download and install NuGet packages. Our machines are running out of space all the time because of files left behind by this DLL. What are the options to replace Nuget.Core.dll? Is there a new DLL with the same functionality? Should I install nuget.exe and invoke the process from the application? Please let me know if you have any recommended approach to download and install packages programmatically.

emgarten commented 7 years ago

@kiewic you need to call the purge method once you finish using nuget.core.dll

https://github.com/NuGet/NuGet2/blob/e89c4b2fc6dc2fc752d2fceecd2a2175b0f4ab69/src/Core/Packages/OptimizedZipPackage.cs#L315