search

stazz / UtilPack

Home of UtilPack - library with various useful and generic stuff for .NET.
17 stars 6 forks source link

Caching of restores #28

Closed jnevins-gcm closed 6 years ago

jnevins-gcm commented 6 years ago

I'm using a package that uses UtilPack.NuGet (GitVersionTask). It's extremely slow because every project that uses GitVersionTask in turn has UtilPack.NuGet try to resolve all dependencies with NuGet every time. So for every single project being built in a solution, I see:

[NuGet Minimal]: Restoring packages for C:\Users\user\AppData\Local\Temp\yr2nqvrh.qcc\dummy...

is there a way to have UtilPack not try to do a restore every time?

stazz commented 6 years ago

Hi,

I've added caching of BoundRestoreCommandUser objects (which in turn cache restores themselves) in version 2.5.0 of UtilPack.NuGet.MSBuild package. Is the GitVersionTask using that version or newer?

Edit: I forgot to mention that the cache being used is in-process. So if you have one big build which utilizes the UtilPack.NuGet.MSBuild task factory, and it is used to restore the same package, that package restoration result will be cached on first usage within the same build and used on subsequent usages, as long as the process is the same. The feature to cache the restoration results so that they would be persistable across multiple build runs (basically, writing the lock file to disk) is not yet implemented.

stazz commented 6 years ago

If your issue was about persistent caching of restore results, it is now tracked in #29 .

jnevins-gcm commented 6 years ago

Updating to 2.5.0 manually within the projects that use GitVersionTask worked! Thanks!!

Should I upgrade it to 2.5.0 or 2.6.0? Also, is there any way to suppress this single warning without changing the logging level of our whole build?

Task factory warning NMSBT010: There is a mismatch between SDK NuGet version (4.7.0) and the NuGet version the task factory was compiled against (4.6.0). There might occur some exotic errors.

stazz commented 6 years ago

Glad to hear that caching in 2.5.0 worked for you! The different between 2.5.0 and 2.6.0 is that the latter is aware of NuGet 4.7.0, so you will get rid of that warning by upgrading to 2.6.0.

I am also preparing version 2.7.0, which can automatically deduce .NET Core 2.1 environment, but I am not sure yet when it'll be out.

jnevins-gcm commented 6 years ago

This is wonderful, thanks.

Though I'm still getting MSBUILD : Task factory warning NMSBT010: There is a mismatch between SDK NuGet version (4.7.0) and the NuGet version the task factory was compiled against (4.6.0). There might occur some exotic errors.

with 2.6.0

Does GitVersionTask also need to update to 2.6.0 itself?

stazz commented 6 years ago

No, I don't think it does. I tried reproducing that by running a task but I didn't get the warning. Maybe try setting verbosity to detailed or even diagnostic to see which path MSBuild loads the task factory from, to reveal which version of UtilPack.NuGet.MSBuild is actually used?

jnevins-gcm commented 6 years ago

Wow! That's bizarre - I tried to remove 2.4.0 and 2.5.0 from my nuget cache which made me notice that dotnet cli was doing its process caching thing and that both 2.4.0, 2.5.0 and 2.6.0 files were all locked)...so I stopped all dotnet.exe processes and the warning went away after the next package restore. I guess you're doing AssemblyResolution at the dotnet.exe process itself, not within an isolated LoadContext?

stazz commented 6 years ago

All assembly resolution in both .NET Desktop and .NET Core is always done within the process itself, but by default, the UtilPack.NuGet.MSBuild locks the files themselves and not their copies (i.e. not doing the shadow-copying).

You can enable the shadow-copying of assembly files in UtilPack.NuGet.MSBuild by setting CopyToFolderBeforeLoad task factory parameter (see also full list of parameters) to true or to some path which will hold the copied assemblies. That way, the original assemblies remain unlocked. :)

dazinator commented 6 years ago

@stazz @jnevins-gcm thanks both for this, I have updated GitVersion's GitVersionTask to use 2.6.0. Hopefully this will carry with it the speed improvements for restoration and resolve the build warning.

dazinator commented 6 years ago

This issue of build warning has resurfaced again with the release of nuget 4.8.0.

stazz commented 6 years ago

Geezus, the NuGet versions sure keep on coming! I will publish new version of UtilPack.NuGet which is 4.8.0-aware asap, probably this weekend!

P.S. Making UtilPack.NuGet as .NET Core Global Tool is something that might help with NuGet version problems (tracked in #31 now).

stazz commented 6 years ago

Release version 2.7.0 which contains support for NuGet 4.8.0.

Important! The desktop assemblies are not contained in build/net46 folder instead of build/net45 folder, as I finally moved the desktop to use newer version of NuGet as well. So if @dazinator you have any hard-coded paths to net45, you should change them to net46 for this release. :)