Building clean repo modifies many project.lock.json files

[ghost]

Version Used:

Update 2

Steps to Reproduce:

1. Fork dotnet/roslyn
2. Clone the fork
3. Restore.cmd
4. msbuild /v:m /m Roslyn.sln

Expected Behavior: Build succeeds

Actual Behavior: Build succeeds, but 99 project.lock.json files are modified. The only change is that the sha256 of System.Reflection.Metadata/1.2.0 has changed.

[davkean]

Same in future, can't figure out how we keep regressing this.

[AdamSpeight2008]

What is it that the project.lock.json files do? Are they like the visual studio lock files, which are generate when you up a document, from with a project. So that it indicates that it is being edited? .

[ghost]

To understand what project.lock.json is, you first need to know what project.json is.

In NuGet version 3, you specify your NuGet dependencies in a file named project.json -- unlike version 2, where you used a file called packages.config. Also unlike NuGet v2, in NuGet v3 you only have to specify your "top-level" dependencies. If you need package A, which depends on B and C, you only need to specify "A" in your project.json.

NuGet v3 "transitively resolves" these dependencies, and decide which version of each package to use. The file project.lock.json records the result of this resolution.

Now, also unlike NuGet v2, NuGet v3 does not alter your project files on disk by adding references to binaries obtained from NuGet packages. Instead, at build time, the project system uses the information in project.lock.json to dynamically add the necessary references to an in-memory copy of your project file, and then the build system acts on the resulting "enhanced" project file.

HTH, Larry

[paul1956]

Could upgrading to NuGet version 3 cause a working VB project to break? I can no longer find a reference to System.Disgnostics.Process. I asked about this in #9762.

[AdamSpeight2008]

@lgolding Thank you. That leads the to question, what stages in the process could lead to that in-memory copy to be output to disk? or overrides the "true" project file? As the files on disc would have to changed to affect the git repo.

[ghost]

@AdamSpeight2008 That in-memory copy of the project file (synthesized by combining the information in the on-disk project file with the information in the project.json.lock file) is never written back to disk. It is recomputed dynamically on each build.

[AdamSpeight2008]

@lgolding At some stage in the process those modified project.lock.json files, have to output on to disc. If is after step 3? or after step 4?

[ghost]

Oh, the lock files. I thought you meant the csproj file. I don't know.

@tagoo ?

[Pilchie]

Any update here? Many people (@dpoeschl, @DustinCampbell, myself) are all still running into this in several branches. Is it possible that there are different versions of System.Reflection.Metadata on NuGet and MyGet and so it depends which one downloads first or something like that?

[tmat]

@Pilchie The packages on nuget and myget appear to be identical. Maybe someone from nuget team should look into this. I plan to update master to SRM 1.3.0 soon(ish), which might avoid the problem.

[Pilchie]

@emgarten @ericstj any ideas here?

[emgarten]

There appears to be a problem with this package on the dotnet-core myget feed. The nupkg at the v3 url does not match nuget.org or the myget v2 feed.

Since Roslyn contains both the dotnet-core feed and nuget.org, and the package exists on both NuGet will take the fastest source. In this case the packages are different so the copy you end up with may be changing back and forth.

The myget v3 url here: https://dotnetmyget.blob.core.windows.net/artifacts/dotnet-core/nuget/v3/flatcontainer/system.reflection.metadata/1.2.0/system.reflection.metadata.1.2.0.nupkg

Contains this nuspec:

<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2013/01/nuspec.xsd">
  <metadata minClientVersion="2.8.6">
    <id>System.Reflection.Metadata</id>
    <version>1.2.0</version>
    <title>System.Reflection.Metadata</title>
    <authors>Microsoft</authors>
    <owners>microsoft,dotnetframework</owners>
    <licenseUrl>http://go.microsoft.com/fwlink/?LinkId=329770</licenseUrl>
    <iconUrl>http://go.microsoft.com/fwlink/?LinkID=288859</iconUrl>
    <requireLicenseAcceptance>true</requireLicenseAcceptance>
    <description>This packages provides a low-level .NET (ECMA-335) metadata reader. It's geared for performance and is the ideal choice for building higher-level libraries that intend to provide their own object model, such as compilers.

Commonly Used Types:
System.Reflection.Metadata.MetadataReader
System.Reflection.PortableExecutable.PEReader
 \r\n TFS ID: 1581887, GitHub SHA: https://github.com/dotnet/corefx/tree/dacca1618c0dbb266945d42e58b9584e7c72126c</description>
    <copyright>© Microsoft Corporation.  All rights reserved.</copyright>
    <tags></tags>
    <dependencies>
      <group targetFramework=".NETPortable0.0-Profile7">
        <dependency id="System.Collections.Immutable" version="[1.1.37, )" />
      </group>
    </dependencies>
  </metadata>
</package>

nuget.org, myget v2, and the myget website have this nupkg: https://dotnet.myget.org/F/dotnet-core/api/v2/package/System.Reflection.Metadata/1.2.0

with nuspec:

<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2013/01/nuspec.xsd">
  <metadata minClientVersion="2.8.6">
    <id>System.Reflection.Metadata</id>
    <version>1.2.0</version>
    <title>System.Reflection.Metadata</title>
    <authors>Microsoft</authors>
    <owners>microsoft,dotnetframework</owners>
    <licenseUrl>http://go.microsoft.com/fwlink/?LinkId=329770</licenseUrl>
    <iconUrl>http://go.microsoft.com/fwlink/?LinkID=288859</iconUrl>
    <requireLicenseAcceptance>true</requireLicenseAcceptance>
    <description>This packages provides a low-level .NET (ECMA-335) metadata reader. It's geared for performance and is the ideal choice for building higher-level libraries that intend to provide their own object model, such as compilers.

Commonly Used Types:
System.Reflection.Metadata.MetadataReader
System.Reflection.PortableExecutable.PEReader</description>
    <copyright>© Microsoft Corporation.  All rights reserved.</copyright>
    <tags></tags>
    <dependencies>
      <group targetFramework="MonoAndroid1.0">
        <dependency id="System.Collections.Immutable" version="[1.1.37, )" />
      </group>
      <group targetFramework="MonoTouch1.0">
        <dependency id="System.Collections.Immutable" version="[1.1.37, )" />
      </group>
      <group targetFramework=".NETFramework4.5">
        <dependency id="System.Collections.Immutable" version="[1.1.37, )" />
      </group>
      <group targetFramework=".NETStandard1.1">
        <dependency id="System.Collections" version="[4.0.0, )" />
        <dependency id="System.Collections.Immutable" version="[1.1.37, )" />
        <dependency id="System.Diagnostics.Debug" version="[4.0.0, )" />
        <dependency id="System.IO" version="[4.0.0, )" />
        <dependency id="System.Linq" version="[4.0.0, )" />
        <dependency id="System.Reflection" version="[4.0.0, )" />
        <dependency id="System.Reflection.Extensions" version="[4.0.0, )" />
        <dependency id="System.Reflection.Primitives" version="[4.0.0, )" />
        <dependency id="System.Resources.ResourceManager" version="[4.0.0, )" />
        <dependency id="System.Runtime" version="[4.0.0, )" />
        <dependency id="System.Runtime.Extensions" version="[4.0.0, )" />
        <dependency id="System.Runtime.InteropServices" version="[4.0.0, )" />
        <dependency id="System.Text.Encoding" version="[4.0.0, )" />
        <dependency id="System.Text.Encoding.Extensions" version="[4.0.0, )" />
        <dependency id="System.Threading" version="[4.0.0, )" />
      </group>
      <group targetFramework=".NETPortable0.0-Profile7">
        <dependency id="System.Collections.Immutable" version="[1.1.37, )" />
      </group>
      <group targetFramework="Windows8.0">
        <dependency id="System.Collections.Immutable" version="[1.1.37, )" />
      </group>
      <group targetFramework="WindowsPhoneApp8.1">
        <dependency id="System.Collections.Immutable" version="[1.1.37, )" />
      </group>
      <group targetFramework="Xamarin.iOS1.0">
        <dependency id="System.Collections.Immutable" version="[1.1.37, )" />
      </group>
      <group targetFramework="Xamarin.Mac2.0">
        <dependency id="System.Collections.Immutable" version="[1.1.37, )" />
      </group>
      <group targetFramework="Xamarin.TVOS1.0">
        <dependency id="System.Collections.Immutable" version="[1.1.37, )" />
      </group>
      <group targetFramework="Xamarin.WatchOS1.0">
        <dependency id="System.Collections.Immutable" version="[1.1.37, )" />
      </group>
    </dependencies>
  </metadata>
</package>

//cc @maartenba

[maartenba]

Seems like the package has been updated in a big bulk push a while ago and the v3 one did not propagate. We've fixed this on MyGet side.

[Pilchie]

Thanks @maartenba!

[dpoeschl]

Current project.lock.json noise:

	Closed	Open
master	0	0
future	7	36

Looks like @agocke deleted the project.lock.json files from master. Can we remove them from future as well?

[dpoeschl]

@agocke Can we port your project.lock.json deletion to other branches?

[dpoeschl]

@jasonmalinowski - Doesn't look like this got merged to future appropriately. @agocke fixed this in master so all that's left is merging the fix around.

Also, unassigning @tmat

[jasonmalinowski]

@agocke's change did merge successfully, but only deleted the remaining project.lock.jsons in master. I had already deleted the EditorFeatures-and-higher ones in master, but it appears those were mismerged in future. I'll fix that now.

[jasonmalinowski]

And #10316 is out to fix that issue.