search

xunit / visualstudio.xunit

VSTest runner for xUnit.net (for Visual Studio Test Explorer and dotnet test)
https://xunit.net/
Other
144 stars 81 forks source link

Modified date of redistributed xunit.abstractions.dll is wrong #399

Closed ViktorHofer closed 7 months ago

ViktorHofer commented 7 months ago

For a better solution to https://github.com/xunit/xunit/issues/1651, @akoeplinger and I were trying to understand why the modified date of xunit.abstractions.dll in the xunit.runner.visualstudio.nupkg package is different to the one in the xunit.abstractions.nupkg on nuget.org.

When I locally build this repository, I see the correct last modified date in the produced package:

image

But when I look at the officially released package on nuget.org, the last modified date is wrong (same for the most recent version on feedz.io):

image

ℹ️ I unzipped with WinRAR which better handles preserving timestamps than Windows Explorer

@bradwilson any idea where this difference could be coming from?

bradwilson commented 7 months ago

I appreciate this investigation, but to what end? It's already too late to do anything.

ViktorHofer commented 7 months ago

What do you mean by too late? The time stamp in the xunit.abstractions nuget package on nuget.org is correct.

bradwilson commented 7 months ago

The timestamps of everything are the date it was packaged, even when the binaries are identical:

image

bradwilson commented 7 months ago

It's possible that NuGet is modifying the packages when it signs them.

bradwilson commented 7 months ago

My point of it being "too late" is that the four versions of xunit.abstractions.dll all have different dates already, so even if you "fixed" future packages, it still doesn't retroactively fix the four versions of xunit.abstractions.

bradwilson commented 7 months ago

Actually, I'm looking at the package that I downloaded from the CI build, and it appears that the binary signing process is what's changing the dates.

Local build (built just now):

2024-01-16 11:04:38 .....          519          291  _rels\.rels
2024-01-16 11:04:38 .....         1526          751  xunit.runner.visualstudio.nuspec
2023-10-10 06:47:44 .....         1895         1900  _content\logo-128-transparent.png
2023-11-18 21:03:08 .....         4520         1281  _content\README.md
2018-08-25 19:54:14 .....        21928        10879  build\net462\xunit.abstractions.dll
2023-12-22 23:16:34 .....        80928        44341  build\net462\xunit.runner.reporters.net452.dll
2023-12-22 23:16:34 .....       267808       132886  build\net462\xunit.runner.utility.net452.dll
2024-01-16 19:04:32 .....       136192        72586  build\net462\xunit.runner.visualstudio.testadapter.dll
2024-01-15 19:40:02 .....          969          322  build\net462\xunit.runner.visualstudio.props
2024-01-15 19:53:56 .....          389          264  build\net462\xunit.runner.visualstudio.targets
2018-08-25 19:54:14 .....        21928        10955  build\net6.0\xunit.abstractions.dll
2023-12-22 23:16:34 .....        83488        46660  build\net6.0\xunit.runner.reporters.netcoreapp10.dll
2023-12-22 23:16:34 .....       245280       123463  build\net6.0\xunit.runner.utility.netcoreapp10.dll
2024-01-16 19:04:34 .....       167424        87462  build\net6.0\xunit.runner.visualstudio.testadapter.dll
2023-12-18 19:21:26 .....          993          325  build\net6.0\xunit.runner.visualstudio.props
2024-01-15 19:53:56 .....          389          264  build\net6.0\xunit.runner.visualstudio.targets
2023-10-10 06:47:44 .....            0            0  lib\net462\_._
2023-10-10 06:47:44 .....            0            0  lib\net6.0\_._
2024-01-16 11:04:38 .....          783          236  [Content_Types].xml
2024-01-16 11:04:38 .....          819          493  package\services\metadata\core-properties\0e1897253a17448081adc2f818565cb0.psmdcp

Downloaded CI build (built yesterday):

2024-01-15 20:34:48 .....         1526          745  xunit.runner.visualstudio.nuspec
2024-01-15 20:34:48 .....          783          230  [Content_Types].xml
2024-01-15 20:32:08 .....         1895         1900  _content\logo-128-transparent.png
2024-01-15 20:32:08 .....         4520         1281  _content\README.md
2024-01-15 20:34:48 .....          519          286  _rels\.rels
2024-01-15 20:34:54 .....        24608        13623  build\net462\xunit.abstractions.dll
2024-01-15 20:34:54 .....        80928        44350  build\net462\xunit.runner.reporters.net452.dll
2024-01-15 20:34:54 .....       267808       132891  build\net462\xunit.runner.utility.net452.dll
2024-01-15 20:32:08 .....          969          322  build\net462\xunit.runner.visualstudio.props
2024-01-15 20:32:08 .....          389          264  build\net462\xunit.runner.visualstudio.targets
2024-01-15 20:34:54 .....       146976        80331  build\net462\xunit.runner.visualstudio.testadapter.dll
2024-01-15 20:34:54 .....        24608        13700  build\net6.0\xunit.abstractions.dll
2024-01-15 20:34:54 .....        83488        46664  build\net6.0\xunit.runner.reporters.netcoreapp10.dll
2024-01-15 20:34:54 .....       245280       123463  build\net6.0\xunit.runner.utility.netcoreapp10.dll
2024-01-15 20:32:08 .....          993          325  build\net6.0\xunit.runner.visualstudio.props
2024-01-15 20:32:08 .....          389          264  build\net6.0\xunit.runner.visualstudio.targets
2024-01-15 20:34:54 .....       178208        95144  build\net6.0\xunit.runner.visualstudio.testadapter.dll
2024-01-15 20:32:08 .....            0            0  lib\net462\_._
2024-01-15 20:32:08 .....            0            0  lib\net6.0\_._
2024-01-15 20:34:48 .....          819          487  package\services\metadata\core-properties\48e1f79a2c9d46a69be616803b628c4c.psmdcp
2024-01-15 20:34:54 .....        11831        11831  .signature.p7s

Also looks like some of the dates may be being restored improperly on GitHub (like the files under the _content folder, which aren't being signed and therefore aren't being changed, unlike the rest of the files with the 2024-01-15 20:34:54 date, which are the signed binaries).

bradwilson commented 7 months ago

looks like some of the dates may be being restored improperly on GitHub

Nevermind on that part, Git does not preserve file times. The differences are fresh pull vs. how long they've been sitting on my machine. So note that: putting something into the repository will destroy its date/time.

https://archive.kernel.org/oldwiki/git.wiki.kernel.org/index.php/Git_FAQ.html#Why_isn.27t_Git_preserving_modification_time_on_files.3F

bradwilson commented 7 months ago

It's possible that NuGet is modifying the packages when it signs them.

This is true, but it only touches .signature.p7s Interestingly, it signs them with the incorrect date/time, because the file it generates has a time that is in the past.

CI package:

2023-12-22 23:30:14 .....        11832        11832  .signature.p7s

NuGet version (which was uploaded from the CI package):

2023-12-22 15:38:00 .....        23226        23226  .signature.p7s

Looks very much like a time zone bug, but probably one nobody cares about.

bradwilson commented 7 months ago

Given that this is the binary signing process at work, I'm going to mark this as an external issue. You may wish to open an issue here: https://github.com/dotnet/sign

Note, however, that the signing process will change the file contents, so it still won't make the up-to-date check work. So chase it if you wish, it's not going to solve the original problem.

ViktorHofer commented 7 months ago

Why do we need to sign xunit.abstractions.dll again? It's already coming in from the nuget.org package so I would expect it to already be signed. Or am I wrong on that?

bradwilson commented 7 months ago

That would be a question to ask of the dotnet/sign project. The binary that's in there is already signed, but I'm guessing they re-sign it anyways.

bradwilson commented 7 months ago

To be clear: the signing request we make is "sign this NuGet package" and they sign the package itself as well as any signable binaries inside of it. We don't pick which DLLs get signed.

bradwilson commented 7 months ago

Looks like this issue might be related: https://github.com/dotnet/sign/issues/647

bradwilson commented 7 months ago

Available in 2.5.7-pre.6

2018-08-25 19:54:14 .....        21928        10879  build\net462\xunit.abstractions.dll
2023-12-22 23:16:34 .....        80928        44341  build\net462\xunit.runner.reporters.net452.dll
2023-12-22 23:16:34 .....       267808       132886  build\net462\xunit.runner.utility.net452.dll
2024-01-17 19:26:30 .....       146976        80381  build\net462\xunit.runner.visualstudio.testadapter.dll

2018-08-25 19:54:14 .....        21928        10955  build\net6.0\xunit.abstractions.dll
2023-12-22 23:16:34 .....        83488        46660  build\net6.0\xunit.runner.reporters.netcoreapp10.dll
2023-12-22 23:16:34 .....       245280       123463  build\net6.0\xunit.runner.utility.netcoreapp10.dll
2024-01-17 19:26:30 .....       178208        95128  build\net6.0\xunit.runner.visualstudio.testadapter.dll

2024-01-17 19:26:30 .....        11832        11832  .signature.p7s
ViktorHofer commented 7 months ago

Amazing!! Thanks for following-up. Can we please revert https://github.com/xunit/visualstudio.xunit/commit/e266e72fbd58b43a34636e481d5b1523104bc040 & https://github.com/xunit/visualstudio.xunit/commit/3cb1ad21eb3cda20218cafd48b696813a054efe5 now? Those changes aren't necessary anymore and I'm anxious that they might cause some other issues.

bradwilson commented 7 months ago

Yep, was planning to do that after lunch. 😄