KirillOsenkov / MetadataTools

Various tools and helpers to read assembly metadata.
MIT License
195 stars 25 forks source link

App.config is not read when analyzing a folder of only dlls #30

Open bbronisz opened 7 hours ago

bbronisz commented 7 hours ago

Hi,

I wanted to analyze Microsoft.Kiota.Serialization.Json.dll and its dependencies, with some redirects to see if they work correctly. But when I've put them in one folder and redirects in app.config it reported issues that should be handled by those redirects, as if they were not read. After debugging I've found that app.config is not taken into account but app.exe.config is OK. I think that command-line option to explicitly specify config file, or report that no config files were used or better documentation on how to name the config files would be helpful. Or use any *.config files, instead of only *.exe.config.

Thanks for the tool. Unfortunately it didn't helped me to find the cause of compatibility issue :), but I'll probably use it more often in future.

bbronisz commented 6 hours ago

It's maybe not totally related but might be: the tool reports dependencies from .NET 6: C:\Program Files\dotnet\shared\Microsoft.NETCore.App\6.0.36\System.Text.Encodings.Web.dll .NET 8: C:\Program Files\dotnet\shared\Microsoft.NETCore.App\8.0.11\System.Text.Encodings.Web.dll and .NET Framework GAC: C:\WINDOWS\Microsoft.NET\assembly\GAC_MSIL\System.Numerics\v4.0_4.0.0.0__b77a5c561934e089\System.Numerics.dll Could there be an option to specify framework? Or should it be read form app.config?

KirillOsenkov commented 3 hours ago

are all the dlls you're analyzing public? any chance you could zip the whole directory and upload here, along with the command line used?

app.config files aren't used at runtime, they are converted to yourapp.exe.config and the runtime looks for the exe.config next to the exe of the same name. You can manually specify your app.config as a separate argument on the command line and it will be checked.

bbronisz commented 2 hours ago

BinaryCompat.zip Simply: checkbinarycompat.

I know about app.config converting to app.exe.config but I was investigating a specific issue with incompatible dlls/assembly redirects which didn't have an executable that I could use. So I just copied the dlls to single folder + config to mimick the scenario where the issue happens to see what the tool would tell me.

What is the command line argument to specify my app.config? I've tried checkbinarycompat *.dll;app.config and checkbinarycompat -p:*.dll;*.config and some combinations but still don't see the app.config being used.

bbronisz commented 2 hours ago

I've found that the issue was caused by lack of assembly redirect for System.Text.Encodings.Web to version 8.0.0.0. Microsoft.Kiota.Serialization.Json.KiotaJsonSerializationContext.Default was throwing method not found for System.Text.Json.JsonEncodedText Encode(System.String, System.Text.Encodings.Web.JavaScriptEncoder). It was because the dll method had this signature:

static System.Text.Json.JsonEncodedText, System.Text.Json, Version=8.0.0.5, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51 
    Encode(
        System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 value, 
        System.Text.Encodings.Web.JavaScriptEncoder, System.Text.Encodings.Web, Version=8.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51 encoder)

but Microsoft.Kiota.Serialization.Json was expecting this:

static System.Text.Json.JsonEncodedText, System.Text.Json, Version=8.0.0.5, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51 
    Encode(
        System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 value, 
        System.Text.Encodings.Web.JavaScriptEncoder, System.Text.Encodings.Web, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51 encoder)

If checkbinarycompat would be able to detect such compatibility issues that would be great and amazing 😃.

KirillOsenkov commented 1 hour ago

Regarding the docs, we do have this: image

KirillOsenkov commented 1 hour ago

you can run checkbinarycompat *.dll;app.config to include your app.config

KirillOsenkov commented 1 hour ago

hmm, wait, that doesn't work. It is a bug indeed.

bbronisz commented 1 hour ago

And running it from powershell it says at the end:

[general]
  The command "app.config" was not found, but does exist in the current location.
  PowerShell does not load commands from the current location by default (see 'Get-Help about_Command_Precedence').

  If you trust this command, run the following command instead:
    ➤ .\app.config
KirillOsenkov commented 1 hour ago

Indeed, I see the references to 6.0.0.0 and 8.0.0.0: image

I guess they're not reported as a problem because System.Text.Encodings.Web.dll is not present in the directory. The tool then resolves the 6.0.0.0 version from C:\Program Files\dotnet\shared\Microsoft.NETCore.App\6.0.36\System.Text.Encodings.Web.dll and the 8.0.0.0 version from C:\Program Files\dotnet\shared\Microsoft.NETCore.App\8.0.11\System.Text.Encodings.Web.dll.

There's currently no way to specify the runtime to resolve from, it will attempt to resolve from all runtimes. As a workaround, if you copy either .dll to the directory, it should resolve it from there and it should participate in analysis:

=================================
These actual lines are new:
Assembly `System.Text.Encodings.Web` is referencing `System.Memory, Version=6.0.0.0, PublicKeyToken=cc7b13ffcd2ddd51` but found `System.Memory, Version=4.0.1.2, PublicKeyToken=cc7b13ffcd2ddd51` at `System.Memory.dll`
Assembly `System.Text.Encodings.Web` is referencing `System.Numerics.Vectors, Version=6.0.0.0, PublicKeyToken=b03f5f7f11d50a3a` but found `System.Numerics.Vectors, Version=4.1.4.0, PublicKeyToken=b03f5f7f11d50a3a` at `System.Numerics.Vectors.dll`
Assembly `System.Text.Json` is referencing `System.Text.Encodings.Web, Version=8.0.0.0, PublicKeyToken=cc7b13ffcd2ddd51` but found `System.Text.Encodings.Web, Version=6.0.0.0, PublicKeyToken=cc7b13ffcd2ddd51` at `System.Text.Encodings.Web.dll`
In assembly 'System.Text.Encodings.Web, Version=6.0.0.0, PublicKeyToken=cc7b13ffcd2ddd51': Failed to resolve member reference 'System.Void System.Numerics.Vector`1<System.UInt32>::.ctor(System.ReadOnlySpan`1<System.Byte>)' in assembly 'System.Numerics.Vectors, Version=6.0.0.0, PublicKeyToken=b03f5f7f11d50a3a'
In assembly 'System.Text.Encodings.Web, Version=6.0.0.0, PublicKeyToken=cc7b13ffcd2ddd51': Failed to resolve member reference 'System.Void System.Numerics.Vector`1<System.UInt32>::.ctor(System.Span`1<!0>)' in assembly 'System.Numerics.Vectors, Version=6.0.0.0, PublicKeyToken=b03f5f7f11d50a3a'
In assembly 'System.Text.Encodings.Web, Version=6.0.0.0, PublicKeyToken=cc7b13ffcd2ddd51': Failed to resolve member reference 'System.Void System.Numerics.Vector`1<System.UInt32>::CopyTo(System.Span`1<!0>)' in assembly 'System.Numerics.Vectors, Version=6.0.0.0, PublicKeyToken=b03f5f7f11d50a3a'
In assembly 'System.Text.Encodings.Web, Version=6.0.0.0, PublicKeyToken=cc7b13ffcd2ddd51': reference 'System.Numerics.Vectors, Version=6.0.0.0, PublicKeyToken=b03f5f7f11d50a3a' resolved from 'System.Numerics.Vectors.dll' as 'System.Numerics.Vectors, Version=4.1.4.0, PublicKeyToken=b03f5f7f11d50a3a'
=================================
KirillOsenkov commented 1 hour ago

I guess when using powershell or pwsh you need to surround the argument in quotes: checkbinarycompat '*.dll;app.config'

KirillOsenkov commented 1 hour ago

OK so there are at least two bugs here

  1. passing app.config doesn't work
  2. resolves from all .NET runtime directories and there's no way to limit to a single runtime

For 1 I just made a fix: https://github.com/KirillOsenkov/MetadataTools/commit/4f873476030d41f04f1de69fd265c86f48dfa7d2

I don't want to support .config files because there might be non-app-config files with this extension and I don't want random errors for these files. Also there's no easy way to distinguish the files/patterns explicitly passed on the command line vs. picked up implicitly from the current directory, so there's no simple way to respect the .config files passed by the user. I imagine it'd be possible, but would be complicated to implement and I don't think it's worth it.

Regarding the second problem, not sure what the right way to fix this would be. Perhaps a new option: -t:net6.0 or -t:net472 to tell it to only resolve framework assemblies from a specific runtime.

KirillOsenkov commented 50 minutes ago

OK I added a switch -doNotResolveFromFramework: https://github.com/KirillOsenkov/MetadataTools/commit/159f06f9a5a6482972a4be49aff6386bd76f45da

This way you can specify the exact runtime directory yourself: *.dll;app.config -resolve:"C:\Program Files\dotnet\shared\Microsoft.NETCore.App\8.0.11" -doNotResolveFromFramework

KirillOsenkov commented 47 minutes ago

Hmm, your System.Numerics.Vectors.dll is so old it doesn't have the target framework attribute: image

it's from net4.0 I'm guessing

KirillOsenkov commented 44 minutes ago

I think we got unlucky here, because System.Text.Encodings.Web.dll is not in the GAC for desktop netfx, but it is present in .NET 6.0 and 8.0 runtime directories, so we were resolving it from there. Your System.Text.Json.dll is targeting net462, but references System.Text.Encodings.Web 8.0.0.0.

Seems to be a pretty weird mishmash of dlls.

KirillOsenkov commented 44 minutes ago

Let me know if you think there's anything else I can fix to improve this situation on the tool side. Otherwise I can just publish the two fixes I made already.

bbronisz commented 36 minutes ago

I will give it a try tomorrow and let you know. I will also rerun the test on machine with some of those DLLs in GAC as this is closer to the situation where the problem first showed up. Thanks.