search

smdn / Smdn.Reflection.ReverseGenerating

.NET libraries and tools to reverse-generate C# code from type and member information using reflection
https://smdn.jp/works/libs/Smdn.Reflection.ReverseGenerating/
MIT License
3 stars 0 forks source link

list-api not finding DLLs in other directories including SDK like System.Runtime.dll #39

Closed rene-descartes2021 closed 9 months ago

rene-descartes2021 commented 9 months ago

Description

I run list-api on DLL built by project. list-api doesn't find DLLs in other directories, but does for the same directory. I think that is what occurs. I might be doing something wrong?

Here list-api is run on a dotnet 8 DLL I built:

~/dev/space-station-14/artifacts/bin/Robust.Packaging/release$ list-api -o /tmp Robust.Shared.dll
System.IO.FileNotFoundException: Could not load file or assembly 'System.Runtime, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.

File name: 'System.Runtime, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'
   at System.ModuleHandle.ResolveMethod(QCallModule module, Int32 methodToken, IntPtr* typeInstArgs, Int32 typeInstCount, IntPtr* methodInstArgs, Int32 methodInstCount)
   at System.ModuleHandle.ResolveMethodHandleInternal(RuntimeModule module, Int32 methodToken, IntPtr[] typeInstantiationContext, Int32 typeInstCount, IntPtr[] methodInstantiationContext, Int32 methodInstCount)
[...]

Reproduction steps

$ tar xzf dotnet-sdk-8.0.101-linux-arm64.tar.gz -C ~/.dotnet
$ tar xzf dotnet-sdk-6.0.418-linux-arm64.tar.gz -C ~/.dotnet
$ dotnet tool install -g Smdn.Reflection.ReverseGenerating.ListApi

Run list-api on a dll in the dotnet SDK:

$ list-api -o /tmp ~/.dotnet/sdk/8.0.101/DotnetTools/dotnet-format/System.CommandLine.dll

Expected behavior

list-api should find other DLLs like System.Runtime in the SDK.

Actual behavior

$ list-api -o /tmp ~/.dotnet/sdk/8.0.101/DotnetTools/dotnet-format/System.CommandLine.dll
System.IO.FileNotFoundException: Could not load file or assembly 'System.Runtime, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
The system cannot find the file specified.
File name: 'System.Runtime, Version=7.0.0.0, Culture=neutral,  PublicKeyToken=b03f5f7f11d50a3a'
   at System.ModuleHandle.ResolveMethod(QCallModule module, Int32 methodToken, IntPtr* typeInstArgs, Int32 typeInstCount, IntPtr* Methods, Int32 methodInstCount)
   at System.ModuleHandle.ResolveMethodHandleInternal(RuntimeModule module, Int32 methodToken, IntPtr[] typeInstantiationContext, Int32 typeInstCount, IntPtr[] methodInstantiationContext, Int32 methodInstCount)        
   at System.ModuleHandle.ResolveMethodHandle(Int32 methodToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
   at System.RuntimeType.GetMethodBase(RuntimeModule scope, Int32 typeMetadataToken)                          
   at System.Reflection.RuntimeCustomAttributeData..ctor(RuntimeModule scope, MetadataToken caCtorToken, ConstArray& blob)                                          
   at System.Reflection.RuntimeCustomAttributeData.GetCustomAttributes(RuntimeModule module, Int32 tkTarget)
   at System.Reflection.RuntimeCustomAttributeData.Get
[...]

In this case no System.CommandLine.apilist.cs output. But others give just header:

$ cat Microsoft.Build.Tasks.CodeAnalysis-net6.0.apilist.cs
// Microsoft.Build.Tasks.CodeAnalysis.dll (Microsoft.Build.Tasks.CodeAnalysis)
//   Name: Microsoft.Build.Tasks.CodeAnalysis
//   AssemblyVersion: 4.8.9.7201
//   InformationalVersion: 4.8.0-7.23572.1+7b75981cf3bd520b86ec4ed00ec156c8bc48e4eb
//   TargetFramework: .NETCoreApp,Version=v6.0
//   Configuration: Release
//   Referenced assemblies:
//     Microsoft.Build.Framework, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
//     Microsoft.Build.Tasks.Core, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
//     Microsoft.Build.Utilities.Core, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
//     System.Collections, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
//     System.Console, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
//     System.Diagnostics.Process, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
//     System.Diagnostics.StackTrace, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
//     System.IO.Pipes, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
//     System.IO.Pipes.AccessControl, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
//     System.Linq, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
//     System.Memory, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
//     System.Runtime, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
//     System.Runtime.InteropServices, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
//     System.Security.Claims, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
//     System.Security.Cryptography.Algorithms, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
//     System.Security.Cryptography.Primitives, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
//     System.Security.Principal.Windows, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
//     System.Text.Encoding.Extensions, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
//     System.Text.RegularExpressions, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
//     System.Threading, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
//     System.Threading.Thread, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
//   Embedded resources:
//     Microsoft.CodeAnalysis.BuildTasks.ErrorString.resources (5,247 bytes, Embedded, ContainedInManifestFile)

Configuration

$ tar xzf dotnet-sdk-8.0.101-linux-arm64.tar.gz -C ~/.dotnet
$ tar xzf dotnet-sdk-6.0.418-linux-arm64.tar.gz -C ~/.dotnet
$ dotnet tool install -g Smdn.Reflection.ReverseGenerating.ListApi

I installed dotnet SDK 6 in order to get list-api to run.

user@localhost:~$ dotnet --info
.NET SDK:                                          
  Version:           8.0.101
 Commit:            6eceda187b                         
 Workload version:  8.0.100-manifests.69afb982

Runtime Environment:                                
   OS Name:     debian                                 
   OS Version:  12                                    
   OS Platform: Linux                                  
   RID:         linux-arm64                            
   Base Path:   /home/user/.dotnet/sdk/8.0.101/                                                               .NET workloads installed:
  Workload version: 8.0.100-manifests.69afb982         There are no installed workloads to display.

Host:
  Version:      8.0.1                                   
  Architecture: arm64
  Commit:       bf5e279d92 

.NET SDKs installed:                              
   6.0.418 [/home/user/.dotnet/sdk]
   8.0.101 [/home/user/.dotnet/sdk]  

.NET runtimes installed:                                
  Microsoft.AspNetCore.App 6.0.26 [/home/user/.dotnet/shared/Microsoft.AspNetCore.App]                     
  Microsoft.AspNetCore.App 8.0.1 [/home/user/.dotnet/shared/Microsoft.AspNetCore.App]                         
  Microsoft.NETCore.App 6.0.26 [/home/user/.dotnet/shared/Microsoft.NETCore.App]                          
  Microsoft.NETCore.App 8.0.1 [/home/user/.dotnet/shared/Microsoft.NETCore.App]       

Other architectures found:                           
   None                                                                                                      

Environment variables:                                  
  DOTNET_ROOT       [/home/user/.dotnet]                                                                    

global.json file:                                    
   Not found                                                                                                 

Learn more:                                             
  https://aka.ms/dotnet/info                                                                                

Download .NET:                                          
  https://aka.ms/dotnet/download

The environment is unusual. Termux on arm64 Android, and proot (like chroot) into Debian system. Could be my environment, or maybe I don't use list-api correctly? I will try on x86_64 desktop in a few days to try a different environment.

Other information

First time trying list-api. List-api works as expected on other DLLs in ~/.dotnet/sdk/8.0.101/ I think because System.Runtime is in same directory.

smdn commented 9 months ago

Thank you for the detailed report and sorry for inconvenience.

I am currently working on other tasks and it may take several days to research and respond this issue. Until then, try it with the option --load-reflection-only.

list-api --load-reflection-only Your.App.dll

But list-api is behind from newest updates, so this may not work either. Alternatively, Smdn.Reflection.ReverseGenerating.ListApi.MSBuild.Tasks is updated, so you can try this instead.

I would like to update the list-api first, so please wait.

rene-descartes2021 commented 9 months ago

I appreciate your prompt reply. I did try --load-reflection-only but that didn't work, with different and similar error message.

I experimented and found a way to get list-api working as expected using DOTNET_ROLL_FORWARD=Major list-api.

So instead of:

$ tar xzf dotnet-sdk-8.0.101-linux-arm64.tar.gz -C ~/.dotnet
$ tar xzf dotnet-sdk-6.0.418-linux-arm64.tar.gz -C ~/.dotnet
$ dotnet tool install -g Smdn.Reflection.ReverseGenerating.ListApi

I just do version 8, not version 6:

$ tar xzf dotnet-sdk-8.0.101-linux-arm64.tar.gz -C ~/.dotnet
$ dotnet tool install -g Smdn.Reflection.ReverseGenerating.ListApi

Originally I had this problem on launching list-api:

You must install or update .NET to run this application.

App: /home/user/.dotnet/tools/list-api
Architecture: arm64
Framework: 'Microsoft.NETCore.App', version '6.0.0' (arm64)
.NET location: /home/user/.dotnet

The following frameworks were found:
  8.0.1 at [/home/user/.dotnet/shared/Microsoft.NETCore.App]

Learn more:
https://aka.ms/dotnet/app-launch-failed

To install missing framework, download:
https://aka.ms/dotnet-core-applaunch?framework=Microsoft.NETCore.App&framework_version=6.0.0&arch=arm64&rid=linux-arm64&os=debian.12

But the advice from searching that problem was to install the missing version. Also the output says exactly that "To install missing framework, download [...]", which misled me.

On reading documentation I thought to remove version 6 and try DOTNET_ROLL_FORWARD=Major list-api. Everything now works as expected. If it should have worked with version 6, maybe that is an error in the SDK?

Documentation on DOTNET_ROLL_FORWARD: https://learn.microsoft.com/en-us/dotnet/core/tools/dotnet-environment-variables#dotnet_roll_forward

I think I saw a document somewhere that app/tool authors can override the default version roll forward logic.

Thank you for this useful tool! Closing since I got it working.

smdn commented 9 months ago

Thanks for digging into the issue and how to resolve it! And I'm glad to see that the problem has been resolved.

I will continue to work on updating the list-api. If I release it, I will let you know here in the comments.

smdn commented 9 months ago

list-api version 1.3.0 is now available, including recent changes. https://github.com/smdn/Smdn.Reflection.ReverseGenerating/releases/tag/releases%2FSmdn.Reflection.ReverseGenerating.ListApi-1.3.0

This version does not include the workaround for this issue, but it includes some improvements related to assembly loading. Please update it if you would like.

By the way, the target framework 'net8.0' has been added from version 1.3.0. If you need to specify the target framework when updating, use the following command to update.

dotnet tool update --framework net6.0 -g Smdn.Reflection.ReverseGenerating.ListApi dotnet tool update --framework net8.0 -g Smdn.Reflection.ReverseGenerating.ListApi

Once again, thanks for reporting the issue!