Open andresparrab opened 3 years ago
Hi !
The SourcetrailDotnetIndexer is actually 2 different programs:
SourcetrailDotnetIndexer.exe
for the "classic" .NET Framework (e.g. 2.x up to 4.x)
SourcetrailDotnetCoreIndexer.exe
for the "new" .net core and .net 5+
You have to use the correct one depending on the assembly you want to index.
You tried to index a .net 5.0 assembly, so you have to use SourcetrailDotnetCoreIndexer.exe
The next step is to find the correct location for the framework assemblies.
Open a command prompt or a PowerShell and type dotnet --list-runtimes
This lists all installed .net core framework locations.
Beside some 2.x and 3.x folders, the output on my machine included the following:
Microsoft.AspNetCore.App 5.0.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Use this folder for everything aspnet-related
Microsoft.NETCore.App 5.0.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Use this folder for console-apps
Microsoft.WindowsDesktop.App 5.0.7 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Use this folder for Windows Desktop-Apps
Since you mentioned the Weatherforecast API, i assume you have created an aspnet webapi project.
That means, you have to specify the folder for the Microsoft.AspNetCore.App
with the -s
parameter
To demonstrate this, I also quickly created a new aspnet 5.0 webapi project and build it. My command line looked like this in PowerShell:
.\SourcetrailDotnetCoreIndexer.exe -i C:\Users\uncle\source\repos\ExampleWebApi\ExampleWebApi\bin\Debug\net5.0\ExampleWebApi.dll -o . -s "C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\5.0.7"
Note that i have created the sourcetrail database in the current directory by specifying a dot (.) for the -o
parameter.
The output from the command:
Indexing assembly C:\Users\uncle\source\repos\ExampleWebApi\ExampleWebApi\bin\Debug\net5.0\ExampleWebApi.dll
Loaded: C:\Users\uncle\source\repos\ExampleWebApi\ExampleWebApi\bin\Debug\net5.0\ExampleWebApi.pdb Collecting types... Load: C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\5.0.7\Microsoft.AspNetCore.Mvc.Core.dll Load: C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\5.0.7\Microsoft.Extensions.Hosting.Abstractions.dll Load: C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\5.0.7\Microsoft.Extensions.Configuration.Abstractions.dll Load: C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\5.0.7\Microsoft.Extensions.DependencyInjection.Abstractions.dll Load: C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\5.0.7\Microsoft.AspNetCore.Http.Abstractions.dll Load: C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\5.0.7\Microsoft.AspNetCore.Hosting.Abstractions.dll Load: C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\5.0.7\Microsoft.AspNetCore.Mvc.Abstractions.dll Load: C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\5.0.7\Microsoft.Extensions.Logging.Abstractions.dll
Collected 4 types
Parsing IL... (18 methods)
Load: C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\5.0.7\Microsoft.Extensions.Hosting.dll Load: C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\5.0.7\Microsoft.AspNetCore.dll Load: C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\5.0.7\Microsoft.AspNetCore.Mvc.dll Load: C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\5.0.7\Microsoft.AspNetCore.Diagnostics.dll Load: C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\5.0.7\Microsoft.AspNetCore.Routing.dll Load: C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\5.0.7\Microsoft.AspNetCore.Authorization.Policy.dll
Sourcetrail database has been generated at .\ExampleWebApi.srctrldb Time taken: 00:00:00.2823453
Beside the mentioned file ExampleWebApi.srctrldb
, there is also a file created called ExampleWebApi.srctrlprj
in the same folder.
This is the file you load in Sourcetrail.
Please let me know if this resolves your issue.
@packdat Hi! Thank you for your answer. It did work better with SourcetrailDotnetCoreIndexer. in the way that the System.Runtime.InteropServices.COMException (0x806D000C): Exception from HRESULT: 0x806D000C is gone.
and then i specified the assembly for dotnet for aspnet with the dotnet --list-runtimes Microsoft.AspNetCore.App 5.0.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Then I run the command like so:
.\SourcetrailDotnetCoreIndexer.exe -i C:\Users\andres.parra\source\repos\myapp1\bin\Debug\net5.0\myapp1.dll -o C:\Users\andres.parra\source\repos\MyIndex -s "C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\5.0.7\"
and the output i get is :
Indexing assembly C:\Users\andres.parra\source\repos\myapp1\bin\Debug\net5.0\myapp1.dll
Loaded: C:\Users\andres.parra\source\repos\myapp1\bin\Debug\net5.0\myapp1.pdb Collecting types... Unable to resolve assembly Microsoft.AspNetCore.Mvc.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60. Hint: Specify additional assembly-locations using the -s switch.
An exception occurred: System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types. Could not load file or assembly 'Microsoft.AspNetCore.Mvc.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified. at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module) at System.Reflection.RuntimeModule.GetTypes() at System.Reflection.Assembly.GetTypes() at SourcetrailDotnetIndexer.SourcetrailDotnetIndexer.Index(String outputFileName) in C:\Users\andres.parra\source\repos\SourcetrailDotnetIndexer\SourcetrailDotnetIndexer\SourcetrailDotnetIndexer.cs:line 50 at SourcetrailDotnetIndexer.Program.Main(String[] args) in C:\Users\andres.parra\source\repos\SourcetrailDotnetIndexer\SourcetrailDotnetCoreIndexer\Program.cs:line 51 System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.AspNetCore.Mvc.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified. File name: 'Microsoft.AspNetCore.Mvc.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'
And when i try on my DatinApi app that has EntityFramwork:
.\SourcetrailDotnetCoreIndexer.exe -i C:\Users\andres.parra\source\repos\CSharp\DatingApp\API\bin\Debug\net5.0\API.dll -o C:\Users\andres.parra\source\repos\MyIndex -s "C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\5.0.7\"
I get :
Indexing assembly C:\Users\andres.parra\source\repos\CSharp\DatingApp\API\bin\Debug\net5.0\API.dll
Loaded: C:\Users\andres.parra\source\repos\CSharp\DatingApp\API\bin\Debug\net5.0\API.pdb Collecting types... Load: C:\Users\andres.parra\source\repos\CSharp\DatingApp\API\bin\Debug\net5.0\Microsoft.EntityFrameworkCore.dll Unable to resolve assembly Microsoft.AspNetCore.Mvc.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60. Hint: Specify additional assembly-locations using the -s switch.
An exception occurred: System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types. Could not load file or assembly 'Microsoft.AspNetCore.Mvc.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified. at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module) at System.Reflection.RuntimeModule.GetTypes() at System.Reflection.Assembly.GetTypes() at SourcetrailDotnetIndexer.SourcetrailDotnetIndexer.Index(String outputFileName) in C:\Users\andres.parra\source\repos\SourcetrailDotnetIndexer\SourcetrailDotnetIndexer\SourcetrailDotnetIndexer.cs:line 50 at SourcetrailDotnetIndexer.Program.Main(String[] args) in C:\Users\andres.parra\source\repos\SourcetrailDotnetIndexer\SourcetrailDotnetCoreIndexer\Program.cs:line 51 System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.AspNetCore.Mvc.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified. File name: 'Microsoft.AspNetCore.Mvc.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60' PS C:\Users\andres.parra\source\repos\SourcetrailDotnetIndexer\SourcetrailDotnetCoreIndexer\bin\Debug\net5.0>
it seems this is the problem : Could not load file or assembly 'Microsoft.AspNetCore.Mvc.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified. and its source is probably: System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.AspNetCore.Mvc.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified.
I was wondering if i have to specify all the nuget and extensions and plugins the every application has?
Becasue the goal solution that i want to get working in sourcetrial is actually a big project that has consosle application, apis, frontend, with entetiesframworks, and lot of other things, eventshandlers, service commands, and more, Is like lets say Amazon kind of think, that handles orders, customers, databases, payments, invoices, etc. and span over several solutions, and each solution has around 10 projects, maybe solution one take care of the order a customer make, then it jump over to the other solution that take care of the payments, then continue to another solution that continua with logistics, and so forth. There are probably a couple of thousand of code, that i want to understand, And i think that with help of SourcetrailDotnetIndxer i can do exactly that without read ever singe line of code :)
I just have to get this working first with the default weather forecast api first and like the datinApp with EntityFramwork,
And thanks again for your help
Here is the screenshot when debugging also, maybe it helps
Please remove the trailing backslash from the search-path.
e.g. instead of
-s "C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\5.0.7\"
use
-s "C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\5.0.7"
(the combination \"
actually escapes the double-quote, resulting in an invalid path)
Depending on where your assemblies are located, you may have to specify multiple search paths, e.g
-s path1 -s path2 -s path3
...
But this may still not be what you are looking for... Reading your description, it seems you're expecting once you have indexed all your assemblies, you're able to easily follow the code-paths through all your assemblies. I'm in doubt that this will work as you expect. SourcetrailDotnetIndexer creates an index for a SINGLE assembly, that means inside of Sourcetrail you're able to follow the code-paths INSIDE this assembly, but not outside of it. Maybe there is a way to combine several index-databases in Sourcetrail, but I'm not proficient enough in using Sourcetrail to tell you how to do that. (maybe the guys maintaining Sourcetrail can help you with that)
In the meantime I'm working on an enhancement for this kind of use-case, allowing the indexer to follow types outside the main assembly if they match some namespace-pattern.
This would allow the indexer to create a single database for multiple assemblies.
What is the namespace-convention used in your application ?
Does it follow some common pattern like MyCompany.SomeComponent.etc
?
Then this should be relatively easy; e.g. allowing the indexer to follow all types whose namespace start with 'MyCompany' would index all of your assemblies into one single database. (IF and only if they are referenced directly or indirectly from the main assembly; that is, the assembly you specify with the -i
switch)
@packdat Wohoo it works perfectly now. I feel so stupid for missing the "\" at the end. but Im happy that you could explain this, i been sitting several hour trying to read line by line to see what i do wrong.
No, the Mycomapny is not the common name throughout the application, all application have their own namespace. Like Frontend, has namespace letsay webshop and the hub that calls other solution has namespace hub.common, and so on. see example below:
But maybe to make this more clear i will explain one of the flows here
I have a folder that contain a solution GDPR, and inside it has 2 projects GDPR |__Gdpr.Common |--Services |... other folders |_GdprApi
Now GdprAi contains an controller GdprController: [Route("api/depersonalize")] with the function: Depersonalize(some object) This calls DepersonalizeData that is in the project Gdpr.common.Services And that calls each client of the application by calling to another Solution hub.common.. that acts like a hub for call other Solution
I try to ilustrate. In Gdpr.common.Services function DepersonalizeData -- calls the function DeperzonalizeCustomer in each client by calling hub.common (another solution) ...and this calls |---- client in the solution customerApi.Controller (DeperzonalizeCustomer() ) |----client in the solution orderApi.Controller (DeperzonalizeCustomer()) |----mailServices in the solution mailservices (DeperzonalizeCustomer())
So this is my dilema.... I dont know if i can do it with this applicationation, that would be awsome. or maybe you can recommend another application to do this. i been readin the code for 3 months now and i understand maybe 10% if i could vizualise this like the sourcode way or another way, like 3d or something that would make me more efficient.
Or maybe this could be the thing you are working on, like ----> https://www.youtube.com/watch?v=LUiy1M9hUKU is CODE park, but i dont know if its abandon project, can find to download. or https://www.youtube.com/watch?v=F3yFixY9sVk&t ---> How to learn large codebase really fast (and find a specific function relations from 2 million LoC)
I dont know im looking for options, so far the one most promising is Sourcetrail. And thank you for your patience
by the way how can i get assemblies from nuget packages for example:
i try to import .\SourcetrailDotnetCoreIndexer.exe -i C:\Git\customer\GdprApi\Gdpr.Common\bin\Debug\net5.0\Gdpr.Common.dll -o C:\Users\andres.parra\source\repos\MyIndex\ -s "C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\5.0.7" -s "C:\Users\andres.parra\.nuget\packages\microsoft.azure.cosmos.table\1.0.8\lib\netstandard2.0\Microsoft.Azure.Cosmos.Table.dll"
System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Azure.Cosmos.Table, Version=1.0.8.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'. The system cannot find the file specified.
and i get the same error for the custom nuget hub.common
In Gdpr.common.Services function DepersonalizeData -- calls the function DeperzonalizeCustomer in each client by calling hub.common (another solution) ...and this calls |---- client in the solution customerApi.Controller (DeperzonalizeCustomer() ) |----client in the solution orderApi.Controller (DeperzonalizeCustomer()) |----mailServices in the solution mailservices (DeperzonalizeCustomer())
Are these calls "direct" calls to a method or are these Controllers invoked by the means of a WebRequest ? If these are called per WebRequest, then there is a "break" in the flow and the indexer cannot follow that call-path as it does not have any knowledge about which one of your components is actually addressed by the request. On the other hand, if this is a "direct" call to a method in a referenced assembly, then the indexer should be able to follow that. (that would be the behavior I'm currently planning to implement, but i have no idea how this will work out, if at all)
by the way how can i get assemblies from nuget packages for example: i try to import .\SourcetrailDotnetCoreIndexer.exe -i C:\Git\customer\GdprApi\Gdpr.Common\bin\Debug\net5.0\Gdpr.Common.dll -o C:\Users\andres.parra\source\repos\MyIndex\ -s "C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\5.0.7" -s "C:\Users\andres.parra.nuget\packages\microsoft.azure.cosmos.table\1.0.8\lib\netstandard2.0\Microsoft.Azure.Cosmos.Table.dll"
Remove the assembly name from the -s
parameter.
Paths you specify for the -s
parameter should be folder-paths, not file-paths, e.g.
-s "C:\Users\andres.parra\.nuget\packages\microsoft.azure.cosmos.table\1.0.8\lib\netstandard2.0"
I just added functionality that may help you with your project. Please download the Latest release and study the paragraph Building a multi-assembly database from the Readme.
You can use the new command-line switches -fn
and/or -ff
to specify namespaces that should be included in the Sourcetrail-Database.
As mentioned in my previous reply, the output you're looking for is only possible, if the code from foreign assemblies is directly called from other code and not by using a WebRequest or something similar.
Please let me know if this helps.
Hi! I have read the Building a multi-assembly database , an it is awesome, this is a huge improvement of functionality and the the screenshot and explanation super clear. I was able to call assemblies that where directly called from other code in the solution. This is a very good and easy example of an application with multiple Assembly's in the same solution. But how would it work if you have several Solutions and each with multiple assemblys in each solution like your example.
like this: SLN1 (solution1) namespace: example order lib1 lib2 calling commonForAllSolutions.Payment.visa() lib3
SLN2 | namespace: commonForAllSolutions
commonForAllSolutions_custom_nugetpackagename.nupack or .nupkg_ this shared with the other solutions, All other solution can call the functions from other solutions trhough this. example Order can call the Payment.visa() throug here. se above
othername1
librarie
lib7
SLN3 namespace: App frontend ----------->(the main website) api shared
SLN4 nameSpace: payments
etc1. here have visa() function
etcA
etc_B
Is it even possible? And i am immense grateful for your help , This help me more than read every line of code in a major way
It basically does not matter how many solutions you have in your application.
As long as the assemblies (e.g. the output of the projects of your solutions) are referenced from your code, they will end up in the Sourcetrail database.
It is just a matter of specifying the correct search-paths for the assemblies (via the -s
switch) and namespace-names of these assemblies (via the -fn
and /or the -ff
switch).
So you have your Order solution which calls methods from the Common solution which in turn calls the Payment solution.
-i
switch.-s
switch.-fn
/ -ff
switch.(to keep things simple, I'd copy all assemblies into one common path, so you have to specify the -s
switch only once)
As long as all the parameters are correct, all solutions should end up in the generated database.
I just got an idea while writing this.
For your case, it may be far easier if you could "force" an assembly to be included in the database.
Say, you just specify all the assemblies making up your application.
(by their file-paths, the -s
and -fn
/ -ff
switches are no longer needed in this case)
Then regardless if the types in these assemblies are referenced or not, they would still be included.
That would even allow you to identify "dead" code, e.g. classes that are not used by any other class.
This shouldn't be too complicated, I'll come back to you soon.
I added a new command-line switch that should help you.
Download the latest release (0.2.5), read the new section about the easy way and try it out !
With the new -if
switch you can now explicitly specify all assemblies that make up your application.
Let me know how this works out.
Hi, thank your for your changes, im sorry that i write late, now everyone in the company is gone for summer holidays, so I the newbie are workin like crazy, and understad 20 % of what im doing hehe. as soon as i have the time i will try your new functionality. Becasuse i thing is good for both of us, I try and give feedback to you and i get better to understand the huge application and you get better and better functionality, when i get better i want to help develop this too :)
Hello, first i want to say thank you for this app extension to C# I am beginner and try to understand a big application that span over many solutions, and each solution have many project, I been keeping reading the code from one month and get more confuse, I have been trying to get a visual way to understan theproject and i found this sourtraikDotnetindexer.
I have download the lates git from you and compiled in microsoft VS wihout any problems. But when i try to run it for test a simple API, the weatherforcast API that comes when you star new project. with the command:
I get this:
Then I write like this to remove the errors about Unable to resolve assembly System.Runtime, Version=5.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a.
and i still get errors :
_
_ Please help And thank you in advance. Maybe if possible to have like a screenshot tutorial for C# dotnet from downaload to working in SourcetrailCode? I think many people would benefit from this.
Thank you in advance.