Closed francotiveron closed 1 year ago
@francotiveron If you try to build the solution, does it still build with the build gesture? Or does that fail too?
@jasonmalinowski It fails with the same error
Then can you do a command line build and pass the /bl switch? That'll produce a binary log file that we can investigate further with. That file can potentially contain private info, so if you want to email me the file directly my email is on my GitHub profile.
msbuild.binlog (481K) sent to your email
Thanks for sending over the binlog. I see that The NinjaTrader.Core from "C:\Program Files\NinjaTrader 8\bin\NinjaTrader.Core.dll" " is being passed to the compiler. But when P1 was being built the underlying assembly resolution logic did say this:
Considered "C:\Temp\Ninja\Test1\Solution1\P2\bin\Debug\net4.8\NinjaTrader.Core.dll",
but its name "NinjaTrader.Core, Version=8.1.2.0, Culture=neutral, PublicKeyToken=null"
didn't match the expected name "NinjaTrader.Core, Version=8.1.2.0, Culture=neutral, PublicKeyToken=0907d8af90186095".
This makes me think the problem here is there are two versions of NinjaTrader.Core floating around: one has a strong name (PublicKeyToken = 0907d8af90186095) and one does not have a strong name, and those are considered different per assembly resolution logic. The follow up question then is "where did the two copies come from?" Unfortunately your binlog doesn't show a build for P2 or P3 (just that they're up to date), but if you do a clean and build again, does the problem go away? Otherwise we might need a new binlog where not only passing /bl you are also passing /t:Rebuild to force a rebuild.
(and marking this as Area-Compilers rather than Area-IDE since it's definitely not an IDE problem. I suspect this is still a build going funny in some way though and not really a compiler issue either.)
Solution Cleaned msbuild /bl /t:Rebuild log file sent
@dotnet/fsharp-team-msft do you think you can take a look at this as a next step of investigation? It seems that the F# assembly is being recorded as having a reference to an assembly with a public key token, but the build shows it's getting the same input as the C# compiler. I don't see where the public key token for the reference is coming from anywhere else, so it almost feels like this is an F# compiler bug (although I don't know how!)
The invocation of Fsc is this. The FusionName that's tacked on by RAR implies it's PublicKeyToken=null:
But RAR when it's processing the consuming C# project:
So how do I fix it? Or is it a bug?
@francotiveron Don't know yet, but at this point I'm not sure it's a Roslyn issue. We're getting the F# team to take a look.
@dotnet/fsharp-team-msft do you think you can take a look at this as a next step of investigation? It seems that the F# assembly is being recorded as having a reference to an assembly with a public key token, but the build shows it's getting the same input as the C# compiler. I don't see where the public key token for the reference is coming from anywhere else, so it almost feels like this is an F# compiler bug (although I don't know how!)
I'm not entirely sure how can that happen, but 17.7 ships with quite old compiler, we've had a bunch of fixes since.
Does the same issue show with 17.8/17.9?
@jasonmalinowski also - isn't Fsc task in binlog showing what was passed to it as reference when msbuild is invoking it?
@jasonmalinowski also - isn't Fsc task in binlog showing what was passed to it as reference when msbuild is invoking it?
Yeah that's why I'm confused. I see the reference going in that looks like it doesn't have a public key token. But later when we try processing the output of it, it seems like output did have one. And to be very clear, I can't explain how the compiler could make a mistake like that, but I'm not seeing any alternative either. :-/
Actually @francotiveron that might be one more thing to ask: can you upload the P2.dll file that gets built? You can upload that here, no reason to email.
@francotiveron , could I get the binary log file to look at.
I've shared the logs internally.
@francotiveron , So ... from the error message it looks as if the F# compiler is writing out the reference to NinjaTrader.Core incorrectly. When compiling P2.dll
C:\Temp\Ninja\Test1\Solution1\P1\Class1.cs(10,23): error CS0012:
The type 'IndicatorBase' is defined in an assembly that is not referenced. You must add a reference to assembly
'NinjaTrader.Core, Version=8.1.2.0, Culture=neutral, PublicKeyToken=0907d8af90186095'.
[C:\Temp\Ninja\Test1\Solution1\P1\P1.csproj]
Would it be possible for you to provide p1.dll and p2.dll so that I can look at them locally, You can put them in a zip and add them as an attachment to this conversation.
It would also be very useful to be able to examine ninjatrader.core.dll, but their terms of use specifically prohibit decompilation
So perhaps if you could run these commands on the assembly and provide the output:
sn -vf NinjaTrader.Core.dll
sn -Tp NinjaTrader.Core.dll
and:
dumpbin /clrheader NinjaTrader.Core.dll
dumpbin /headers NinjaTrader.Core.dll
It's interesting to me, that the particular public key, has appeared in several issues on the internet, so I harbor the suspicion that whatever we are doing, is a bit more subtle than I outlined above.
thanks
Kevin Ransom.
PS C:\Program Files\NinjaTrader 8\bin> sn -vf NinjaTrader.Core.dll
Microsoft (R) .NET Framework Strong Name Utility Version 4.0.30319.0
Copyright (c) Microsoft Corporation. All rights reserved.
NinjaTrader.Core.dll does not represent a strongly named assembly
PS C:\Program Files\NinjaTrader 8\bin> sn -Tp NinjaTrader.Core.dll
Microsoft (R) .NET Framework Strong Name Utility Version 4.0.30319.0
Copyright (c) Microsoft Corporation. All rights reserved.
NinjaTrader.Core.dll does not represent a strongly named assembly
PS C:\Program Files\NinjaTrader 8\bin> dumpbin /clrheader NinjaTrader.Core.dll
Microsoft (R) COFF/PE Dumper Version 14.38.33130.0
Copyright (C) Microsoft Corporation. All rights reserved.
Dump of file NinjaTrader.Core.dll
File Type: DLL
clr Header:
48 cb
2.05 runtime version
54970 [ 252AE0] RVA [size] of MetaData Directory
1 flags
IL Only
0 entry point token
4BB98C [ 48FE0] RVA [size] of Resources Directory
0 [ 0] RVA [size] of StrongNameSignature Directory
0 [ 0] RVA [size] of CodeManagerTable Directory
0 [ 0] RVA [size] of VTableFixups Directory
0 [ 0] RVA [size] of ExportAddressTableJumps Directory
0 [ 0] RVA [size] of ManagedNativeHeader Directory
Summary
2000 .reloc
2000 .rsrc
504000 .text
PS C:\Program Files\NinjaTrader 8\bin> dumpbin /headers NinjaTrader.Core.dll
Microsoft (R) COFF/PE Dumper Version 14.38.33130.0
Copyright (C) Microsoft Corporation. All rights reserved.
Dump of file NinjaTrader.Core.dll
PE signature found
File Type: DLL
FILE HEADER VALUES
8664 machine (x64)
3 number of sections
6536447A time date stamp Mon Oct 23 21:01:30 2023
0 file pointer to symbol table
0 number of symbols
F0 size of optional header
2022 characteristics
Executable
Application can handle large (>2GB) addresses
DLL
OPTIONAL HEADER VALUES
20B magic # (PE32+)
48.00 linker version
502C00 size of code
503000 size of initialized data
0 size of uninitialized data
5049DA entry point (00000000009049DA)
2000 base of code
400000 image base (0000000000400000 to 0000000000909FFF)
2000 section alignment
200 file alignment
4.00 operating system version
0.00 image version
6.00 subsystem version
0 Win32 version
50A000 size of image
200 size of headers
0 checksum
3 subsystem (Windows CUI)
8560 DLL characteristics
High Entropy Virtual Addresses
Dynamic base
NX compatible
No structured exception handler
Terminal Server Aware
100000 size of stack reserve
1000 size of stack commit
100000 size of heap reserve
1000 size of heap commit
0 loader flags
10 number of directories
0 [ 0] RVA [size] of Export Directory
50496C [ 57] RVA [size] of Import Directory
506000 [ 3E8] RVA [size] of Resource Directory
0 [ 0] RVA [size] of Exception Directory
0 [ 0] RVA [size] of Certificates Directory
508000 [ C] RVA [size] of Base Relocation Directory
5049EC [ 1C] RVA [size] of Debug Directory
0 [ 0] RVA [size] of Architecture Directory
0 [ 0] RVA [size] of Global Pointer Directory
0 [ 0] RVA [size] of Thread Storage Directory
0 [ 0] RVA [size] of Load Configuration Directory
0 [ 0] RVA [size] of Bound Import Directory
2000 [ 10] RVA [size] of Import Address Table Directory
0 [ 0] RVA [size] of Delay Import Directory
2010 [ 48] RVA [size] of COM Descriptor Directory
0 [ 0] RVA [size] of Reserved Directory
SECTION HEADER #1
.text name
502A98 virtual size
2000 virtual address (0000000000402000 to 0000000000904A97)
502C00 size of raw data
200 file pointer to raw data (00000200 to 00502DFF)
0 file pointer to relocation table
0 file pointer to line numbers
0 number of relocations
0 number of line numbers
60000020 flags
Code
Execute Read
Debug Directories
Time Type Size RVA Pointer
-------- ------- -------- -------- --------
91AF9D82 cv 8F 00504A08 502C08 Format: RSDS, {75486FF0-67CD-4456-B436-4A5B86D8DFA7}, 1, C:\Users\Administrator\Documents\JenkinsAgentLocal\workspace\NT8\NinjaTrader.Core\obj\x64\Release\NinjaTrader.Core.pdb
SECTION HEADER #2
.rsrc name
3E8 virtual size
506000 virtual address (0000000000906000 to 00000000009063E7)
400 size of raw data
502E00 file pointer to raw data (00502E00 to 005031FF)
0 file pointer to relocation table
0 file pointer to line numbers
0 number of relocations
0 number of line numbers
40000040 flags
Initialized Data
Read Only
SECTION HEADER #3
.reloc name
C virtual size
508000 virtual address (0000000000908000 to 000000000090800B)
200 size of raw data
503200 file pointer to raw data (00503200 to 005033FF)
0 file pointer to relocation table
0 file pointer to line numbers
0 number of relocations
0 number of line numbers
42000040 flags
Initialized Data
Discardable
Read Only
Summary
2000 .reloc
2000 .rsrc
504000 .text
P2.dll attached. P1.dll is not generated because of the error we are dealing with P2.zip
Slight edit ....
I'm not entirely sure how can that happen, but 17.7 ships with quite old compiler, we've had a bunch of fixes since.
Does the same issue show with 17.8/17.9?
@vzarytovskii now using VS 17.8,2 - error persists
@francotiveron
Okay, I think I understand what is going on here.
I believe the ninjatrader.core.dll has been obfuscated, when we import it we think it is a strong signed assembly, even though the dumpbin shows that it is clearly not. I'm guessing the obfuscator zero's out the strong signed flag, and the public key record as well as the KeyFile attribute filename. So the resulting assembly doesn't look quite the same as if it was originally unsigned.
When we write out the reference because we thought it was signed when we imported it, we write the assembly reference into P2.dll with the public key set to the bytes we thought represented the public key on import.
I don't think this is a regression, it's probably worked this way for a long time. l will try to work on a fix tomorrow, it shouldn't be too tricky.
Thanks for persevering with the issue
Kevin
/cc @vzarytovskii @jasonmalinowski @francotiveron
How do I install this version without waiting for an update of Visual Studio?
@francotiveron, IMO the easiest for you ("works on your machine") only:
git clone
this repositorybuild
from a command prompt in the repository location (it will most likely be green, matter of few minutes)visualfsharp.sln
, and run the VisualFSharpDebug target without debugger attached (ctrl+f5 in my case, not sure if it is default)
After this, if it is all fine:
build fails
FSC : error FS0225: Source file 'C:\Root\Work\github\dotnet\fsharp\artifacts\obj\FSharp.Compiler.Interactive.Settings\D
ebug\netstandard2.0\FSInteractiveSettings.fsi' could not be found [C:\Root\Work\github\dotnet\fsharp\src\FSharp.Compile
r.Interactive.Settings\FSharp.Compiler.Interactive.Settings.fsproj::TargetFramework=netstandard2.0]
FSC : error FS0225: Source file 'C:\Root\Work\github\dotnet\fsharp\artifacts\obj\FSharp.Build\Debug\netstandard2.0\FSBu
ild.fsi' could not be found [C:\Root\Work\github\dotnet\fsharp\src\FSharp.Build\FSharp.Build.fsproj]
FSC : error FS0225: Source file 'C:\Root\Work\github\dotnet\fsharp\artifacts\obj\FSharp.DependencyManager.Nuget\Debug\n
etstandard2.0\FSDependencyManager.fsi' could not be found [C:\Root\Work\github\dotnet\fsharp\src\FSharp.DependencyManag
er.Nuget\FSharp.DependencyManager.Nuget.fsproj::TargetFramework=netstandard2.0]
EmptyAssembly -> C:\Root\Work\github\dotnet\fsharp\artifacts\bin\EmptyAssembly\Debug\net472\EmptyAssembly.dll
TaskPerf -> C:\Root\Work\github\dotnet\fsharp\artifacts\bin\TaskPerf\Debug\net8.0\TaskPerf.dll
DummyProviderForLanguageServiceTesting -> C:\Root\Work\github\dotnet\fsharp\artifacts\bin\DummyProviderForLanguageSer
viceTesting\Debug\net472\DummyProviderForLanguageServiceTesting.dll
TestTP -> C:\Root\Work\github\dotnet\fsharp\artifacts\bin\TestTP\Debug\netstandard2.0\TestTP.dll
Build FAILED.
FSC : error FS0225: Source file 'C:\Root\Work\github\dotnet\fsharp\artifacts\obj\FSharp.Compiler.Interactive.Settings\D
ebug\netstandard2.0\FSInteractiveSettings.fsi' could not be found [C:\Root\Work\github\dotnet\fsharp\src\FSharp.Compile
r.Interactive.Settings\FSharp.Compiler.Interactive.Settings.fsproj::TargetFramework=netstandard2.0]
FSC : error FS0225: Source file 'C:\Root\Work\github\dotnet\fsharp\artifacts\obj\FSharp.Build\Debug\netstandard2.0\FSBu
ild.fsi' could not be found [C:\Root\Work\github\dotnet\fsharp\src\FSharp.Build\FSharp.Build.fsproj]
FSC : error FS0225: Source file 'C:\Root\Work\github\dotnet\fsharp\artifacts\obj\FSharp.DependencyManager.Nuget\Debug\n
etstandard2.0\FSDependencyManager.fsi' could not be found [C:\Root\Work\github\dotnet\fsharp\src\FSharp.DependencyManag
er.Nuget\FSharp.DependencyManager.Nuget.fsproj::TargetFramework=netstandard2.0]
0 Warning(s)
3 Error(s)
Time Elapsed 00:02:19.01
Build failed with exit code 1. Check errors above.
@francotiveron, sorry you are facing issues, I just made a clean build and it comes green:
Could you share which VS version you have installed?
I ran the command from VS 2022 command prompt and I'm having 17.8 and 17.9 (preview) installed on the machine, just as a disclaimer, this repository generally needs pretty recent version of VS 2022, I sometimes have to update but don't remember if the error is explicit.
Your version in this ticket seems rather old, could you try updating your VS 2022 installation and try again, from a VS 2022 developer command prompt so we are on same page?
@smoothdeveloper I am using VS 17.8.2, the latest non-preview. I have updated the initial post to reflect this.
I have reset the repo and now the build succeeds. Then have Ctrl-F5 VisualFSharpDebug. It starts another instance of VS. What should I do now?
You should open the solution that were causing the issue to see if it reproduces, hopefully it got solved.
Unfortunately it still gets the same error
@francotiveron, taking from the initial bug, it would be interesting to know what happens if your main program (with P1.Class1) was implemented in F#.
Do you see the same error in this case?
P1Fs.fsproj
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net48</TargetFramework>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<WarnOn>3390;$(WarnOn)</WarnOn>
</PropertyGroup>
<ItemGroup>
<Reference Include="NinjaTrader.Core">
<HintPath>..\..\..\..\..\Program Files\NinjaTrader 8\bin\NinjaTrader.Core.dll</HintPath>
</Reference>
<Reference Include="PresentationCore" />
<Reference Include="PresentationFramework" />
<Reference Include="WindowsBase" />
</ItemGroup>
<ItemGroup>
<Compile Include="Class1.fs" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\P2\P2.fsproj" />
<ProjectReference Include="..\P3\P3.csproj" />
</ItemGroup>
</Project>
Note: I had to add references to PresentationCore, PresentationFramework, WindowsBase not required by P1.csproj.
Class1.fs
namespace P1Fs
open NinjaTrader.NinjaScript
type internal Class1() as this =
inherit IndicatorBase()
let _c2 = P2.Class2(this)
let _c3 = P3.Class3(this)
P1fs.fsproj compiles without errors. Note: of course this could have been worked even before the fix.
If the code runs fine, I think we can mark this as obscure inconsistency between how F# and C# compilers handle case of strong named assemblies with 0 length signature.
It seems the F# team adjusted one thing to make it behave more "correctly" (if signature is zero length despite saying strong named, consider the assembly as not strong named IIUC), but I don't think it is an issue with the compiler per say, and more related to that particular assembly.
You may see if the C# compiler team is willing to make similar adjustment (which may or may not look the same as here), in case it helps your endeavours with consuming this dependency.
Can you help with transferring this issue to the right team/repo, please?
@francotiveron, I think it should be https://github.com/dotnet/roslyn, best is to open a new ticket, and try to provide the sample in a zip for easier check on their end.
You can refer to this ticket as the diagnostic the team made you go through contains important info.
Note: of course this could have been worked even before the fix.
Deploying vsix won't fix it. Compiler is used from sdk. Two FS projects just happen to read the key the same way.
It's also has nothing to do with Roslyn. They read key correctly.
@vzarytovskii I am confused. What do you recommend I do to get to issue fixed?
@vzarytovskii I am confused. What do you recommend I do to get to issue fixed?
SDK with the fix should be used. So either wait for release or use nightly one from dotnet/installer repo. I'm not entirely sure though it was inserted in nightlies already (update: it wasn't: https://github.com/dotnet/fsharp/pull/16365)
@vzarytovskii I have seen that https://github.com/dotnet/fsharp/pull/16365 has been merged. I have downloaded the latest version of Visual studio Preview (17.9.0 Preview 2.0), but I still get the error. Is this expected?
@vzarytovskii I have seen that https://github.com/dotnet/fsharp/pull/16365 has been merged. I have downloaded the latest version of Visual studio Preview (17.9.0 Preview 2.0), but I still get the error. Is this expected?
I think so. Fix was merged after preview 2 was released
Version Used:
Microsoft Visual Studio Professional 2022 Version 17.8.2 VisualStudio.17.Release/17.8.2+34322.80 Microsoft .NET Framework Version 4.8.09037
Installed Version: Professional
ADL Tools Service Provider 1.0 This package contains services used by Data Lake tools
ASA Service Provider 1.0
ASP.NET and Web Tools 17.8.358.6298 ASP.NET and Web Tools
Azure App Service Tools v3.0.0 17.8.358.6298 Azure App Service Tools v3.0.0
Azure Data Lake Tools for Visual Studio 2.6.5000.0 Microsoft Azure Data Lake Tools for Visual Studio
Azure Functions and Web Jobs Tools 17.8.358.6298 Azure Functions and Web Jobs Tools
Azure Stream Analytics Tools for Visual Studio 2.6.5000.0 Microsoft Azure Stream Analytics Tools for Visual Studio
C# Tools 4.8.0-7.23572.1+7b75981cf3bd520b86ec4ed00ec156c8bc48e4eb C# components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.
Common Azure Tools 1.10 Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.
Cookiecutter 17.0.23262.1 Provides tools for finding, instantiating and customizing templates in cookiecutter format.
Extensibility Message Bus 1.4.39 (main@e8108eb) Provides common messaging-based MEF services for loosely coupled Visual Studio extension components communication and integration.
File Differ 3.0.49 The easiest way to diff two files directly in Solution Explorer
ILSpy.AddIn 1.0 Integration of the ILSpy Decompiler into Visual Studio.
Microsoft Azure Hive Query Language Service 2.6.5000.0 Language service for Hive query
Microsoft Azure Stream Analytics Language Service 2.6.5000.0 Language service for Azure Stream Analytics
Microsoft Azure Tools for Visual Studio 2.9 Support for Azure Cloud Services projects
Microsoft JVM Debugger 1.0 Provides support for connecting the Visual Studio debugger to JDWP compatible Java Virtual Machines
Mono Debugging for Visual Studio 17.8.17 (957fbed) Support for debugging Mono processes with Visual Studio.
NuGet Package Manager 6.8.0 NuGet Package Manager in Visual Studio. For more information about NuGet, visit https://docs.nuget.org/
Office Developer Tools for Visual Studio 17.6.34024.01 Microsoft Office Developer Tools for Visual Studio
Python - Django support 17.0.23262.1 Provides templates and integration for the Django web framework.
Python - Profiling support 17.0.23262.1 Profiling support for Python projects.
Python with Pylance 17.0.23262.1 Provides IntelliSense, projects, templates, debugging, interactive windows, and other support for Python developers.
Razor (ASP.NET Core) 17.8.3.2353105+f0a326f3e15b2370371693a76fd379635387c74d Provides languages services for ASP.NET Core Razor.
SQL Server Data Tools 17.8.119.0 Microsoft SQL Server Data Tools
Text Macros for Visual Studio 2012-2022 1.18 Brings back text-editing macro support in Visual Studio 2012-2022
ToolWindowHostedEditor 1.0 Hosting json editor into a tool window
TypeScript Tools 17.0.20920.2001 TypeScript Tools for Microsoft Visual Studio
Visual Basic Tools 4.8.0-7.23572.1+7b75981cf3bd520b86ec4ed00ec156c8bc48e4eb Visual Basic components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.
Visual F# Tools 17.8.0-beta.23475.2+10f956e631a1efc0f7f5e49c626c494cd32b1f50 Microsoft Visual F# Tools
Visual Studio IntelliCode 2.2 AI-assisted development for Visual Studio.
VisualStudio.DeviceLog 1.0 Information about my package
VisualStudio.Mac 1.0 Mac Extension for Visual Studio
VSPackage Extension 1.0 VSPackage Visual Studio Extension Detailed Info
Workflow Manager Tools 1.0 1.0 This package contains the necessary Visual Studio integration components for Workflow Manager.
Xamarin 17.8.0.155 (d17-8@7042beb) Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android.
Xamarin Designer 17.8.3.6 (remotes/origin/d17-8@eccf46a291) Visual Studio extension to enable Xamarin Designer tools in Visual Studio.
Xamarin Templates 17.8.16 (830b56a) Templates for building iOS, Android, and Windows apps with Xamarin and Xamarin.Forms.
Xamarin.Android SDK 13.2.2.0 (d17-5/45b0e14) Xamarin.Android Reference Assemblies and MSBuild support. Mono: d9a6e87 Java.Interop: xamarin/java.interop/d17-5@149d70fe SQLite: xamarin/sqlite/3.40.1@68c69d8 Xamarin.Android Tools: xamarin/xamarin-android-tools/d17-5@ca1552d
Xamarin.iOS and Xamarin.Mac SDK 16.4.0.23 (9defd91b3) Xamarin.iOS and Xamarin.Mac Reference Assemblies and MSBuild support. Steps to Reproduce:
Create solution with 3 projects
P2.fsproj
P3.csproj
P1.csproj
Class2.fs
Class3.cs
Class1.cs
Note:
Expected Behavior: Code should compile without errors
Actual Behavior: Visual studio complains on the instantiation of Class2 in Class1, but not on Class3; the error is