search

dotnet / fsharp

The F# compiler, F# core library, F# language service, and F# tooling integration for Visual Studio
https://dotnet.microsoft.com/languages/fsharp
MIT License
3.92k stars 786 forks source link

project with type provider builds in VS but not with "dotnet build" #10323

Open smoothdeveloper opened 4 years ago

smoothdeveloper commented 4 years ago

I'm facing issue with project referencing FSharp.Management not building from dotnet build

fsmgt.zip

When opening the same project in visual studio, it is possible to build.

Expected behavior

Able to build with dotnet build

Actual behavior

FSC : warning FS3005: Referenced assembly 'C:\Users\gauthier.segay.nuget\packages\fsharp.management\0.4.5\lib\net40\FSharp.Management.PowerShell.dll' has assembly level attribute 'Microsoft.FSharp.Core.CompilerServices.TypeProviderAssemblyAttribute' but no public type provider classes were found [C:\tmp\fsmgt\fsmgt.fsproj] FSC : warning FS3005: Referenced assembly 'C:\Users\gauthier.segay.nuget\packages\fsharp.management\0.4.5\lib\net40\FSharp.Management.WMI.dll' has assembly level attribute 'Microsoft.FSharp.Core.CompilerServices.TypeProviderAssemblyAttribute' but no public type provider classes were found [C:\tmp\fsmgt\fsmgt.fsproj] FSC : error FS3049: The type provider 'C:\Users\gauthier.segay.nuget\packages\fsharp.management\0.4.5\lib\net40\FSharp.Management.PowerShell.dll' reported an error: The type provider designer assembly 'C:\Users\gauthier.segay.nuget\packages\fsharp.management\0.4.5\lib\net40\FSharp.Management.PowerShell.dll' could not be loaded from folder 'C:\Users\gauthier.segay.nuget\packages\fsharp.management\0.4.5\lib\net40'. The exception reported was: System.IO.FileNotFoundException - Could not load file or assembly 'System.ServiceModel, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. The system cannot find the file specified. [C:\tmp\fsmgt\fsmgt.fsproj] FSC : error FS3049: The type provider 'C:\Users\gauthier.segay.nuget\packages\fsharp.management\0.4.5\lib\net40\FSharp.Management.WMI.dll' reported an error: The type provider designer assembly 'FSharp.Management.WMI.DesignTime' could not be loaded from folder 'C:\Users\gauthier.segay.nuget\packages\fsharp.management\0.4.5\lib\net40'. The exception reported was: System.IO.FileNotFoundException - Could not load file or assembly 'System.Management, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified. [C:\tmp\fsmgt\fsmgt.fsproj] 2 Warning(s) 2 Error(s)

Related information

smoothdeveloper commented 4 years ago

Similar stuff happening with FSharp.Data.SqlClient, building in VS but not from command line using dotnet sdk compiler.

This boils down to full framework fsc.exe used in VS and dotnet sdk not working with type providers AFAIU:

error FS3033: The type provider 'FSharp.Data.SqlProgrammabilityProvider' reported an error: Could not load type 'System.Data.SqlClient.SqlConnection' from assembly 'System.Data, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'

This kind of a big deal and not sure how to work around the issue.

cartermp commented 4 years ago

As far as I can tell this type provider is still using old-style projects and targeting .NET Framework. I don't think dotnet build would be expected to work.

smoothdeveloper commented 4 years ago

@cartermp this is not the case with FSharp.Data.SqlClient which use new style SDK.

Also, are you saying that dotnet SDK version of compiler cannot compile projects using type providers when the target is .NET framework?

cartermp commented 4 years ago

No, but it's complicated. Old-style projects I would not expect to be buildable by the .NET SDK. New-style is .. it depends. How much messing around with loading different toolsets is being defined in the type provider? A common problem I see is an enormously complex build for type providers that attempt to programmatically load compilers and targets based on certain conditions. That stuff just rarely ever works.

smoothdeveloper commented 4 years ago

@cartermp thanks for giving hint about the underlying issue.

A common problem I see is an enormously complex build for type providers that attempt to programmatically load compilers and targets based on certain conditions. That stuff just rarely ever works.

In case of FSharp.Data.SqlClient, it has two "DesignTime" assemblies:

image

the netstandard one references the types from:

// Detected Target-Framework-Id: .NETStandard,Version=v2.0

// Referenced assemblies (in metadata order):
// FSharp.Core, Version=4.7.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
    // Assembly reference loading information:
    // There were some problems during assembly reference load, see below for more information!
    // Error: Could not find reference: FSharp.Core, Version=4.7.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a

// Microsoft.SqlServer.TransactSql.ScriptDom, Version=15.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91
    // Assembly reference loading information:
    // There were some problems during assembly reference load, see below for more information!
    // Error: Could not find reference: Microsoft.SqlServer.TransactSql.ScriptDom, Version=15.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91

// mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
    // Assembly reference loading information:
    // Info: Success - Loading from: C:\Program Files\dotnet\shared\Microsoft.NETCore.App\2.1.4\mscorlib.dll

// netstandard, Version=2.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
    // Assembly reference loading information:
    // Info: Success - Loading from: C:\Program Files\dotnet\shared\Microsoft.NETCore.App\2.1.4\netstandard.dll

// System.Configuration.ConfigurationManager, Version=4.0.1.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
    // Assembly reference loading information:
    // Info: Success - Loading from: C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App\3.0.0-preview6-27804-01\System.Configuration.ConfigurationManager.dll

// System.Data.SqlClient, Version=4.4.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
    // Assembly reference loading information:
    // Info: Success - Loading from: C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\2.1.4\System.Data.SqlClient.dll

// System.Runtime.Caching, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
    // Assembly reference loading information:
    // Info: Success - Loading from: C:\WINDOWS\Microsoft.NET\assembly\GAC_MSIL\System.Runtime.Caching\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Runtime.Caching.dll
// Detected Target-Framework-Id: .NETFramework,Version=v4.6.1

// Referenced assemblies (in metadata order):
// FSharp.Core, Version=4.4.3.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a (unresolved)
// Microsoft.SqlServer.TransactSql.ScriptDom, Version=15.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91 (unresolved)
// Microsoft.SqlServer.Types, Version=14.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91 (unresolved)
// mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
    // Assembly reference loading information:
    // Info: Success - Found in Assembly List

// System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 (unresolved)
// System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a (unresolved)
// System.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 (unresolved)
// System.Data, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 (unresolved)
// System.Numerics, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 (unresolved)
// System.Runtime.Caching, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
    // Assembly reference loading information:
    // Info: Success - Found in Assembly List

Is there anything which should be done in the implementation of that particular provider which would enable support from dotnet core version of fsc to compile net framework? (In which case we'd close this issue and improve the TP / TP SDK about those problems).

As a work around, is there a way to specify another fsc.exe to be used in those projects causing troubles so building from command line through dotnet build would call into the .NET framework version of the compiler?

edit: https://github.com/MarneeDear/FSharp.Data.SqlClient-dotnet/blob/master/fsc.props seems like a viable workaround.

I'd still like to better understand the whole lifecycle that lead to the issue happening, and get some pointers to the code paths in compiler dealing with this.

I assume the dotnet SDK compiler is loading the netstandard "DesignTime" assembly, but not sure what follows.

isaacabraham commented 4 years ago

@smoothdeveloper I think the best thing to do would be create an empty (new) TP using the latest TP template and see how that one works to diff between it and the SQLClient one. That was going to be my approach for the Azure one, although I never got around to it.

cartermp commented 3 years ago

I think your best bet is to stop trying to support net461. Just move only to .NET Standard 2.0 and simplify everything. The template here shows how to do that: https://github.com/fsprojects/FSharp.TypeProviders.SDK

smoothdeveloper commented 3 years ago

After a quick test removing .net framework target, a .net framework command line project fails to build in VS:

C:\dev\src\github.com\fsprojects\FSharp.Data.SqlClient\src\SqlClient.TestProjects\SqlClient.Tests.NET40\Program.fs(8,21): error FS3033: The type provider 'FSharp.Data.SqlCommandProvider' reported an error: Could not load file or assembly 'System.Configuration.ConfigurationManager, Version=4.0.1.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. Reference assemblies should not be loaded for execution. They can only be loaded in the Reflection-only loader context. (Exception from HRESULT: 0x80131058)

This is despite adding that nuget package to the project, I'm not sure what is missing (in this case, that project is compiling with net48 target despite the name).

Also, the type provider targets net40 runtime, droping the .net framework binaries of the package would drop support for that target.

I guess my main concern is that dotnet core compiler resolves assemblies differently than .net framework even when targetting .net framework, eventually the .net framework compiler won't be updated anymore.

In this context, it may be worth the effort to make the dotnet core compiler behave the same when targetting .net framework.

cartermp commented 3 years ago

In this context, it may be worth the effort to make the dotnet core compiler behave the same when targetting .net framework.

Nooooo, that will absolutely never happen. @KevinRansom might get a heart attack if that were to happen.

I do not recommend supporting unsupported .NET FX runtimes like net40. It will truly be in the best long-term interest of this type provider to just target netstandard2.0 and build only with the .NET SDK compiler. Running into fundamental design issues like this are just what you're going to be signing yourself up for if you don't do that.

KevinRansom commented 3 years ago

@smoothdeveloper , @cartermp --- I will read this later ... the notion that I may have a heart attack is not attractive to me. Coffee and breakfast first, then i will have the fortitude to risk a medical event.

KevinRansom commented 3 years ago

@smoothdeveloper

I hope this clears things up, and at least explains some of the things you are seeing?

Kevin

cartermp commented 3 years ago

The current TP SDK demonstrates how to do this.

Actually, not anymore, since it was so fragile/fraught with complexity. It is now entirely .NET Standard 2.0-focused.

smoothdeveloper commented 3 years ago

Thanks for the feedback and fuller overview guys, it makes it clear that it is on the TP providers (!) side to deal with this.

I guess the challenge for now is to get net framework projects consume a net standard only release of the providers I'm dealing with and can be tracked in the specific projects instead.

I'm closing this, maybe should be tagged by-design or such.

goswinr commented 3 years ago

@smoothdeveloper @KevinRansom I have similar problem with the Excel provider. I am using the latest Type provider SDK and target net48 in my own fork, but dotnet build still fails when using the Type provider with error FS3033: The type provider 'FSharp.Interop.Excel.ExcelProvider.ProviderImplementation+ExcelProvider' reported an error: Could not create ExcelDataReader. NotSupportedException - No data is available for encoding 1252. For information on defining a custom encoding, see the documentation for the Encoding.RegisterProvider method. However, building in Visual Studio works just fine. cc @quintusm Is there a workaround that would allow me to use dotnet build ?

kerams commented 3 years ago

@goswinr, that does not seem like the original issue. I've seen the error, albeit on .NET Core and at runtime, and solved it with Encoding.RegisterProvider CodePagesEncodingProvider.Instance. Try calling that during the initialization of the design part of your TP, it could help.

KevinRansom commented 3 years ago

@goswinr --- could you provide a simplified repro or point to a github repo with instructions, and we can take a look ... thanks

goswinr commented 3 years ago

thanks, @kerams CodePagesEncodingProvider is .NET core only. 🤷‍♂️ @KevinRansom @vzarytovskii the problem is in how I build my own fork. With the Excelprovider Nuget it works fine. I noticed that the Runtime.dll from my own build is 1234 kB while the one from Nuget is just 45 kB in the Nuget. But building the original TP seems broken too. Anyway, I think you can close this issue for now, since it only affects my fork.

smoothdeveloper commented 2 years ago

@vzarytovskii / @KevinRansom can we remove "needs-repro", and reopen this issue?

I'm attaching another sample, which involves a type provider that supports netstandard, switching the target to net6.0 in the sample doesn't change the outcome.

sqlclienttp.zip

Maybe there is a way to improve the compiler report more details about the precise issue hit during the compilation and how to adjust for it?

Actual

Builds with:

Fails to build with:

error FS3033: The type provider 'FSharp.Data.SqlCommandProvider' r eported an error: Could not load file or assembly 'System.Data.SqlClient, Version=4.4.0.0, Culture=neutral, PublicKeyTo ken=b03f5f7f11d50a3a'. Reference assemblies should not be loaded for execution. They can only be loaded in the Reflect ion-only loader context. (0x80131058) error FS3033: The type provider 'FSharp.Data.SqlCommandProvider' r eported an error: Could not load file or assembly 'System.Data.SqlClient, Version=4.4.0.0, Culture=neutral, PublicKeyTo ken=b03f5f7f11d50a3a'. Reference assemblies should not be loaded for execution. They can only be loaded in the Reflect ion-only loader context. (0x80131058)

Expected

Builds in all those scenarios.

jimfoye commented 1 year ago

I have a project that targets net5.0-windows and uses SqlCommandProvider and I get this error with dotnet publish. But if I right click on the project in VS and select "Publish..." from the context menu, it works. I even call dotnet publish with the parameter -p:PublishProfile=FolderProfile.pubxml so I would think this is actually the exact equivalent of asking VS to do it. I'm not really sure what to do - I don't think the 5.0 target is the problem, right? (That's the highest version VS 2019 supports, so I'm stuck there). I'm trying to onboard someone with "see how great F# and type providers are" and I could use a good explanation on why I'm having this problem and so cannot automate the build entirely with a build script.

[Edit] I realize this is not a showstopper, dotnet publish is just building and copying files, I can do a build script that doesn't rely on it.