Closed sergey-tihon closed 2 years ago
It looks like things worked once you set the paths to fsc.exe, but I was under the impression that this was no longer required, per the fsproj files in FSharp.Data: https://github.com/fsharp/FSharp.Data/blob/master/src/FSharp.Data/FSharp.Data.fsproj#L6
I'm not quite sure what's going on with that, but we should be able to build for net45 using the dotnet sdk only, right?
I do still have ignored tests for ResX provider (the issue looks not related to TP).
I'm not quite sure what's going on with that, but we should be able to build for net45 using the dotnet sdk only, right?
Yes, you are right, I also did not use fsc.props
in TP projects. It is needed to build projects (projects that use TP) because otherwise dotnet build
report the issue that is cannot find the class with TypeProvided
attributed in assembly marked by TypeProviderAssembly
attribute.
at least it works for Linux and macOS =)
on windows, MSBuild cannot find assembly because of version mismatch of SharpYaml dependency ...
X:\FSharp.Configuration\tests\FSharp.Configuration.Tests\YamlProvider.Tests.fs(8,17): error FS3033: The type provider 'FSharp.Configuration.ConfigTypeProvider+FSharpConfigurationProvider' reported an error: Could not load file or assembly 'System.Reflection.TypeExtensions, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified. [X:\FSharp.Configuration\tests\FSharp.Configuration.Tests\FSharp.Configuration.Tests.fsproj]
I am not sure how to pass binding redirect in this case ...
Update: I think is possible to create SharpYaml version that work without binding redirects https://github.com/xoofx/SharpYaml/issues/54
@baronfel Why Fsharp.Data
NuGet package does not contain net standard
facade assemblies? https://github.com/fsprojects/FSharp.TypeProviders.SDK#making-a-net-standard-20-tpdtc
That's a good question. To be honest I'm still very shaky on it, and we may want to get clarification from @dsyme, but I believe it's because on .net 45+ FSharp.Data provides a net45 TP Design-Time Component. I think the netstandard facade dll would be required if the TP only shipped a netstandard2.0 version of the Design-Time component, and then only in order to make the netstandard2.0 component run on .net461 and greater.
First alpha with 2 TPs for netstandard2.0
(Yaml
& Ini
) and all TPs for net45
is released on NuGet - https://www.nuget.org/packages/FSharp.Configuration/2.0.0-alpha1
I tried YamlConfig on Mac, the repo is https://github.com/vasily-kirichenko/ConsoleApp4
It work properly in Rider:
but compilation fails:
$ dotnet build
Microsoft (R) Build Engine version 15.6.82.30579 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.
Restoring packages for /Users/vaskir/RiderProjects/ConsoleApp4/ConsoleApp4/ConsoleApp4.fsproj...
Generating MSBuild file /Users/vaskir/RiderProjects/ConsoleApp4/ConsoleApp4/obj/ConsoleApp4.fsproj.nuget.g.props.
Restore completed in 192.34 ms for /Users/vaskir/RiderProjects/ConsoleApp4/ConsoleApp4/ConsoleApp4.fsproj.
error FS3053 : The type provider 'FSharp.Configuration.ConfigTypeProvider+FSharpConfigurationProvider' reported an error : The type provider constructor has thrown an exception: Exception has been thrown by the target of an invocation. [/Users/vaskir/RiderProjects/ConsoleApp4/ConsoleApp4/ConsoleApp4.fsproj]
FSC : warning FS3005: Referenced assembly '/Users/vaskir/.nuget/packages/fsharp.configuration/2.0.0-alpha1/lib/netstandard2.0/FSharp.Configuration.dll' has assembly level attribute 'Microsoft.FSharp.Core.CompilerServices.TypeProviderAssemblyAttribute' but no public type provider classes were found [/Users/vaskir/RiderProjects/ConsoleApp4/ConsoleApp4/ConsoleApp4.fsproj]
Build FAILED.
FSC : warning FS3005: Referenced assembly '/Users/vaskir/.nuget/packages/fsharp.configuration/2.0.0-alpha1/lib/netstandard2.0/FSharp.Configuration.dll' has assembly level attribute 'Microsoft.FSharp.Core.CompilerServices.TypeProviderAssemblyAttribute' but no public type provider classes were found [/Users/vaskir/RiderProjects/ConsoleApp4/ConsoleApp4/ConsoleApp4.fsproj]
error FS3053 : The type provider 'FSharp.Configuration.ConfigTypeProvider+FSharpConfigurationProvider' reported an error : The type provider constructor has thrown an exception: Exception has been thrown by the target of an invocation. [/Users/vaskir/RiderProjects/ConsoleApp4/ConsoleApp4/ConsoleApp4.fsproj]
1 Warning(s)
1 Error(s)
Time Elapsed 00:00:02.95
@vasily-kirichenko Yes.... one manual change required in fsproj
file
<Project Sdk="Microsoft.NET.Sdk">
<Import Project="../../fsc.props" />
and fsc.props
file - https://github.com/fsprojects/FSharp.TypeProviders.SDK/blob/master/fsc.props that specify FscToolPath
& FscToolExe
Type providers currently can't run inside the .NET Core 2.0 hosted compiler, see https://github.com/Microsoft/visualfsharp/pull/3658#issuecomment-334773415
@sergey-tihon thanks, it works, but... Is it possible to add the props file into the nuget package and auto modify project file when it's installed? I know this is done by FSharp.Compiler.Tools, for example.
@vasily-kirichenko hm... interesting idea. as I know other TPs do not do it yet...
But what if you use several TPs NuGet packages and each import such fsc.props
file...
They may become out of sync. For example Fsharp.Configuration for Windows uses path
<FscToolPath>C:\Program Files (x86)\Microsoft SDKs\F#\10.1\Framework\v4.0</FscToolPath>
but SDK file is still pointing to
<FscToolPath>C:\Program Files (x86)\Microsoft SDKs\F#\4.1\Framework\v4.0</FscToolPath>
This probably means that path depends on VS version installed on the end-user machine... and there is no one file that covers all cases at the moment.
We could try to invert one and release as a separate NuGet package and add the dependency from all type provider project, but I am not sure how long we need it.
I want to believe that that very soon TPs will be able to run inside the .NET Core 2.0 hosted compiler...
Finally had some time to test the new package (https://www.nuget.org/packages/FSharp.Configuration/2.0.0-alpha1). Built a project successfully both on Windows and Mac using either Visual Studio or Rider. Will have to run some runtime tests, but looks very promising so far. Great work guys!
@baronfel Do you have an idea how Win build could be fixed?
No, I was poking around locally this weekend a bit and didn't really get anywhere :(
We are green again! This time 2.0.0-alpha2
is released with AppSettingsProvider
compiled for netstandard 2.0.
https://www.nuget.org/packages/FSharp.Configuration/2.0.0-alpha2
Hello,
Any idea why I can't import latest FSharp.Configuration 2.0.0-alpha2 in FAKE 5?
Error with fake is now :
Script is not valid:
startup (1,0)-(1,0): Error FS3053: The type provider 'FSharp.Configuration.ConfigTypeProvider+FSharpConfigurationProvider' reported an error: The type provider constructor has thrown an exception: System.Runtime.Caching is not supported on this platform.
startup (1,0)-(1,0): Warning FS3005: Referenced assembly 'C:\Users\Franc\Documents\dev\dotnet\FSharp\fake-test\packages\build\FSharp.Configuration\lib\netstandard2.0\FSharp.Configuration.dll' has assembly level attribute 'Microsoft.FSharp.Core.CompilerServices.TypeProviderAssemblyAttribute' but no public type provider classes were found
Minimum steps to reproduce
dotnet new fake
paket add -g Build FSharp.Configuration -V 2.0.0-alpha2
fake build
Regards,
@fradav I've just encountered exactly the same problem when setting the VSTS build with FAKE5. Did you find any workarounds?
@fradav @grishace Did I understand correctly that you want to use TP from the FAKE script itself?
@matthid Do you know about successful usage of TP from FAKE 5 scripts (on dotnet core)?
@sergey-tihon Nope. It happens on the build step. No TP in the build script itself.
I think FSharp.Configuration is required on the Build group because of TP compile-time phase.
Most probably, you have to specify F# compiler location - https://github.com/fsprojects/FSharp.Configuration/pull/139#issuecomment-385191137
If you compile for full framework on mac\linux you need one more props file with reference to Mono location.
@sergey-tihon I do have fsc.props reference in my fsproj files. Not sure how to control fake.exe behavior though
Yes I think I have used type providers in fake 5 after they have been ported to netcore and the new type provider sdk.
If you don't want fake to pickup the type provider you need to install the package in a different paket group. Fake 5 always references all packages of a complete group...
@matthid
I do have a "solution" folder, that contains .fake, .paket folders + paket.dependencies and paket.lock + build.fsx. All sources for build targets have their own paket.references files.
You mean I'd need to add some other group besides Build and add project references to this group?
Oh, and I forgot to mention that everything is more or less fine with the local build. What I'm trying to do - set the VSTS build with hosted agent because SQLProvider requires connection to on prems DB.
I have no details about your setup but yes recommendation would be to have separate groups for the build script and your projects (there is usually no reason to unify those)
@matthid Thanks for the tip! I've got the green build!
There's one strange issue which seems to not affect the compiled app behavior:
E:\A_work\80\s\src\fsharp\FSharp.Core\async.fsi(450,16): warning FS0073: internal error: CodeGen check: type checking did not quantify the correct number of type variables for this method, #parentTypars = 1, #ctps = 1, #mtps = 0, #thisArgTys = 2
@grishace I assume this is due to the latest fable update can you please create an issue there with more details?
@matthid it will take me some time to create a minimal code sample to reproduce this problem.
@sergey-tihon Is this ready to go? I believe the bug above
E:\A_work\80\s\src\fsharp\FSharp.Core\async.fsi(450,16): warning FS0073: internal error: CodeGen check: type checking did not quantify the correct number of type variables for this method, #parentTypars = 1, #ctps = 1, #mtps = 0, #thisArgTys = 2
was an F# compiler bug which is now fixed
Using this in Ionide on a project targeting net471 I get IDE errors for each file that uses a type provided by this TP due to missing references. I'm experimenting locally, but it seems that this can be fixed by identifying the missing references and shipping them in the net45
directory the same way we do with the netstandard2.0
directory.
The total set of dependencies I had to add were:
but there are some odd assembly versions involved there. IE mono 5.14 doesn't have a System.Diagnostics.PerformanceCounter.dll from what I can see, or a System.Configuration.ConfigurationManager.dll.
Interesting notes: you can see that the TP lib being referenced above is the net45 version, which monop reports has having the following dependencies:
➜ net45 git:(master) ✗ monop --refs -r FSharp.Configuration.dll
mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
FSharp.Core, Version=4.4.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
YamlDotNet, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null
System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
System.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
System.Runtime.Caching, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
this list doesn't contain System.Configuration.ConfigurationManager
at all, why is it being referenced?
For comparison here is the reference list from the netstandard2.0 version of the FSharp.Configuration assembly:
➜ net45 git:(master) ✗ monop --refs -r ../netstandard2.0/FSharp.Configuration.dll
netstandard, Version=2.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
FSharp.Core, Version=4.4.3.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
YamlDotNet, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null
System.Runtime, Version=4.1.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Reflection, Version=4.1.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Globalization, Version=4.0.11.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Console, Version=4.0.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.IO, Version=4.1.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Runtime.Caching, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
System.Configuration.ConfigurationManager, Version=4.0.1.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
this does have the aforementioned dependency.
I've tried this in both the mono and netcore versions of ionide as well, in an attempt to see if it was the netcore-hosted tooling that was failing, but I get the same errors in both environments.
In fact, if I go to the local package directory and completely delete the netstandard2.0 dlls, modify the nuspec to remove references to netstandard2.0, etc, I still get those errors.
I am very confused 😕
@baronfel tests failed on Win :(
This is the funny fact, when you develop on win - build fails on mono, when you develop on mono - it fails on win =)
hard to tell for which framework they are failing, but it seems to be a problem finding System.Configuration.ConfigurationManager:
C:\projects\fsharp-configuration\tests\FSharp.Configuration.Tests\AppSettingsProvider.Tests.fs(7,17): error FS3033: The type provider 'FSharp.Configuration.ConfigTypeProvider+FSharpConfigurationProvider' reported an error: Could not load file or assembly 'System.Configuration.ConfigurationManager, Version=0.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. The system cannot find the file specified. [C:\projects\fsharp-configuration\tests\FSharp.Configuration.Tests\FSharp.Configuration.Tests.fsproj]
The tests for net461 framework don't seem to be including a reference to System.Configuration:
C:\Program Files\dotnet\sdk\2.1.402\FSharp\RunFsc.cmd -o:obj\Release\net461\FSharp.Configuration.Tests.exe
--debug:portable
--noframework
--define:NET461
--define:RELEASE
--define:NETFRAMEWORK
--define:NET461
--optimize+
--platform:anycpu
--resource:obj\Release\net461\FSharp.Configuration.Tests.Resource1.resources
-r:C:\Users\appveyor\.nuget\packages\argu\5.1.0\lib\net45\Argu.dll
-r:C:\Users\appveyor\.nuget\packages\expecto\8.4.2\lib\net461\Expecto.dll
-r:C:\projects\fsharp-configuration\src\FSharp.Configuration\bin\Debug\net45\FSharp.Configuration.dll
-r:C:\Users\appveyor\.nuget\packages\fsharp.core\4.3.4\lib\net45\FSharp.Core.dll
-r:C:\Users\appveyor\.nuget\packages\mono.cecil\0.10.1\lib\net40\Mono.Cecil.dll
-r:C:\Users\appveyor\.nuget\packages\mono.cecil\0.10.1\lib\net40\Mono.Cecil.Mdb.dll
-r:C:\Users\appveyor\.nuget\packages\mono.cecil\0.10.1\lib\net40\Mono.Cecil.Pdb.dll
-r:C:\Users\appveyor\.nuget\packages\mono.cecil\0.10.1\lib\net40\Mono.Cecil.Rocks.dll
-r:C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.1\mscorlib.dll
-r:C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.1\System.Core.dll
-r:C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.1\System.Data.dll
-r:C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.1\System.dll
-r:C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.1\System.Drawing.dll
-r:C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.1\System.IO.Compression.FileSystem.dll
-r:C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.1\System.Numerics.dll
-r:C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.1\System.Runtime.Serialization.dll
-r:C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.1\System.Xml.dll
-r:C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.1\System.Xml.Linq.dll
-r:C:\Users\appveyor\.nuget\packages\yamldotnet\5.2.1\lib\net45\YamlDotNet.dll
--target:exe
--nowarn:NU1603,NU1604,NU1605,NU1608
--warn:3
--warnaserror:76
--fullpaths
--flaterrors
--subsystemversion:6.00
--highentropyva+
--targetprofile:mscorlib
--simpleresolution
--nocopyfsharpcore
C:\Users\appveyor\AppData\Local\Temp\1\.NETFramework,Version=v4.6.1.AssemblyAttributes.fs
obj\Release\net461\FSharp.Configuration.Tests.AssemblyInfo.fs
AppSettingsProvider.Tests.fs
ResXProvider.Tests.fs
YamlProvider.Tests.fs
IniFileProvider.Tests.fs
Program.fs
that could be the root cause. The netcoreapp2.0 build does have the System.Configuration.ConfigurationManager reference in it.
Are you able to reproduce on windows locally, @sergey-tihon?
Yes, it fails on my win VM
I can repro on my windows machine as well. I'm very confused, because the layouts of the two generated packages are the same.
I tried a variant of the test project where I generate the nuget package before building the tests and include that package as a PackageReference
for the Tests to build with, using it like an actual consumer would, but it still doesn't work.
I tried copying the missing System.Configuration.ConfigurationManager reference into the bin/lib/net45
directory and got a bit further with this error:
D:\code\FSharp.Configuration\tests\FSharp.Configuration.Tests\AppSettingsProvider.Tests.fs(7,17): error FS3033: The type provider 'FSharp.Configuration.ConfigTypeProvider+FSharpConfigurationProvider' reported an error: Could not load type 'System.Configuration.ExeConfigurationFileMap' from assembly 'System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. [D:\code\FSharp.Configuration\tests\FSharp.Configuration.Tests\FSharp.Configuration.Tests.fsproj]
My takeaway from this is that I should probably instrument the assembly load probing and see which folders are being checked on windows vs on mono. What do you think?
My guess is that we should not rely on assemblies from GAC and should request from NuGet when assembly is there.
We compile TP with reference to System.Configuration.dll
from GAC (when compile for net45
) and we do not copy this assembly to bin/lib/net45
(because it is in GAC)
Then we build FSharp.Configuration.Tests
for net461
(not net45
) + we have some binding redirects in app.config
and assume that it should somehow find System.Configuration.dll
FWIW @panesofglass pointed out that FSharp.Data has a caching implementation that works cross-platform in a way that is probably good-enough for us here, so I can/should replace the caching here before we merge. (In addition to fixing the windows stuff for sure).
Out of curiousity, I want to try building on linux/osx and consuming the package from windows. If it is consumable would that be 'good enough'?
We definitely could release next beta with binary compiled on linux, if it works fine on windows.
Untimely I would expect that we will find solution for Win build.
Also would be awesome if caching solution implemented for FSharp.Data is referenceble as file using Paket GitHub dependency feature.
Agree on all points! I'm getting some testing done on Windows right now with a osx-built variant, to verify that the dll works regardless of consumer.
If that is the case I'll work on using the FSharp.Data caching implementation, and then try to figure out the windows build issue.
Windows testing found that we need to have netcoreapp2.1 builds include a reference to the System.Configuration.ConfigurationManager
package or else you get a runtime error
The other issue found was that the yaml TP fails at design-time:
@baronfel Maybe it worth to migrate to new TP Template with design and runtime component?
I am currently trying to port ResXNode and stuff from WF source to netstandard and net core, and it looks like I can make it work. Will try to port ResxProvider if I succeed with the first part.
@sergey-tihon should I port project to new sdk template before any attempts to port ResxProvider? With separate design-time/runtime projects.
@Liminiens I think that would be better to fix/finish ResxProvider, merge this PR and then migrate to template in separate PR (if it is possible)
@sergey-tihon it seems like I made System.Resources.Resx...
stuff to work on netstandard (as a separate package), at least it should work for provider purposes. But tests are failing with the same error as in https://github.com/fsprojects/FSharp.Configuration/pull/139#issue-183292161 maybe i should push current changes here?
System.Resources.MissingManifestResourceException: Could not find any resources appropriate for the specified culture or the neutral culture. Make sure "Resource1.resources" was correctly embedded or linked into assembly "FSharp.Configuration.Tests" at compile time, or that all the satellite assemblies required are loadable and fully signed.
at System.Resources.ManifestBasedResourceGroveler.HandleResourceStreamMissing(String fileName)
at System.Resources.ManifestBasedResourceGroveler.GrovelForResourceSet(CultureInfo culture, Dictionary`2 localResourceSets, Boolean tryParents, Boolean createIfNotExists, StackCrawlMark& stackMark)
at System.Resources.ResourceManager.InternalGetResourceSet(CultureInfo requestedCulture, Boolean createIfNotExists, Boolean tryParents, StackCrawlMark& stackMark)
at System.Resources.ResourceManager.InternalGetResourceSet(CultureInfo culture, Boolean createIfNotExists, Boolean tryParents)
at System.Resources.ResourceManager.GetObject(String name, CultureInfo culture, Boolean wrapUnmanagedMemStream)
at FSharp.Configuration.ResXProvider.readValue[a](String resourceName, Assembly assembly, String key)
at FSharp.Configuration.Tests.ResXTests.tests@19-14.Invoke(Unit _arg3)
@sergey-tihon yeah I agree with migrating to the new template. I was going to try to do at least the yaml one in that way over the christmas break. The good news is that the TPs are pretty separated, so most of the work should be just getting the first one up and running in the new template, then porting over each other TP into the new format.
Status
net45
Unit tests for ResX type provider temporary ignored because provided code failed to load actual value in runtime
Here should be some magic that fixes it when we build for full frameworks...
As of today .net core has the very limited support for resource files - https://github.com/Microsoft/msbuild/issues/2221
Status
netstandard2.0
TPs migrated to
netstandard2.0
:Tip for
YamlDotNet
serializer customization - https://www.cyotek.com/blog/using-custom-type-converters-with-csharp-and-yamldotnet-part-1