Closed dsyme closed 6 years ago
Note, I think we can have a better setting for FscToolPath which works on both Windows and Mono, though it needs a conditional to determine the right location
Thanks @dsyme!
The FscToolPath/FscToolExe workaround seems to work just fine even with SDK 1.0.4.
@cartermp @KevinRansom As I've now documented in the issue description, it is possible to use the above technique to enable type providers for .NET Core programming, as long as the type providers have been written as a cross targeting type provider. The main cross-targeting type provider is FSharp.Data.
Here are the steps I used to verify that this can be used with .NET Core 2.0 programming:
open System
open FSharp.Data
type NugetStats = HtmlProvider<"https://www.nuget.org/packages/FSharp.Data">
let rawStats = NugetStats().Tables.``Version History``
rawStats.Rows |> Seq.iter (printfn "row = %A")
<FscToolPath>C:\Program Files (x86)\Microsoft SDKs\F#\4.1\Framework\v4.0</FscToolPath>
<FscToolExe>fsc.exe</FscToolExe>
output:
row = ("F# Data 2.3.3 (this version)", 22954M, 10/04/2017 00:00:00)
...
row = ("FSharp.Data 1.0.2", 234M, 14/12/2012 00:00:00)
row = ("FSharp.Data 1.0.1", 187M, 14/12/2012 00:00:00)
row = ("FSharp.Data 1.0.0", 469M, 13/12/2012 00:00:00)
šššš we have used type providers for .NET Core programming for the first time :)
Notes:
When using VSCode/Ionide you also get full intellisense:
The above technique also works for .NET Standard 2.0 programming (dotnet new lib -lang F# tplib1
). However at the time of writing you need an F# compiler that includes this fix, e.g. use Visual Studio 2017 preview 3, or use an F# compiler built from source. I tested with <FscToolPath>C:\GitHub\dsyme\visualfsharp\release\net40\bin</FscToolPath>
You get some warnings because FSharp.Data for .NET 4.x is being used. However it should be possible to produce a fully compliant .NET Standard 2.0 version of FSharp.Data, that is TBD, and until then the .NET Framework version is not that likely to cause major problems since the surface area of FSharp.Dataa is relatively low
@dsyme you are the freaking man! If I'm ever at a conference you're speaking at dinner one night is on me...
So I journeyed into the matrix to get my application running inside a Docker container on Linux despite it's heavy JSON Type Provider usage due to your workaround and I'm happy to report it works like a champion!
Pros: This workaround is elegant, nice, and awesome Runs in Docker on Linux!!!!! Only developer IDEs and build servers need Windows which is perfectly livable until type providers compile on Core I can still write all my build scripts for the build servers to be dotnet cli compliant
Cons: There were 3 things that came up and I've attached a screenshot for all of them to see if I'm doing something wrong..
FSC was complaining about netstandard2.0 but it works when I put in netcoreapp2.0
NuGet shows no packages installed for F#
My Intellisense is broken but I'm pretty certain this probably has something to do with item 2
The only other con was dotnet build, run, restore, publish all throw errors when I include the docker-compose.dcproj file in the .sln file. I have read that is because the SDK hasn't been open sourced which is a requirement to be included in the tool chain. This is not a concern as I can simply script the commands to all run against the top level project file rather than the solution.
I have a feeling my cons are entirely a PEBCAK and I've thought up workarounds in the event they are not. I'm still way happier about actually being able to build out my services against dockerswarm on Linux and still being able to use the Type Providers.
Regards, Brian
The PEBCAK is that I couldn't read.... vscode/ionide I will leave the items up here just in case we want the VS team to see they are very close.
Not sure about (2), but (3) is due to an issue in the language service. #3260 is tracking that issue, and @KevinRansom is currently working on it. This is wonderful to hear, though, @buvinghausen! Happy to hear that it's working for you in Docker/Linux.
Only developer IDEs and build servers need Windows which is perfectly livable until type providers compile on Core
Atm will work also on non windows, using mono. two big ways:
use mono on dev on non win. same workaround
build
a mono docker image for build, build there => get outputrun
run the build output in another container with just .net corepro of 2 is that with multi stage, the image size contains only netcore (use official dotnet core
docker image) and is slimmer
maybe i should gist the dockerfile i use for that. is a good setup, also for ci who are docker based (we use jenkins).
@dsyme as a note, dotnet new lib -lang F#
works the same of classlib
, and is less wierd š
@cartermp thanks for the clarification I'm actually working on getting VS Code launch & debug into the Docker container like I can from VS 2017 then I may just cut the cord on VS full I've been looking for an excuse to do that as I have been using VS Code exclusively for TypeScript development.
@enricosada Those are some excellent points but for now I'm totally happy with simply making my CI server run on Windows and letting the FE developers run everything in Node but talking to our integration services on the back end. That should give me enough runway to let all the frameworks and tooling catch up such that I can then tell them they have to install .NET Core & OmniSharp and learn the dotnet cli. As an aside I think today is the day that I'm going to see how feasible it is to run the React Redux template in Docker. I know when you create the project they have the docker option disabled. Should be fun.
I really appreciate all the work you guys are putting into this platform I cannot stress that enough it really is amazing to think what the future holds.
Thx for the workaround but I encountered another issue:
error FS3033: The type provider 'FSharp.Data.TypeProviders.DesignTime.DataProviders' reported an error: The .NET SDK 4.0 or 4.5 tools could not be found
Source code tells the truth: SDK paths are fixed. On my relatively new machine (or on VSTS agent) there're no such registry keys. I added new ones (which existed on my machine):
@"Software\Microsoft\Microsoft SDKs\NETFXSDK\4.6.1\WinSDK-NetFx40Tools" @"Software\Microsoft\Microsoft SDKs\NETFXSDK\4.6.2\WinSDK-NetFx40Tools"
and now it works. Are you interested in pull request for that change?
My solution currently to finding the proper fsc.exe on macOS/Linux is to use a small helper script under my ./tools
dir called fsharpc.exe
#!/bin/sh
fsharpc "$@"
or if you want more control over your fsc.exe
version by using nuget/paket
#!/bin/sh
mono $(git rev-parse --show-toplevel)/packages/build/FSharp.Compiler.Tools/tools/fsc.exe --exename:$(basename "$0") "$@"
Then my project file
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net462</TargetFramework>
</PropertyGroup>
<PropertyGroup Condition="$(OS)=='Unix'">
<FscToolPath>../../tools</FscToolPath>
<FscToolExe>fsharpc.exe</FscToolExe>
</PropertyGroup>
All it does it delegate to fsharpc
, another shell script usually installed my mono and put on the path. For some reason the tooling is looking for an .exe
at the end and won't execute the script without it. I saw it once somewhere but I can't remember where it was doing the check.
This seems to work with:
$ mono -V
Mono JIT compiler version 4.8.1 (mono-4.8.0-branch/22a39d7 Fri Apr 7 12:00:08 EDT 2017)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
TLS: normal
SIGSEGV: altstack
Notification: kqueue
Architecture: x86
Disabled: none
Misc: softdebug
LLVM: yes(3.6.0svn-mono-master/8b1520c)
GC: sgen
$ dotnet --info
.NET Command Line Tools (1.0.1)
Product Information:
Version: 1.0.1
Commit SHA-1 hash: 005db40cd1
Runtime Environment:
OS Name: Mac OS X
OS Version: 10.11
OS Platform: Darwin
RID: osx.10.11-x64
Base Path: /usr/local/share/dotnet/sdk/1.0.1
Libs I've tested so far: FSharp.Data FSharp.Configuration FSharp.Management
Additional note: mdb doesn't seem to get copied over so I have to add this target to copy mdbs
<Target Name="Copy MDBs"
Condition="$(OS)=='Unix'"
AfterTargets="Build">
<Copy SourceFiles="$(IntermediateOutputPath)/$(AssemblyName).exe.mdb" DestinationFolder="$(OutputPath)" ContinueOnError="true" />
</Target>
@TheAngryByrd Thanks for that info
Additional note: mdb doesn't seem to get copied over so I have to add this target to copy mdbs...
@KevinRansom It seems this should be part of standard F# targets?
Repo with instructions to enable the workaround : https://github.com/rfrerebe/TypeProviderDotNetCorePreview2
One thing I've also noticed is I don't get incremental build support using that macos/linux workaround. Is there something else I need to set or is it just not supported?
i.e. normal .net core builds
Skipping target "CoreCompile" because all output files are up-to-date with respect to the input files.
Seems like the fsc.exe shipped with mono 4.8 is still 4.4.0.0
monodis --assembly /Library/Frameworks/Mono.framework/Versions/4.8.1/lib/mono/fsharp/fsc.exe
Assembly Table
Name: fsc
Hash Algoritm: 0x00008004
Version: 4.4.0.0
Flags: 0x00008001
PublicKey: BlobPtr (0x000000ea)
Even though it reports it's the compiler for 4.1
mono /Library/Frameworks/Mono.framework/Versions/4.8.1/lib/mono/fsharp/fsc.exe
F# Compiler for F# 4.1
Either using mono 5.0.1
's version or downloading the latest FSharp.Compiler.Tools from nuget doesn't have the project rebuild every single time.
I need to do something specific to running on SDK 1.0.4 and targetting netcoreapp1.1
? Trying target to net462
give me a issue on dotnet run
:
āÆ dotnet run
/opt/dotnet/sdk/1.0.4/Microsoft.Common.CurrentVersion.targets(1111,5): error MSB3644: The reference assemblies for framework ".NETFramework,Version=v4.6.2" were not found. To resolve this, install the SDK or Targeting Pack for this framework version or retarget your application to a version of the framework for which you have the SDK or Targeting Pack installed. Note that assemblies will be resolved from the Global Assembly Cache (GAC) and will be used in place of reference assemblies. Therefore your assembly may not be correctly targeted for the framework you intend. [/home/lerax/Desktop/dchart/backend/backend.fsproj]
The build failed. Please fix the build errors and run again.
I should able to compile and running with mono, but is not working when I try running with .NET Core. My setup is .NET Core SDK 1.0.4 and .NET Core runtime 1.1.2 on Linux.
Alternatively, fsc
can be obtained from FSharp.Compiler.Tools
nuget. With Paket:
paket.dependencies:
nuget FSharp.Compiler.Tools
.fsproj:
<PropertyGroup>
<FscToolPath>../packages/FSharp.Compiler.Tools/tools</FscToolPath>
<FscToolExe>fsc.exe</FscToolExe>
</PropertyGroup>```
Tested on Windows 7, but it should work cross-platform (?)
@jindraivanek you are targetting to which framework? (I just want know)
@ryukinix net46
@dsyme Does cross-targeting type providers
now work with generative providers?
I was also wondering if an elaborate bait and swich could be used, like targeting 4.6.1 in consuming a type provider then using a net core facade assembly to redirect to the consumer dll?
Does anyone know how to make this workaround work on Linux (or microsoft/dotnet:sdk
Docker image)? We successfully use workaround @jindraivanek mentioned (using fsc.exe
from FSharp.Compiler.Tools
) on Windows, but anytime we want to build our project on Linux, Travis CI (based on Linux) or Docker, we got:
error MSB6006: "fsc.exe" exited with code 8.
FAKE script build.sh Publish
neither dotnet publish
, none of them work. We also tried to change paths in fsproj to try /usr/share/dotnet/sdk/2.0.0/FSharp/fsc.exe
instead of Compiler Tools one. Do we need to use different fsc.exe
? Any help would be great!
We resolve it now (it is combination of @TheAngryByrd and @jindraivanek approach), so here is the description, maybe someone finds it useful:
On windows, everything is the same - we use fsc.exe
directly from FSharp.Compiler.Tools
. To have it working on Linux, we did this:
Created file tools/fsc.exe
with content:
#!/bin/sh
chmod +x ../../packages/FSharp.Compiler.Tools/tools/fsc.exe
mono ../../packages/FSharp.Compiler.Tools/tools/fsc.exe "$@"
Changed fsproj
to have property groups like this:
<PropertyGroup Condition="$(OS)=='Unix'">
<FscToolPath>../../tools</FscToolPath>
<FscToolExe>fsc.exe</FscToolExe>
</PropertyGroup>
<PropertyGroup Condition="$(OS)=='Windows_NT'">
<FscToolPath>../../packages/FSharp.Compiler.Tools/tools</FscToolPath>
<FscToolExe>fsc.exe</FscToolExe>
</PropertyGroup>
OS
to have Unix
value in build.sh
script file.Please note: We use [root]/src/AppName folder structure, so if you do it differently, change paths in script file and fsproj.
... to have it working on Linux, we did this:... Created file tools/fsc.exe with content...
We should include the fsharpc
and fsharpi
scripts in packages/FSharp.Compiler.Tools/tools
as part of the FSharp.Compiler.Tools package
I'm trying to build an erasing type provider on .Net core and my problem is:
FSC : error FS3031: The type provider 'myProvider.dll' reported an error: Assembly
attribute 'TypeProviderAssemblyAttribute' refers to a designer assembly 'myProvider' which cannot
be loaded or doesn't exist. Could not load file or assembly 'System.Runtime, Version=4.2.0.0,
Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find
the file specified. [C:\tests\myTests.fsproj]
If I target to netstandard2.0 I fail compiling the typeprovider, and if I target to netcoreapp2.0 everything goes ok, until I try to build my test project: when compilation of the test project is done, the compiler executes some code related to TypeProviderAssemblyAttribute which somehow refers a non-core-version of System.Runtime.
edit: I think that the problems are inheriting from FieldInfo:
type ProvidedField(fieldName:string,fieldType:Type) =
inherit System.Reflection.FieldInfo()
Fieldinfo contains a member GetValueDirect taking TypedReference
as a parameter.
FSharp.TypeProviders.StarterPack\src\ProvidedTypes.fs(1597,6): error FS1109: A reference to the type 'System.TypedReference' in assembly 'System.Runtime' was found, but the type could not be found in that assembly
I'm trying to build an erasing type provider on .Net core ....
@Thorium This is not yet supported, see #2406
Yet another solution to get fsc.exe
working on any platform combining the ones above and https://github.com/Microsoft/msbuild/issues/539#issuecomment-289930591 (no need of $(OS)
environment variable).
Please, note that YMMV for <FscToolPath>
.
Maybe we can get the ultimate solution combining @Dzoukr solution using tools from FSharp.Compiler.Tools
with scripts add-ons proposed by @dsyme.
<PropertyGroup>
<IsWindows Condition="'$(OS)' == 'Windows_NT'">true</IsWindows>
</PropertyGroup>
<PropertyGroup Condition="'$(MSBuildRuntimeType)' == 'Core'">
<IsOSX Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::OSX)))' == 'true'">true</IsOSX>
<IsLinux Condition="'$([System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform($([System.Runtime.InteropServices.OSPlatform]::Linux)))' == 'true'">true</IsLinux>
</PropertyGroup>
<PropertyGroup Condition="'$(IsWindows)' == 'true'">
<FscToolPath>C:\Program Files (x86)\Microsoft SDKs\F#\4.1\Framework\v4.0</FscToolPath>
<FscToolExe>fsc.exe</FscToolExe>
</PropertyGroup>
<PropertyGroup Condition="'$(IsOSX)' == 'true'">
<FscToolPath>/Library/Frameworks/Mono.framework/Versions/Current/Commands</FscToolPath>
<FscToolExe>fsharpc</FscToolExe>
</PropertyGroup>
<PropertyGroup Condition="'$(IsLinux)' == 'true'">
<FscToolPath>/usr/bin</FscToolPath>
<FscToolExe>fsharpc</FscToolExe>
</PropertyGroup>
@manchoz Thanks for that workaround - I've added it above. I removed the Condition="'$(MSBuildRuntimeType)' == 'Core'"
- I don't think it's needed or correct when using Mono? Could you check please? Thanks
@dsyme yes, it works. Thanks!
I included netstandard.dll, System.Console.dll, System.IO.dll, System.Reflection.dll, System.Runtime.dll from the .NET Standard SDK to the type provider NuGet package and now it seems to work... I don't know if there will be problems but I'll probably find out soon...
Would this trick working on FSharp.Compiler.Service like fable? I'm trying using FSharp.Data.GraphQL client TP in fable, I update TP in FSharp.Data.GraphQL follow the cross target tutorial, and build successfully in dotnet build, but keeps throwing error for cannot find the provided type in Fable.
Has anyone been able to get this to work with FSharp.Data.TypeProviders to connect to a SQL Server database yet?
I've updated the description of what to do to be clearer, specifically adding this text:
Add the file
fsc.props
to the root of your project and import it into each of your project files that reference a type provider
@Disco-Dave FSharp.Data.TypeProviders is not a cross-targeting type provider at this stage. If doing .NET Framework programming you should be able to use it with a .NET SDK 2.0 style project file targeting net45
or net461
though I haven't checked
If you would like a SQL type provider to use with .NET Standard 2.0 programming, see https://github.com/fsprojects/SQLProvider, also @Thorium may have latest details on that.
@zaaack Re your question above - I'm not quite sure what you're asking? If it involves Fable it may be better to ask via a Fable issue
@manchoz If you could check the fsc.props file mentioned above, and if possible use it, that would be great
Hi, @dsyme. Sorry for the bother, I want to know does FSharp.Compiler.Service work on .netcoreapp2.0 with non-generative type providers with this configuration? Because I tried this in fable and cannot get the tp working(cannot find the type from type provider), if FCS works, then must be fable's issue. I find this issue https://github.com/fsharp/FSharp.Compiler.Service/issues/568 and wondering whether this still work when targeting .netcoreapp2.0..
SQLProvider status:
| Win10 | Mac | Ubuntu
Microsoft SQL Server | [x] | [x] | [x]
Postgres | [x] | [x] | [x]
MySQL | [x] | [x] | [x]
SQLite | [x] | |
I know you probably have your favourite database type provider, and I'm looking forward to see the others to support .Net Core also. Having multiple options for tools is a good thing.
@zaaack Thanks, I see. It is not yet under test to load type providers into FCS running in .netcoreapp 2.0. It might work after recent fixes by @KevinRansom in this repo, but it's possible FCS needs to be republished. But as I said, it's not yet under test even if it works
@dsyme, thanks a lot!
Just a "heads up"...
The second link in
- Add the file fsc.props to the root of your project and import it into each of your project files that reference a type provider
points to an incorrect example.
The correct syntax for importing the fsc.props
file is something along the lines of: <Import Project="fsc.props" />
.
Took me quite a while (and help from @enricosada) to figure this out.
@pblasucci Thanks for pointing that out, I fixed it
@dsyme I have a type provider which is built as Combo NET Standard TPDTC. Works fine with NET Core on Windows with any tooling VS/Rider/VSCode. Deployed on Linux works at run-time too. On Mac and Linux at design/build time throws "Could not resolve field token 0x0400055b" Any suggestions?
I use the latest Provided Types SDK. Tooling redirection applied as suggested https://github.com/Microsoft/visualfsharp/issues/3303#issuecomment-331426983
Thanks
@dsyme I got it. It's same as https://github.com/npgsql/npgsql/issues/1603#issuecomment-338495817 https://github.com/npgsql/npgsql/issues/1633#issuecomment-338495347
What's the best way to solve it? Because the way it's done in SQLProvider is not really acceptable.
@dmitry-a-morozov we accept PRs to SQLProvider if you find any better solution.
@Thorium Unfortunately I don't have better solution. I was surprised to see you guys ship mix of NetStandard and NET461 components in netstandard2.0 folder.
@dmitry-a-morozov @Thorium Just to say sorry for not getting back to this earlier. I'm getting back from vacation now and will start looking into these TPSDK bugs
@dmitry-a-morozov With regard to this:
@dsyme I have a type provider which is built as Combo NET Standard TPDTC. Works fine with NET Core on Windows with any tooling VS/Rider/VSCode. Deployed on Linux works at run-time too. On Mac and Linux at design/build time throws
Could not resolve field token 0x0400055b
I haven't seen this bug previously, it must be a difference in reflection on Mono. We will need a detailed repro for this. Could you submit as much information a possible to the TPSDK repo please?
@dsyme Sorry for delayed response. I'm on vacation with limited access to internet. I believe this issue was resolved by installing the latest Mono.
I just wanted to say thank you to all the contributors on this thread! We've got the build agent docker images retrofitted with Mono and are using the fsc.props file that @dsyme has in the updated suggestions and everything works marvelously. This saved me some major headaches as one of my vendors has a not fun XML solution in place that I was able to make sane via the XML type provider. The only heads up I have to offer is if you do run your build commands against the Solution File make sure your pathing has the *nix forward slashes in them. Windows seems just fine with the forward slash as it is the back slash the only caveat is any time you add a project to it VS will re-write all the slashes to back slashes...
I'm pretty sure that I followed the workaround from above: https://github.com/Microsoft/visualfsharp/issues/3303#issuecomment-312966306.
I get 'FS2024: Static linking may not be used on an assembly referencing mscorlib' when building the netcoreapp that uses WsdlService.
I have a repo demonstrating my attempt and screenshots in the README. https://github.com/gfritz/tp-apps
Can anyone identify how the WsdlService type provider needs to change to fix that, or if that is the problem? I can't figure out what in WsdlService makes that happen, but I'd be happy to try fixing it.
This issue documents how to enable the use of type providers when using a .NET SDK 2.0 "new style" project file.
This also allows the use of cross-targeting type providers as part of .NET Core 2.0 app programming or .NET Standard 2.0 library programming.
If you don't follow one of the workarounds below you may get a silent failure or
RunFsc.cmd failed
from the F# compiler when executingdotnet build
or "StackOverflowException", if your project references a type provider.What you need to do
Install an F# Compiler that runs on .NET Framework or Mono, see Linux, Windows, OSX. It is likely you already have one installed.
Add the file
fsc.props
to the root of your project and import it into each of your project files that reference a type provider using something along the lines of:<Import Project="fsc.props" />
,The F# compiler will then run using .NET Framework and/or Mono.
Alternative Workaround
On Windows:
On Linux/OSX/Xamarin/Mono:
Explanation: when you compile your project using
msbuild.exe
the F# support in the .NET SDK will assume .NET Framework execution of the F# compiler is required. This workaround is also compatible with using anfsc.props
file as mentioned above.Examples where the problem occurs
Example 1 (building a .NET 4.6.1 application that uses FSharp.Data)
Example 2 (building a .NET Core 2.0 application that uses FSharp.Data)
Expected behaviour is compilation success, or at least nice error point to workaround. Actual behavior is a silent compilation failure, and
dotnet build /v:d
shows lProcess is terminating due to StackOverflowException. Done executing task "Fsc" -- FAILED.
Explanation
As explained in this issue, type providers are not yet supported when the F# compiler runs using .NET Core. By default, the .NET SDK tooling runs the F# compiler using .NET Core.
The workaround is to have the .NET SDK tooling use an F# Compiler running on the .NET Framework or Mono, regardless of which version of .NET you're targeting. By doing this, the type provider is hosted and executed using the .NET Framework. This technique applies even if you are doing .NET Core programming (for cross-targeting type providers), or building a .NET Standard 2.0 library. The F# Compiler cross-generates code for the correct target regardless of how it executes.
Although this workaround may seem "wrong" if targeting .NET Core, it has advantages:
In the future, the plan of record is to allow .NET Standard 2.0 type providers to be used when the F# compiler is running on .NET Core. This is an interim workaround for those who wish to unblock the testing and use of type providers for .NET SDK project files and .NET Core/.NET Standard 2.0 programming.