TypeProvider not working with dotnet core 3.0

[tforkmann]

Description

I'm trying to use the typeprovider with netcoreapp3.0 (3.0.100). But it is not working for me. I alread tried to reference System.Data.SqlClient. Ionide is failing as well.

Repro steps

1. Set up Dotnet core app 3.0

2. Add SqlProvider

3. Try to get datacontext

open FSharp.Data.Sql
open System
open Config

/// Query DME
let [<Literal>]  CompileTimeConnectionString = "myconnectionString"

type SqlDme = SqlDataProvider<Common.DatabaseProviderTypes.MSSQLSERVER, ConnectionString=CompileTimeConnectionString, UseOptionTypes=true> //, ContextSchemaPath = ContextSchemaPath>

Expected behavior

TypeProvider should just work

Actual behavior

I get following error message when building the project: Could not load file or assembly 'System.Data.SqlClient, Version=4.6.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Das System kann die angegebene Datei nicht finden. [C:\Users\tforkmann\Documents\1_Tests\PreveroFeed\src\Server\Server.fsproj]

Known workarounds

1. Add FSharp.Compiler.Tools
2. Add to fsproj file <DotnetFscCompilerPath></DotnetFscCompilerPath>

Related information

• MS Sql
• Windows 10
• netcoreapp3.0

[forki]

Id there a general strategy for type providers in dotnet sdk 3.x? Since we don't get any new fsharp.compiler.tools packages anymore - what's the replacement?

/cc @dsyme @cartermp

[Bjorn-Strom]

Having this same problem on mac.

[cartermp]

@forki There is no replacement, as that was never a supported scenario in the first place. I'd encourage never installing that package.

[forki]

Reality is that this was the only way to get things working. But it's not really important anymore. The question is how can we proceed?

[cartermp]

Realistically there's probably a bug in either the type provider, the provider sdk, or F# compiler that needs to be resolved. Using an unsupported package containing a very old .NET Framework-only version of the F# compiler isn't a scenario you should be considering here.

[Thorium]

I'm happy to approve PRs (and release new packages). But I know nothing about .NET Core 3.0, I still use .NET Framework. For example I have no clue do you still need to set DotnetFscCompilerPath or not, to resolve the duplicate fsc problem.

This type provider loads DB drivers via reflection (to not depend every DB), which is a challenge in Core as the dependencies are broken down to many dlls that are loaded from unknown cache location.

Here is an example typeprovider from the TypeProvider SDK, referring NETStandard.Library.NETFramework, I guess it's outdated as well. However if you don't do that, loading different versions of .NET strandard.dll will cause a loop in reflection.

[tforkmann]

Today I worked a bit on the problem.

I created a fork of the sqlprovider and only focused on the netcore version. I got the compile compiling on NetCore 3.0. The only thing missing is that the SqlClient needs to be present before compile time. With the Compiler Tools I didn’t need to add anything. I don’t understand what the compiler tools is doing differently to make the Client available before compile time

[mcspud]

For example I have no clue do you still need to set DotnetFscCompilerPath or not, to resolve the duplicate fsc problem.

Nope!

@tforkkmann FWIW I can get this to run seamlessly on Ubuntu 16.04 and OSX using .net core 3.0.100, npgsql and postgres 11 and VS Code as long as I set the resolution paths up from the "unknown cache directory".

I am having lots of issues with SaveContextSchema() though, but I've referenced that elsewhere.

[tforkmann]

@mcspud Which directory are you adding as a resolution folder? The Package folder?

[mcspud]

Hey @tforkmann - yep. I'm using the osx package location,

let [<Literal>] ConnectionString =
    "Server=localhost;" +
    "Database=jabronis;" +
    "User Id=localdev;" +
    "Password=password;"

let [<Literal>] DatabaseVendor = Common.DatabaseProviderTypes.POSTGRESQL
let [<Literal>] CaseSensitivityChange = Common.CaseSensitivityChange.ORIGINAL
let [<Literal>] ContextSchemaPath = __SOURCE_DIRECTORY__ + "/.postgres.schema.json"
let [<Literal>] ResolutionPath = @"~/.nuget/packages/Npgsql/4.1.1/lib/netcore3.0"

type Sql =
    SqlDataProvider<
        ConnectionString=ConnectionString,
        DatabaseVendor=DatabaseVendor,
        CaseSensitivityChange=CaseSensitivityChange,
        ResolutionPath=ResolutionPath>

And from my .fsproj:

...
  <TargetFramework>netcoreapp3.0</TargetFramework>
<ItemGroup>
    ...
    <PackageReference Include="Npgsql" Version="4.1.1" />
    <PackageReference Include="SQLProvider" Version="1.1.68" />
  </ItemGroup>

That .nuget path is the default location for .net core packages on mac.

This all works nicely, but on VSCode intellisense can be pretty flakey often times requiring me to restart it to get it kick back in. It doesn't work at all on Rider for the generative type provider. I also can't successfully export the schema so I can't actually build the project in a docker image on a CI/CD environment lol :)

[jl0pd]

Hello everyone. I'm new to F# and learning it.

I've solved problem by adding System.Data.SqlClient to project:

<PackageReference Include="System.Data.SqlClient" Version="4.8.0" />

but got this:

The most strange thing - I can create an instance of IAsyncDisposable:

let a = { new IAsyncDisposable with
    member __.DisposeAsync() = ValueTask()}

How to make it work?

Configuration

• Dotnet core SDK v3.0.101
• Linux mint 19.1
• Ionide-fsharp 4.3.2

[Thorium]

Hi,

I just updated the package to contain System.Data.SqlClient of package 4.8.0 Hopefully helps.

@jl0pd just a guess, sounds like your compile-time version of System.Runtime.dll doesn't match the execution time version of System.Runtime.dll

[tforkmann]

Hi,

i tried with the new SqlProvider version.

Somehow it still asks for the System.Data.SqlClient version 4.6.1

Can you maybe update the 4.6.1 version as well?

[tforkmann]

I also saw that the System.Data.SqlClient doesn't has a net core 3.1 version.

We probably need to migrate to Microsoft.Data.SqlClient https://devblogs.microsoft.com/dotnet/introducing-the-new-microsoftdatasqlclient/

[Thorium]

System.Data.SqlClient are one of the few which are currently not loaded via reflection...

[Thorium]

I think it's version 4.6.1? Or 4.8? Is there a problem? Do you do binding redirects?

[tforkmann]

I have 4.8 in my paket.lock. Is binding redirects required?

[Thorium]

I think I updated it to 4.8, but I may have missed something.

https://github.com/fsprojects/SQLProvider/blob/4d79debfc59fbba1e3fae1e816d1d9309bbfe012/paket.lock#L152

[tforkmann]

Hmm, ok. Updated to 1.1.82. Still the same errror. Why are you using the FSharp.Compiler.Tools now? Is it still required?

[tforkmann]

Ok, it's compiling and running but ionide shows that error. Maybe it is just an Ionide issue?!

[Thorium]

I was planing to use FSharp.Compiler.Tools because people have here lot of compilation issues and some of them come by using wrong fsc.exe, so I thought maybe the compiler tools would provide a consistent version. However it's not used right now.

[cartermp]

I would highly recommend not using FCT. Chiefest of issues being that there will forever be a delta between what you compile with and what your IDE things you're doing.

[njeisecke]

I'm trying this on macOS. After getting rid of Mono and Visual Studio Mac, with only pure .net code 3.1 SDK installed it started to work.

• I've setup the test project roughly following https://docs.microsoft.com/en-us/dotnet/fsharp/get-started/get-started-command-line.
• added the packages "SQLProvider" and "MySqlConnector" using "dotnet add package"
• seems to work without an explicit ResolutionPath (for MySql)

Visual Studio Code:

• Intellisense with ionide in VSC seems to work ok (db schema changes require a restart)
• Ionide: set "Fsac Runtime" to netcode
• Ionide: set "Use SDK Scripts" to true (so fsi works with .net core)

So it looks like the F# compiler (4.5, outdated anyway) from Mono does not work and mixing mono build environment with .net code runtime environment is a bad idea. All the obviously outdated documentation ("how to get startet with f#") from Microsoft is really irritating.

Hope this helps.

[Thorium]

@njeisecke, I'm confused... I thought type providers still need Mono to compile. Has this changed with .NET Core 3.1?

[njeisecke]

Seems to work. I removed all of the mono stuff and use the "dotnet" command for everything. It builds and runs.

[cartermp]

Type Providers have not needed mono to compile for quite a long time. There's no reason to require .NET Framework of any flavor unless you specifically require an API that has no alternative on .NET Standard.

[Thorium]

Type Providers have not needed mono to compile for quite a long time.

Wow, cool. :-D All these undocumented features that I'm not aware of.

[tforkmann]

@njeisecke Could you maybe create a project example? And share it with us? I'm still not able to get SqlClient to work.

[gibranrosa]

Hi,

i tried with the new SqlProvider version.

Somehow it still asks for the System.Data.SqlClient version 4.6.1

Can you maybe update the 4.6.1 version as well?

Hi I am getting this error with Ionide too, on Windows 10, dotnet 3.1. It compiles but Ionide shows the error.

[tforkmann]

jup!

[njeisecke]

https://github.com/njeisecke/fsharp-sqlprovider

Only .net 3.1 core SDK (https://dotnet.microsoft.com/download/dotnet-core/3.1) was installed.

[Thorium]

ok, the Ionide error comes from the old dept.json file...what does generate that...

[Thorium]

System.Data.SqlClient package 4.8.0 does have a file System.Data.SqlClient.dll version 4.6.1.0 So that should be correct. Can that be some caching issue of Ionide? Would dotnet.exe restore -f help?

[tforkmann]

Hey,

I tried it with dotnet.exe restore -f sadly no success. Could we try to use Microsoft.Data.SqlClient instead?

[Thorium]

@tforkmann , Probably, although it supports only .NET Framework 4.6 when the current SQLProvider works with very old .NET Frameworks as well. Do you want to try a PR?

On the other hand, I think there is no point of creating the SQLProvider load *.Data.SQLClient -library via reflection (like current MySql. Postgres, Oracle, etc. does), because the whole reflection have had so many issues of dependencies libraries of the database drivers. (.NET Core references are like nodemodules)

So rather we could have separate NuGet packages, for example: System.Data.SQLProvider.MSSQL (current System.Data) System.Data.SQLProvider.MicrosoftSQL (New) System.Data.SQLProvider.Postgres System.Data.SQLProvider.MySQL System.Data.SQLProvider.SQLite System.Data.SQLProvider.Oracle System.Data.SQLProvider.Odbc System.Data.SQLProvider.MsAccess ...

But I struggle to figure out what should be the most easiest way to create these deployment packages:

• A solution for each one to include the same source files except the database specific file?
• Or create a new base SQLProvider project and then include the database reference assemblies to corresponding projects?
• Or could it be just the old SQLProvider-core-project (100% backward comatible) and then create reference assemblies as meta-packages? But the meta-packaging doesn't work in F#: https://github.com/dotnet/fsharp/issues/6305

And changing the years-old packaging-script (and build-script) will probably cause new issues. However, it seems that most people struggle with the current build script anyway.

What do you think?

[tforkmann]

I personally think we should start from scratch with the official typeprovider template:

dotnet new -i FSharp.TypeProviders.Templates
dotnet new typeprovider -n LemonadeProvider -lang F#

and then only focus on one database provider at first and get it running on dotnet core 3.1. without FCT.

[Thorium]

Should that be build on which .NET Standard frameworks?

Also I'm not sure if the LemonadeProvider is up-to-date starting-point, as it .NET Standard 2.0 package NETStandard.Library.NETFramework : "This package is deprecated." ?

https://github.com/kevmal/tpsub/blob/4f1cb43d318925787403612c1f411a38a0d6a639/src/LemonadeProvider.DesignTime/LemonadeProvider.DesignTime.fsproj#L26

[tforkmann]

@isaacabraham What do you think is a good starting point?

[tforkmann]

Hi guys,

I spent some time and created a fork of the typeprovider targeting netcoreapp3.1 only and using the new Microsoft.Data.SqlClient. The typeprovider builds ok, but the tests are not working yet.

I get following error message:

Could not load file or assembly 'C:\Users\tforkmann\Documents\1_Tests\SqlClientTypeProvider\src\SqlClientTypeProvider.Runtime\bin\Debug\netcoreapp3.1\SqlClientTypeProvider.DesignTime.dll'. Das System kann die angegebene Datei nicht finden. [C:\Users\tforkmann\Documents\1_Tests\SqlClientTypeProvider\tests\SqlClientTypeProvider.Tests\SqlClientTypeProvider.Tests.fsproj]

Maybe someone has an idea how to fix this.

Here is the link to the respository: https://github.com/tforkmann/SqlClientTypeProvider

[aggieben]

I ran into this problem tonight as well, and I seem to have struggled my way out of it, but I don't really know why.

First: dotnet build was failing, and FSAC in Ionide was also failing. It was set to use netcore. Then I tried building with MSBuild instead of dotnet build, and the build worked.
Finally I tried setting the FSAC setting in Ionide to net instead of netcore, and that seems to be working now, too.

I also removed the PreBuild target I had set up in my .fsproj, and the above still holds true.

[tforkmann]

Holly cow! Just switching to FSAC setting in Ionide to 'net' actually works!

No error message anymore!

[isaacabraham]

The days of the net FSAC are numbered. This is the same issue that the SQL Client type provider has - it works with msbuild but not with dotnet.

[tforkmann]

Ok! What could we do?

[estensen]

Is there a workaround for SQL Server?

This seems to be the original problem

[jkone27]

Hello for SQL Server, a simple workaround is just to add in the .csproj the package reference for System.Data.SqlClient (or via nuget manager). This way I got it "working" in netcoreapp31 (also tested in netcoreapp30 works)

  <ItemGroup>
    <PackageReference Include="SQLProvider" Version="1.1.84" />
    <PackageReference Include="System.Data.SqlClient" Version="4.8.1" />
  </ItemGroup>

on windows OS

[Ecofred]

@estensen dotnet/fsharp#6326 may help you

I struggled for quite a while, to make SqlProvider work on Ubuntu/VSCode/netcore3.1 and in the end, the FSAC to net , mono fsharpc and

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp3.1</TargetFramework>
    <!-- Here starts the trick -->
    <FscToolPath>/usr/bin</FscToolPath>
    <FscToolExe>fsharpc</FscToolExe>
    <DotnetFscCompilerPath></DotnetFscCompilerPath>
  </PropertyGroup>

  <ItemGroup>
    <Compile Include="Program.fs" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="SQLProvider" Version="1.1.84" />
    <PackageReference Include="System.Data.SqlClient" Version="4.8.1" />
  </ItemGroup>

</Project>

did the work.

This workaround is also working with FSharp.Data.SqlClient by the way (because I needed to query sys schema table, and I still can't get it with SQLProvider)

[granicz]

@jkone27 I tried what you suggested but it didn't work for me. I have a netcore3.1 websharper-web app, where I added SQLProvider, then a reference to System.Data.SqlClient manually as you suggested. This all typechecks just fine in VS2019 but fails to compile (I understand that there is a"discrepancy issue" between msbuild and dotnet build - but this makes me not any happier), getting the same as without the manual reference:

typecheck error FS3033: The type provider 'FSharp.Data.Sql.SqlTypeProvider' reported an error: Could not load type 'System.Data.SqlClient.SqlConnection' from assembly 'System.Data.SqlClient, Version=4.6.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.

[Thorium]

@cartermp promised to look at this a bit, meanwhile I'll happily stay in the .NET Framework, I can provide a little help on explaining some things:

• ProvidedTypes.fs is loaded via paket directly, I don't want to touch or customise that file. If you want to update the latest TypeProvider.SDK it's done via: paket.exe update --group SourceFiles When I tested that earlier, I had the problem with the latest SDK that when loading reflection, some drivers were dependant of different versions of netstandard.dll, and the ProvidedTypes.fs started to load the base-types in infinite loop.

• Currently there is in src folder 2 different project files: SQLProvider for .NET Framework, and SQLProvider.Standard for .NET Standard 2.0. They both are using the same source files (the later one via linking <Compile Include="..\*.fs">. The idea with that was to maintain full backward compatibility with .NET Framework because this project is used by some enterprises in production. If we get a working solution, we can create a new one and throw these away, or however you like.

• Probably you don't need to touch the source files a lot, there are not many conditional compilations. But if you do, here is some helps how the system works: http://fsprojects.github.io/SQLProvider/core/techdetails.html

• The provider was created in a world where database drivers were single dll files. You just copied the driver to ResolutionPath-folder (given as a static parameter to the provider), and that's it. The reason to avoid hard-coded dependencies was: We didn't want to depend e.g. Oracle.dll because you might never use Oracle, but Postgres instead. Single-dll-drivers is not the world we live in anymore, so the dependency loading might be an issue: .NET Reflection wants dependency dlls to be near, and they are loaded both design-time, and runtime. To find correct dlls in all the phases could be a bit tricky, because they come from different NuGet-packages. Loading the first dll .NET will trigger a ResolveEvent for related dlls. The optimal end-user experience would be no ResolutionPath but just add the NuGet packages. That's why the comment https://github.com/fsprojects/SQLProvider/issues/645#issuecomment-585188011

[Thorium]

I created new project files to net-standard branch. This is done:

• Separate Runtime and DesignTime dlls.
• Fake 5, compiles on .NET Core, both VS2019 and console.

Note, this is initial commit, not yet "I'll just move to this". This is not done:

• Other things in build-script: Build & run tests. Documentation building. CI.
• The reference loading is as painful as before. I guess the easiest way to get the user experience perfect would be creating extra-packages like SQLProfiler.ReferenceAssemblies.Postgres that would be just basically a reference to other assemblies and a script to collect dlls to the resolution path. So when you'd reference the SQLProvider and then selected assemblies-package, the build would just work.

[cartermp]

@Thorium have you been able to get the scrip tests to work? I ran into this with the Azure Storage TP - got the provider factored correctly and it works fine on .NET Core and .NET Framework when I include it manually. But the test infrastructure relies on some F# scripts that won't work without .NET 5 and some of the recent bug fixes in F# script dependency management. Unfortunately, it seems that the Azure Storage TP repo may need to wait until .NET 5 to fully update. Looking at this repo it may be the same.