Fix type provider to use new SDK

[ReedCopsey]

I merged @dsyme 's PR #72 in order to try to make this work against the new SDK -

However, it's failing, and currently not in a working state. Everything builds, but at runtime, the type provider is unusable, with an error stating: "The method or operation is not implemented."

It also appears that the attributes which cause the types to not be sealed (required for FsXaml) are not being honored properly.

I'm going to try to see if I can figure out what happened in the port - but I wanted to open an issue to document as I try to move the progress forward with this.

[ReedCopsey]

@dsyme So - I've narrowed down the problem: It appears that the behavior of constructors has changed.

When trying to add the code within the constructor to call the InitializeComponent () method (also defined within the type), the updated version fails with: "The method or operation is not implemented."

I'm not sure exactly why this is occurring - so I'd appreciate it if you took a look.

This issue occurs on this line: https://github.com/fsprojects/FsXaml/blob/master/src/FsXaml.Wpf.TypeProvider/XamlTypeUtils.fs#L100

I switched around the build to make it slightly easier for you to debug. If you build in debug, then open the FsXaml.Demos.sln solution (in root) in another VS instance, it'll pick up the debug assemblies and let you see the error directly. Opening the demos solution is the easiest way to confirm proper operation - it really hits all of the functionality, and won't build if the type provider is not implementing the proper operations.

[dsyme]

@ReedCopsey I've found a few more issues, will submit an update to the SDK soon

[dsyme]

@ReedCopsey The SDK is now updated with fixes that could have plausibly caused this. If you have time could you run paket update and submit another PR to try again? Thanks

[ReedCopsey]

@dsyme New error occurs now:

Type mismatch when building 'value': the type of the field was incorrect. Expected 'System.Boolean', but received type 'ctxt type System.Boolean'.

This occurs with various types throughout the type provider demo project, and causes each type to fail. I've pushed the updated version using the latest SDK, so feel free to grab the latest to repro.

(Note that the push includes https://github.com/fsprojects/FSharp.TypeProviders.SDK/pull/145 fixed locally, in order to get it to compile.)

[ReedCopsey]

@dsyme After more investigating - it's still a problem with the provided constructor.

The first exception that's raised while debugging is an index out of range exception on line 6284 of ProvidedTypes.fs. sigptr is 82, but the byte[] is length 82.

When walking back the stack trace, this happens inside of the invoke code for the constructor. I've noticed something odd, though (it may just be a debugger issue, but just in case..) the this parameter seems to be grabbing a provided method, not the type itself:

Just in case that helps narrow it down. Originally, the error was slightly different -so I moved the call to add the methods to the type before the contructor was called, which was when I noticed this.

[dsyme]

Thanks, have it on my list to look into

[dsyme]

I believe this issue is now fixed, see https://github.com/fsprojects/FSharp.TypeProviders.SDK/pull/146

[dsyme]

@ReedCopsey Please try the next TPSDK update, thanks

[ReedCopsey]

@dsyme Made some progress, but a similar error still:

Microsoft.FSharp.Compiler.TypeProviderError: 'Type mismatch when building 'args': invalid parameter for a method or indexer property. Expected 'System.Windows.FrameworkElement', but received type 'ctxt type System.Windows.Window'. Parameter name: receivedType'

(Note that System.Windows.Window is a subclass of FrameworkElement)

[jchidley]

I just cloned a fresh copy of FsXaml and build.cmd failed

     3>C:\Users\jackc\Documents\FsXaml\paket-files\build\fsprojects\FSharp.Type
       Providers.StarterPack\src\ProvidedTypes.fs(3100,16): error FS0039: The v
       alue or constructor 'isNull' is not defined. Maybe you want one of the f
       ollowing:   DBNull [C:\Users\jackc\Documents\FsXaml\src\FsXaml.Wpf.Type
       Provider\FsXaml.Wpf.TypeProvider.fsproj]
     3>Done Building Project "C:\Users\jackc\Documents\FsXaml\src\FsXaml.Wpf.Ty
       peProvider\FsXaml.Wpf.TypeProvider.fsproj" (Rebuild target(s)) -- FAILED
       .
     1>Done Building Project "C:\Users\jackc\Documents\FsXaml\FsXaml.sln" (Rebu
       ild target(s)) -- FAILED.

Build FAILED.

[ReedCopsey]

@jchidley This is expected until the new TP SDK is corrected.

@dsyme Any ideas on what might be causing this issue?

[ReedCopsey]

@dsyme So - with your latest PR (#150 - https://github.com/fsprojects/FSharp.TypeProviders.SDK/pull/150 ), things are moving forward.

The type provider builds correctly, and no longer gives intellisense errors.

However, at build time, new errors are occuring. They appear to be related to the constructor of the main Window types - I suspect it has to do with the fact that the constructor calls provided methods.

An example of the exact error set is:

1>output error FS0073: error : One of your modules expects the type 'Views.MainWindowBase' to be defined within the module being emitted. You may be missing an input file 1>output error FS0073: error : One of your modules expects the type 'Views.MainWindowBase' to be defined within the module being emitted. You may be missing an input file 1>FSC: error FS2014: A problem occurred writing the binary 'C:\Users\Reed\Documents\GitHub\FsXaml\demos\WpfMvvmAgent\obj\Debug\WpfMvvmAgent.exe': Error in pass3 for type Views.MainWindow, error: Error in GetMethodRefAsMethodDefIdx for mref = (".ctor", "Views.MainWindowBase"), error: Exception of type 'Microsoft.FSharp.Compiler.AbstractIL.ILBinaryWriter+MethodDefNotFound' was thrown.

Note that the code calling this is like the following:

type MainWindowBase = XAML<"MainWindow.xaml">

// This is to demonstrate being able to add in "code behind"
type MainWindow() =
    inherit MainWindowBase()
   // ... more stuff here to override and handle events

[ReedCopsey]

I've pushed this - if you want to repro, just pull, build FsXaml.sln in debug, then open the FsXaml.Demos.sln and try to build.

[ReedCopsey]

@dsyme I tried updating today, with hope that some of the fixes for swagger would help, but the behavior is identical to above.

[dsyme]

@ReedCopsey Thanks for trying, I'll take a look now

[dsyme]

You have to change

                    XamlTypeUtils.createProvidedType assembly ...

to

                    XamlTypeUtils.createProvidedType providedAssembly ...

since you are not placing the generated type into the right assembly

[ReedCopsey]

Thanks @dsyme 👍

The "simple cases" are building and working correctly now.

The remaining issues I'm hitting are related to types that are being used written from within the type provider, but from 3rd party (mostly WPF, but I think could be anywhere) assemblies.

It looks like the behavior around ResolveAssembly and/or how that's being written into the final assembly has changed (at least, that's my guess). I now receive errors like the following:

3>C:\Users\Reed\Documents\GitHub\FsXaml\demos\WpfMvvmAgent\MainWindow.xaml.fs(7,23): error FS3033: The type provider 'FsXaml.XamlTypeProvider' reported an error: The target type 'tgt type System.Windows.RoutedEventArgs' utilized by a type provider was not found in the design-time assembly set '[|FsXaml.Wpf.TypeProvider, Version=3.1.6.0, Culture=neutral, PublicKeyToken=611da0f4fbd84e16; mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089; FSharp.Core, Version=4.4.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; System.Xaml, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089; System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089; FsXaml.Wpf, Version=3.1.6.0, Culture=neutral, PublicKeyToken=611da0f4fbd84e16; PresentationFramework, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35; System.Xml, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089; fsxaml_0lzntwes.pmg|]'. Please report this problem to the project site for the type provider.

This occurs when the type provider reads the XAML, determines that (in this case) an event is being used, and wires up a method for the event. The event arg type (in this case RoutedEventArgs) is coming from PresentationCore, not my code, and being used in the resulting quotations.

I'm not sure how to expand the design time assembly set - in the past, this worked with the AssemblyResolve implementation in FsXaml, but it fails on two of the demo projects (the ones using events, and hence writing methods with these "external types").

Any idea on where I can look to get the assemblies into the "design-time assembly set" ? (I'm not sure exactly what that means...)

[ReedCopsey]

Note that this issue is unchanged as of today's SDK (https://github.com/fsprojects/FSharp.TypeProviders.SDK/pull/158 )

[ReedCopsey]

As of Nov 1st, this behavior is still occurring.

[ReedCopsey]

@dsyme I've been doing some experimentation - and I can "force this" to work, but I'm wary that it's a band-aid around this specific issue, and not a general purpose solution.

It will work if I explicitly add PresentationCore here: https://github.com/fsprojects/FsXaml/blob/master/src/FsXaml.Wpf.TypeProvider/XamlTypeProvider.fs#L33 by doing |> Seq.append [XamlTypeUtils.wpfAssembly ; typeof<System.Windows.RoutedEventArgs>.Assembly ]

This seems extremely suspicious, however, as config.ReferencedAssemblies includes PresentationCore.dll in its list of assemblies, and correctly loads. When I break in ResolveAssembly (line 127), the assemblies binding contains PresentationCore 2x. For some reason, when using the version coming from ReferenceAssemblies, this error occurs.

My fear is that this may be required for an event handler type that's custom within a user's referenced assemblies, in which case forcing it this way won't work. I don't understand why the current push isn't resolving, as PresentationCore does show up in the assembly list. Is there a different way I should be loading assemblies with the new approach/SDK? [This worked fine previously]

[dsyme]

@ReedCopsey Thanks for this, added as issue https://github.com/fsprojects/FSharp.TypeProviders.SDK/issues/173. It's definitely the fact the PresentationCore is not in the target assembly set that is causing the issue.

This seems extremely suspicious, however, as config.ReferencedAssemblies includes PresentationCore.dll in its list of assemblies, and correctly loads. When I break in ResolveAssembly (line 127), the assemblies binding contains PresentationCore 2x. For some reason, when using the version coming from ReferenceAssemblies, this error occurs.

• Can you write out repro steps?

• What is the path to PresentationCore.dll showing in debugger, and does that file exist on disk?

• Could you break at places like this and check whether PresentationCore is in the referencedAssemblyPaths and the corresponding file exists on disk?

• If it is present, then also break here and see if we successfully create a reader for it?

• If it is NOT present then maybe step through this logic to see what's going on https://github.com/fsprojects/FSharp.TypeProviders.SDK/blob/master/src/ProvidedTypes.fs#L8924

• Check for any internal exceptions in the code please, thanks

[ReedCopsey]

@dsyme See below:

Can you write out repro steps?

Using latest master:

1. Build FsXaml.sln main project.
2. Open FsXaml.Demos.sln. Errors occur.
3. Close FsXaml.Demos.sln.
4. In FsXaml, open XamlTypeProvider.fs
5. Comment line 33, uncomment line 34, Rebuild
6. Open FsXaml.Demos.sln. Errors no longer occur on build.

What is the path to PresentationCore.dll showing in debugger, and does that file exist on disk?

config.ReferencedAssemblies contains "C:\\Program Files (x86)\\Reference Assemblies\\Microsoft\\Framework\\.NETFramework\\v4.5\\PresentationCore.dll" , which does exist on disk.

Before adding the "fix", the assemblies list contains PresentationCore 1x, with CodeBase of "file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_32/PresentationCore/v4.0_4.0.0.0__31bf3856ad364e35/PresentationCore.dll"

After adding the "fix", the assemblies used for resolution later contains PresentationCore 2x, both with a CodeBase of "file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_32/PresentationCore/v4.0_4.0.0.0__31bf3856ad364e35/PresentationCore.dll", which is the same as before. The only difference I can see is it now sits near the head of the list, instead of further down inside of it:

From what I can see, the "two assemblies" are identical, which is why it's so confusing - I don't understand how adding that can correct the issue. Even looking at the properties of the assemblies in the debugger doesn't illustrate anything obvious.

Could you break at places like this and check whether PresentationCore is in the referencedAssemblyPaths and the corresponding file exists on disk?

Before and after duplicate check, it exists in the list with a path of: "C:\\Program Files (x86)\\Reference Assemblies\\Microsoft\\Framework\\.NETFramework\\v4.5\\PresentationCore.dll"

If it is present, then also break here and see if we successfully create a reader for it?

Yes, and it does show up in the target assemblies:

Check for any internal exceptions in the code please, thanks

The only exception that occurs (at least that I see) during the entire compilation phase is an expected one. It also occurs when it's working. System.Xml.ReaderWriter fails here: https://github.com/fsprojects/FsXaml/blob/master/src/FsXaml.Wpf.TypeProvider/XamlTypeProvider.fs#L30 , then it runs up to the point where I get the exception mentioned before:

Microsoft.FSharp.Compiler.TypeProviderError: The target type 'tgt type System.Windows.RoutedEventArgs' utilized by a type provider was not found in the design-time assembly set '[|FsXaml.Wpf.TypeProvider, Version=3.1.6.0, Culture=neutral, PublicKeyToken=611da0f4fbd84e16;
  mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;
  FSharp.Core, Version=4.4.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a;
  System.Xaml, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;
  System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;
  FsXaml.Wpf, Version=3.1.6.0, Culture=neutral, PublicKeyToken=611da0f4fbd84e16;
  PresentationFramework, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35;
  System.Xml, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;
  fsxaml_d3idkji0.upf|]'. Please report this problem to the project site for the type provider.

At this point, I'm completely stumped.

[ReedCopsey]

@dsyme Did you ever get a chance to look at this? FsXaml is still in an unusable state due to this one remaining issue - and I'm not sure how to proceed.

[MaxWilson]

Are there any workarounds? I'm trying to run the Advent Trees 2017 example and while XAML designer support is not essential I'm curious to see what the experience is supposed to be.

[ReedCopsey]

@MaxWilson This issue should only impact building FsXaml from source. Advent Trees should be usable fine since those projects all use the (working) nuget package.

[MaxWilson]

Hmmm. AdventTrees gives me "MissingMethodException: Method not found: 'Void FsXaml.EventArgsConverter2..ctor(Microsoft.FSharp.Core.FSharpFunc2<!0,!1>, !1)'." when I open MainWindow.xaml. I thought that was the same as this issue. I blame Google. :)

[ReedCopsey]

Are you in VS 2015 with F# power tools? if so, its a known issue w/ power tools, which you can temporarily disable.

On Dec 26, 2017 8:01 PM, "Max Wilson" notifications@github.com wrote:

Hmmm. AdventTrees gives me "MissingMethodException: Method not found: 'Void FsXaml.EventArgsConverter2..ctor(Microsoft.FSharp.Core.FSharpFunc2<!0,!1>, !1)'." when I open MainWindow.xaml. I thought that was the same as this issue.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/fsprojects/FsXaml/issues/73#issuecomment-354048100, or mute the thread https://github.com/notifications/unsubscribe-auth/AFCUmkiw9hz63ZjTgzMKbV5VahJyNWe6ks5tEcExgaJpZM4P3XWE .

[MaxWilson]

I'm in VS 2017. Sorry if this is the wrong place for this discussion--I'm willing to move discussion elsewhere if that's more appropriate.

P.S. I found a workaround: deleting the EventArgsConverter property permits the XAML designer to load, and then un-deleting it leaves the designer still in a working state.

[BillHally]

I think I've got some useful information about this issue. However, I don't know this code well. For example, when I talk about "source" and "target" assembly sets, I'm referring only to the names of the bindings involved - I've only a vague idea of what the intended behavior is.

Anyway, I think I've spotted at least part of the problem.

@dsyme said:

It's definitely the fact the PresentationCore is not in the target assembly set that is causing the issue.

However, the error occurs when the assembly is missing from the source assembly set, rather than the target assembly set.

@ReedCopsey said:

After adding the "fix", the assemblies used for resolution later contains PresentationCore 2x, both with a CodeBase of "file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_32/PresentationCore/v4.0_4.0.0.0__31bf3856ad364e35/PresentationCore.dll"

that's because the code of the fix - as well as adding the assembly to the target set for the 2nd time - has the side effect of ensuring that the compiler doesn't discard the reference to PresentationCore which the FsXaml.Wpf.TypeProvider assembly has.

Without the fix, loading the type provider assembly and dumping its assembly references gives:

> System.Reflection.Assembly                   
-     .LoadFrom("FsXaml.Wpf.TypeProvider.dll") 
-     .GetReferencedAssemblies()               
-     |> Array.map (fun x -> x.Name)           
-     |> Array.sort                            
-     |> Array.iter (printfn "%s");;           
FSharp.Core                                    
FsXaml.Wpf                                     
PresentationFramework                          
System                                         
System.Xaml                                    
System.Xml                                     
mscorlib

Whereas With the fix, gives:

> System.Reflection.Assembly
    .LoadFrom("FsXaml.Wpf.TypeProvider.dll")
    .GetReferencedAssemblies()
    |> Array.map (fun x -> x.Name)
    |> Array.sort
    |> Array.iter (printfn "%s");;
FSharp.Core
FsXaml.Wpf
PresentationCore
PresentationFramework
System
System.Xaml
System.Xml
mscorlib

Since its this type provider assembly (via Assembly.GetExecutingAssembly()) which is used to generate the source assembly set, this is enough to provide the "fix".

This can be demonstrated by ensuring the reference endures with different code. For example, by adding the following immediately below the initialization of assemblies in XamlTypeProvider.fs:

static let __ = System.Windows.RoutedEventArgs()

This doesn't add PresentationCore to the assemblies binding, but it does add it to the assemblies referenced by the type provider assembly itself - and hence also provides the "fix".

[dsyme]

@ReedCopsey @BillHally From what you've said here, it looks as though it may be necessary to do a transitive closure of GetReferencedAssemblies() to determine the correct full set of source assemblies?

[dsyme]

There is a proposed fix for this issue in https://github.com/fsprojects/FSharp.TypeProviders.SDK/pull/187

[ReedCopsey]

@dsyme Looks like this does it. Thanks!