search

fsharp / fsharp-compiler-docs

Doc build for FSharp.Compiler.Service
https://fsharp.github.io/fsharp-compiler-docs
MIT License
279 stars 123 forks source link

FSharp.Compiler.Service dependency on MSBuild #631

Closed nightroman closed 7 years ago

nightroman commented 8 years ago

As far as I know, project related tools moved to FSharp.Compiler.Service.ProjectCracker. At the same time, FSharp.Compiler.Service still depends on MSBuild and cannot be loaded and used without MSBuild installed.

ILSpy shows the following references:

Microsoft.Build.Framework, Version=12.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
Microsoft.Build.Tasks.v12.0, Version=12.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
Microsoft.Build.Utilities.v12.0, Version=12.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a

I wonder if this dependency is by design or not. In the latter case, it would be nice if this dependency is removed. FSharp.Compiler.Service is useful on its own without MSBuild related stuff.

tpetricek commented 8 years ago

I was just going to report this too - Fable, which uses FCS 6.0.2 still does not work on machine with just VS 2015 (you need to install MSBUILD 12 to be able to run it).

tpetricek commented 8 years ago

There was earlier discussion about this, but apparently, this is still an issue: https://github.com/fsharp/FSharp.Compiler.Service/issues/337

7sharp9 commented 8 years ago

Isn't msbuild used for reference resolution too, except in the case of mono which uses simple resolution.

nosami commented 8 years ago

Yeah it's used for reference resolution. Would really like to drop the MSBuild deps though if possible. Ideally, the reference resolution should be done before any FCS interactions.

On 11 Sep 2016 7:22 p.m., "Dave Thomas" notifications@github.com wrote:

Isn't msbuild used for reference resolution too, except in the case of mono which uses simple resolution.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/fsharp/FSharp.Compiler.Service/issues/631#issuecomment-246192018, or mute the thread https://github.com/notifications/unsubscribe-auth/AAouOiA8s_jDTtRX7ufCvVIMcipKiE1Hks5qpDi6gaJpZM4J5yZq .

tpetricek commented 8 years ago

It would be great to solve this - as more and more projects use FSC and not many people use VS 2013, this is a growing issue. If we don't solve this before Progressive F# tutorials in London, I bet many people coming to Suave and Fable tutorials will be affected by this (with Suave, FSC can provide nice live-reloading experience and Fable relies on this directly).

What is a minimal viable solution to make sure we do not give really crappy experience? Should I recover my PR to bundle the DLLs as part of the package?

7sharp9 commented 8 years ago

I wonder why its so hard to rip out, desperately need a #NO_MSBUILD flag to purge it so reference behavior is mono stylee.

tpetricek commented 8 years ago

(more discussion about the earlier PR is here https://github.com/fsharp/FSharp.Compiler.Service/pull/338 - yes, it's not a nice solution, but I think we need some solution)

dsyme commented 8 years ago

Let's just factor the dependency into another DLL (so you can opt in to Visual F#-Tools compatible MSBuild support).

dsyme commented 8 years ago

@nightroman @tpetricek @7sharp9 @nosami See https://github.com/fsharp/FSharp.Compiler.Service/pull/649, please take a look over this.

nightroman commented 8 years ago

@dsyme, it looks very promising, thank you very much! This approach, with some minor adjustments on my side, should work for FSharpFar. This module supports two configuration types, .fsproj and .fs.ini. The latter is effectively just arguments of fsc and fsi in a friendly format with some helpers like environment variables expansion. As far as understand, .fs.ini scenario will be able to work without MSBuild. Developers of F# scripts for Far Manager may still use .fsproj with MSBuild / Visual Studio / VS Code. But F# scripts released for Far Manager users come with .fs.ini, so that users do not have to install MSBuild. This is exactly what I needed.

tpetricek commented 8 years ago

@dsyme Thank you for looking into this!! Solving this will be fantastic. I should be able to look once I'm done with the training I'm doing over the next two days!

nightroman commented 8 years ago

@tpetricek, it would be interesting to know if it works for you. So I far I cannot make it working and I am not sure if it is my fault or not. Microsoft.Build.Framework.dll is loaded even if I opt out, if it is present, and it fails if it is not.

I am trying the latest NuGet package 8.0.0 with FSharpChecker(msbuildEnabled=false) and FsiInteractiveSession(msbuildEnabled=false)

dsyme commented 8 years ago

@nightroman Could you submit repro steps? In particular, which tool is hosting FSHarp.Compiler.Service?

nightroman commented 8 years ago

@dsyme The tool is FSharpFar and the FCS scenarios that it uses are briefly described four comments above.

Let me put the latest changes of FSharpFar in order and release its version that uses FCS 8.0.0 and opts out of MSBuild in cases of .fs.ini configuration. Then I will describe how to get Far Manager, FarNet, and FSharpFar and repro steps.

Here is the summary of the used scenario. When FSharpFar is told to open an interactive session or perform F# checks with .fs.ini configuration then it uses FsiInteractiveSession(msbuildEnabled=false) or FSharpChecker(msbuildEnabled=false). It takes arguments for fsc from the .fs.ini file and does not call the project cracker. Nevertheless, Microsoft.Build.Framework.dll is loaded in this scenario.

nightroman commented 8 years ago

@dsyme

I am trying to investigate. Here is an interesting result.

I added this brutal check and exception:

FSharp.Compiler.Service\src\fsharp\SimulatedMSBuildReferenceResolver.fs(168):

    let internal GetBestAvailableResolver(msbuildEnabled: bool) =
        if msbuildEnabled then invalidOp "unexpected msbuildEnabled"

Then I ran my case with msbuildEnabled = false. I got the exception which I kind of expected. Here is the stack:

    TypeInitializationException:
    The type initializer for 'Microsoft.FSharp.Compiler.SourceCodeServices.FSharpChecker' threw an exception.

    TypeInitializationException:
    The type initializer for '<StartupCode$FSharp-Compiler-Service>.$Service' threw an exception.

    InvalidOperationException:
    unexpected msbuildEnabled

    System.TypeInitializationException: The type initializer for 'Microsoft.FSharp.Compiler.SourceCodeServices.FSharpChecker' threw an exception. ---> System.TypeInitializationException: The type initializer for '<StartupCode$FSharp-Compiler-Service>.$Service' threw an exception. ---> System.InvalidOperationException: unexpected msbuildEnabled
       at Microsoft.FSharp.Compiler.SimulatedMSBuildReferenceResolver.GetBestAvailableResolver(Boolean msbuildEnabled)
       at Microsoft.FSharp.Compiler.SourceCodeServices.FSharpChecker.Create(FSharpOption`1 projectCacheSize, FSharpOption`1 keepAssemblyContents, FSharpOption`1 keepAllBackgroundResolutions, FSharpOption`1 msbuildEnabled)
       at <StartupCode$FSharp-Compiler-Service>.$Service..cctor()
       --- End of inner exception stack trace ---
       at Microsoft.FSharp.Compiler.SourceCodeServices.FSharpChecker..cctor()
       --- End of inner exception stack trace ---
       at Microsoft.FSharp.Compiler.SourceCodeServices.FSharpChecker.Create(FSharpOption`1 projectCacheSize, FSharpOption`1 keepAssemblyContents, FSharpOption`1 keepAllBackgroundResolutions, FSharpOption`1 msbuildEnabled)
       at Microsoft.FSharp.Compiler.Interactive.Shell.FsiEvaluationSession..ctor(FsiEvaluationSessionHostConfig fsi, String[] argv, TextReader inReader, TextWriter outWriter, TextWriter errorWriter, Boolean fsiCollectible, Boolean msbuildEnabled)
       at Microsoft.FSharp.Compiler.Interactive.Shell.FsiEvaluationSession.Create(FsiEvaluationSessionHostConfig fsiConfig, String[] argv, TextReader inReader, TextWriter outWriter, TextWriter errorWriter, FSharpOption`1 collectible, FSharpOption`1 msbuildEnabled)
       at FSharpFar.Session.Session..ctor(String configFile)
       at FSharpFar.Session.Session.FindOrCreate(String path)
       at FSharpFar.FarCommand.Invoke(Object sender, ModuleCommandEventArgs e)
       at FarNet.Far0.InvokeCommand(Char* command, Boolean isMacro)
       at FarNet.Far0.AsOpen(OpenInfo* info)
       at OpenW(OpenInfo* info)

The static initializer

FSharp.Compiler.Service\src\fsharp\vs\service.fs(2721):

    static let globalInstance = FSharpChecker.Create()

calls Create with the default parameters, no matter what I actually provide in my other calls.

As a result, the available MSBuild gets loaded even though I opt out. I am not telling that it is used for reference resolution but it is loaded.

Before I go further, I would like to know if this is by design or not.

The above does not mean that it all is not working on a machine without MSBuild. Why it fails on my machine without MSBuild is yet to be found, please ignore for now.

nightroman commented 8 years ago

I cannot tell exactly what is wrong on a machine without MSBuild. But according to procmon, in spite of msbuildEnabled = false, Microsoft.Build.Framework is searched in the registry and GAC.

It fails in try/with here

FSharp.Compiler.Service\src\fsharp\fsi\fsi.fs:

    /// The single, global interactive checker that can be safely used in conjunction with other operations
    /// on the FsiEvaluationSession. 
    let checker = FSharpChecker.Create()

    let (tcGlobals,frameworkTcImports,nonFrameworkResolutions,unresolvedReferences) =
        try
            let tcConfig = tcConfigP.Get()
            checker.FrameworkImportsCache.Get tcConfig
        with e ->
            stopProcessingRecovery e range0; failwithf "Error creating evaluation session: %A" e

I also tried to buils and test with my change of the global checker Create

    let checker = FSharpChecker.Create(msbuildEnabled = false)

The result/exception is the same.

Now it is getting too cryptic for me. I can make some changes in FCS, build, and try on a "clean" test machine (I have just one). If you want me to make some diagnostic changes and test them, please let me know.

dsyme commented 8 years ago

@nightroman Thanks for the update.

On the first question: using the global FSharpChecker.Instance is deprecated. You should be receiving an "Obsolete" warning when using that property. Please create an FSharpChecker choosing whether to enable msbuild (meaning enable-if-installed) or not

On the second question. If you are not using the ProjectCracker, and you are using msbuildEnabled = false, then you should never be resolving Microsoft.Build.Framework (unless you are actually referencing that in your F# compilation that you are requesting, in which case we will search on disk for that DLL).

Any chance you can list repro steps? I'd be happy to dig into it to help land this cleanly.

Are you using the ProjectCracker?

nightroman commented 8 years ago

@dsyme On the first question: please note that I do not use deprecated FSharpChecker.Instance. But it is always created by the FCS library itself in the static initializer of FSharpChecker. Thus MSBuild is always loaded if it is present, even if I opt out.

The static initializer

FSharp.Compiler.Service\src\fsharp\vs\service.fs(2721):

    static let globalInstance = FSharpChecker.Create()

calls Create with the default parameters, no matter what I actually provide in my other calls.

Are you using the ProjectCracker?

No. I have the dependency but I do not call it in scenarios related to this topic. MSBuild should not be loaded in my scenarios, per my understanding of 8.0.0 and my use of msbuildEnabled = false. But it happens to be loaded.

dsyme commented 8 years ago

But it is always created by the FCS library itself in the static initializer of FSharpChecker. Thus MSBuild is always loaded if it is present, even if I opt out.

Ah, I see. On a machine without MSBuild this attempted load should not cause a failure since it gets caught. So some other exception is happening. Can you share the stack trace and details of the exception?

nightroman commented 8 years ago

It does not fail in Create(). As I wrote above, it fails in try/with here

FSharp.Compiler.Service\src\fsharp\fsi\fsi.fs:

    /// The single, global interactive checker that can be safely used in conjunction with other operations
    /// on the FsiEvaluationSession. 
    let checker = FSharpChecker.Create()

    let (tcGlobals,frameworkTcImports,nonFrameworkResolutions,unresolvedReferences) =
        try
            let tcConfig = tcConfigP.Get()
            checker.FrameworkImportsCache.Get tcConfig
        with e ->
            stopProcessingRecovery e range0; failwithf "Error creating evaluation session: %A" e

That is all I can currently get, not much, unfortunately.

nightroman commented 8 years ago

I will remove try/with, build, and try again on my the only "clean" machine. Hopefully, we will get the stack. I am sorry that my ways to diagnose this issue are very limited.

dsyme commented 8 years ago

ok thanks

nightroman commented 8 years ago

Here is the call stack

StopProcessingExn:
Exception of type 'Microsoft.FSharp.Compiler.ErrorLogger+StopProcessingExn' was thrown.

Microsoft.FSharp.Compiler.ErrorLogger+StopProcessingExn: Exception of type 'Microsoft.FSharp.Compiler.ErrorLogger+StopProcessingExn' was thrown.
   at Microsoft.FSharp.Compiler.Interactive.Shell.ErrorLoggerThatStopsOnFirstError.ErrorSinkHelper[a](PhasedError err) in C:\-\GIT\fs-proj\FSharp.Compiler.Service\src\fsharp\fsi\fsi.fs:line 465
   at Microsoft.FSharp.Compiler.Interactive.Shell.ErrorLoggerThatStopsOnFirstError.ErrorSinkImpl(PhasedError err) in C:\-\GIT\fs-proj\FSharp.Compiler.Service\src\fsharp\fsi\fsi.fs:line 479
   at Microsoft.FSharp.Compiler.ErrorLogger.ErrorLoggerExtensions.ErrorLogger.Error[b](ErrorLogger x, Exception exn) in C:\-\GIT\fs-proj\FSharp.Compiler.Service\src\fsharp\ErrorLogger.fs:line 326
   at Microsoft.FSharp.Compiler.CompileOps.TcImports.BuildFrameworkTcImports(TcConfigProvider tcConfigP, FSharpList`1 frameworkDLLs, FSharpList`1 nonFrameworkDLLs) in C:\-\GIT\fs-proj\FSharp.Compiler.Service\src\fsharp\CompileOps.fs:line 4598
   at Microsoft.FSharp.Compiler.FrameworkImportsCache.Get(TcConfig tcConfig) in C:\-\GIT\fs-proj\FSharp.Compiler.Service\src\fsharp\vs\IncrementalBuild.fs:line 1115
   at Microsoft.FSharp.Compiler.Interactive.Shell.FsiEvaluationSession..ctor(FsiEvaluationSessionHostConfig fsi, String[] argv, TextReader inReader, TextWriter outWriter, TextWriter errorWriter, Boolean fsiCollectible, Boolean msbuildEnabled) in C:\-\GIT\fs-proj\FSharp.Compiler.Service\src\fsharp\fsi\fsi.fs:line 2462
   at Microsoft.FSharp.Compiler.Interactive.Shell.FsiEvaluationSession.Create(FsiEvaluationSessionHostConfig fsiConfig, String[] argv, TextReader inReader, TextWriter outWriter, TextWriter errorWriter, FSharpOption`1 collectible, FSharpOption`1 msbuildEnabled) in C:\-\GIT\fs-proj\FSharp.Compiler.Service\src\fsharp\fsi\fsi.fs:line 2725
   at FSharpFar.Session.Session..ctor(String configFile)
   at FSharpFar.Session.Session.FindOrCreate(String path)
   at FSharpFar.FarCommand.Invoke(Object sender, ModuleCommandEventArgs e)
   at FarNet.Works.ProxyCommand.Invoke(Object sender, ModuleCommandEventArgs e)
   at FarNet.Far0.InvokeCommand(Char* command, Boolean isMacro)
   at FarNet.Far0.AsOpen(OpenInfo* info)
nightroman commented 8 years ago

One more thing. The machine does not have F# runtime installed. I provide FSharp.Core.dll with my package and non-FCS related F# code works fine. Is it perhaps mandatory for FCS that F# runtime is installed?

nightroman commented 8 years ago

So here is the best I can get, some obscure error thrown at

src\fsharp\CompileOps.fs:line 4598
    error(InternalError("BuildFrameworkTcImports: no successful import of "+coreLibraryResolution.resolvedPath,coreLibraryResolution.originalReference.Range))

Please let me know how I can change something to improve diagnostics.

I will be able to publish my changes for 8.0.0 only next week. Then I may explain the steps to install and repro. But a clean machine is needed, too, I presume.

nightroman commented 8 years ago

Investigated. My remaining issue was not related to MSBuild, it was about missing FSharp.Core.optdata and FSharp.Core.sigdata.

Thus, all works fine if MSBuild is not present and the issue may be closed. My module FSharpFar with F# interactive and editor services may work in Far Manager without installing anything but the module itself. This is great!

There is one minor imperfection though, as shown before. If MSBuild is present then it is loaded even if I opt out. This is happening due to two global calls to FSharpChecker.Create() with default parameters:

As far as I opt out, MSBuild is not used for resolution, hopefully, it just gets loaded. Should this be filed as a separate issue perhaps? Or is it not an issue at all?

dsyme commented 8 years ago

@nightroman Super, thanks. See https://github.com/fsharp/FSharp.Compiler.Service/pull/657 for the fix to the FSharpChecker.Create() calls.

nightroman commented 7 years ago

I think the issue is resolved completely in 9.0.0. I am closing it. Thank you!