Closed ctaggart closed 4 years ago
No the problem is not guessing the relative path. The problem is guessing the temp location.
Am 26.02.2017 12:33 schrieb "Gauthier Segay" notifications@github.com:
ScriptCS has a convention for naming their scripts in nuget packages BTW, this is something we should consider learning from their experience.
See https://github.com/scriptcs/scriptcs/wiki/Script-Libraries where they have a structure such as:
\ScriptCs.Calculator \Content \scriptcs CalculatorMain.csx
I assume the file is always the package name folowed by Main.csx
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/fsharp/fslang-suggestions/issues/542#issuecomment-282549671, or mute the thread https://github.com/notifications/unsubscribe-auth/AADgNOcFCD7ecU0kJNERUEWB5Qk2sXBKks5rgWMKgaJpZM4MJFib .
So I guess we would need to introduce #load "paket:..."
And as @k_cieslak said: using paket.core.dll is not a good idea
@forki to find the temp location, we would use same resolution than the #load "temppath/.paket/load/main.group.fsx" which we currently generate right?
In any case we need to patch #load
Am 26.02.2017 12:39 schrieb "Gauthier Segay" notifications@github.com:
@forki https://github.com/forki to find the temp location, we would use same resolution than the #load "temppath/.paket/load/main.group.fsx" which we currently generate right?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/fsharp/fslang-suggestions/issues/542#issuecomment-282549989, or mute the thread https://github.com/notifications/unsubscribe-auth/AADgNNQEGoxDvgY0PItE3ug_uBA1-NSgks5rgWRYgaJpZM4MJFib .
@smoothdeveloper the class name is the prefix before Main, so you get a Calculator class with that example. The main reason for the class was encapsulation, it avoids naming conflicts on members, and it also allows private members / state. Also it provides a sort of name spacing.
Saw this in F# Weekly, and just want to build on @Rickasaurus's suggestion; if there were some way to provide plugins for this, one could reference a library and then use the plugins defined in that library to perform the package management. Something like:
#r "paket"
#paket "NUnit ~> 2.6.3"
#paket "DotNetZip >= 1.9"
....
open NUnit
rather than @forki's sample? If the paket
library or a dotnet
(or NuGet
) library were easily available and packaged by default anywhere that fsi works, this should "just work" by referencing that default manager; but it would easily open up a future npm
package manager to provide its own handling for a #npm
pragma. If it went this way, it would be easy for the first couple package managers to be set up through the plugin architecture, but to also allow other things to plug in as well. Which could of course be dangerous, but it'd be fun to use.
@amazingant if you could look at CompileOps.fs/fsi.fs in the PR https://github.com/Microsoft/visualfsharp/pull/2483
This would help you to figure out an API to feed F# the "side effects" of those custom invocations, as those side effects need to make the behaviour work in tooling and FSI as well.
Thing is that this seem more involving than just experimenting with existing #r
/ #load
route like @forki is doing now.
See also https://github.com/Microsoft/visualfsharp/issues/1177#issuecomment-220180588 and https://github.com/Microsoft/visualfsharp/issues/56
@smoothdeveloper grabbed PR 2483, will poke around a bit, but I have no clue what I'm doing, so it make take me a bit to figure out how to even test any changes I make.
@amazingant cool! changes to FSI are surprisingly easy to test: compile and run/debug fsi from fsharp.sln.
Marking this "approved in principle". We definitely want this in some form, we just have to work out the details
Just a quick update. @smoothdeveloper (1000 thanks!) and I have been actively working on getting paket integrated into scriptcs with the new #r
syntax. We're making good progress and the implementation is basically done! (see screenshots below). Now on to unit / acceptance tests.
One thing that is cool about this from an F# note, is to make this integration work, we're adding a small F# project to scriptcs which gives us a shim that calls against similar code that has been refactored from VFS.
To make consumption easier we're including a copy of paket.exe in scriptcs and having scriptcs use the local copy rather than requiring the user to install a global copy.
If you want to try out the code you can grab the code in the paket-directive branch of scriptcs and build it.
Repeating what I wrote in the script issue (I don't know which is the best place):
Is it worth also having a common syntax which doesn't specify a package provider, in which case the script runner uses its default, e.g.
#r "nuget newtonsoft.json"
That way we can have a common syntax which works across all runners, without requiring all of them to support
packet:
,nuget:
, etc. The provider can still be specified optionally, for those runners which support that particular provider.
No I think that's no a good idea. Paket supports much more in the syntax than just "nuget Newtonsoft". Usually you will have "nuget XYZ ~> 2.1.4" or "nuget XY > 3 framework: net45, redirects true,..."
I think this will be very hard to make that common between nuget and paket.
Am 06.03.2017 11:43 vorm. schrieb "Adam Ralph" notifications@github.com:
Repeating what I wrote in the script issue https://github.com/scriptcs/scriptcs/issues/1220#issuecomment-284361624 (I don't know which is the best place):
Is it worth also having a common syntax which doesn't specify a package provider, in which case the script runner uses its default, e.g.
r "nuget newtonsoft.json"
That way we can have a common syntax which works across all runners, without requiring all of them to support packet:, nuget:, etc. The provider can still be specified optionally, for those runners which support that particular provider.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/fsharp/fslang-suggestions/issues/542#issuecomment-284361919, or mute the thread https://github.com/notifications/unsubscribe-auth/AADgNPd8AZK4RITiSWOUlkIw-PZ8ECqpks5ri-NPgaJpZM4MJFib .
The RFC FS-1027 is here: https://github.com/fsharp/fslang-design/blob/master/RFCs/FS-1027-fsi-references.md
Aha! Now I understand the syntax. It's:
#r "paket: <paket command>"
whereas I was previously reading it as:
#r "<package provider>: nuget <package-name>"
It makes sense now.
And it appears we also have this for a simpler syntax which can potentially be common across all runners:
#r "nuget: <package-name>, <package-version>"
That satisfies my previous suggestion. 👍
paket supports nuget or source files, which is why #r "paket: nuget ...." is still valid
I fully understand that
#r "paket: <paket command>"
can do way more than
#r "nuget: <package-name>, <package-version>"
Still, it's good to also have the latter syntax for the simple case, and it's something that all runners should be able to support.
Agreed. And I'll make sure we support the same in scriptcs.
Not sure if my question is related but would it be possible to support environment variable expansion in #r
and #I
directives? This is what I miss the most. My typical scenario is scripting in an existing application with several assemblies with their locations known via an environment variable.
@nightroman it would be a matter of implementing IDependencyManagerProvider
(https://github.com/Microsoft/visualfsharp/pull/2483/files#diff-0bcccb32cd2c0c6eec8f1aeb1a2ee1f5R67) and providing the implementation to FSI (by having the assembly in a specified path).
Please check the details in the RFC: https://github.com/fsharp/fslang-design/blob/229905b0513362cb4c342db85b9b3f8f898af871/tooling/FST-1027-fsi-references.md
Maybe I'm playing the devils advocate here but
it works "at design time" , i.e. I can open a script containing the text below (or some non-paket variation of it) - without running any command line tools - and quickly get editing and type checking against a resolved set of packages - without needing to run any of the script
it works universally, i.e. it works on any default install of F# editing + scripting
We now have a technical idea on how we want to implement it (ie reflection for example) and created a new extension point. But I guess we need another one for tooling to download those "extensions" otherwise I don't thing we can realize those two points.
For example:
Also if we are doing reflection I'm not sure that will work out for fable /cc @alfonsogarciacaro But I guess the fable story makes this a lot more complicated?
Mathias,
There is a forthcoming compiler tools feature, which will allow the specification of the plugins at startup. We are pretty confident that nuget and paket support will be shipped in the box with VS and dotnet/cli, I expect that is also true for Mono.
Other package mangers should be downloadable and from nuget, using either the nuget or paket plugin.
I.e.
I hope this helps
Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows 10
From: Matthias Dittrich notifications@github.com Sent: Thursday, August 2, 2018 10:25:58 AM To: fsharp/fslang-suggestions Cc: Kevin Ransom; Mention Subject: Re: [fsharp/fslang-suggestions] Package references in F# scripting (#542)
Maybe I'm playing the devils advocate here but
it works "at design time" , i.e. I can open a script containing the text below (or some non-paket variation of it) - without running any command line tools - and quickly get editing and type checking against a resolved set of packages - without needing to run any of the script
it works universally, i.e. it works on any default install of F# editing + scripting
We now have a technical idea on how we want to implement it (ie reflection for example) and created a new extension point. But I guess we need another one for tooling to download those "extensions" otherwise I don't thing we can realize those two points.
For example:
Also if we are doing reflection I'm not sure that will work out for fable /cc @alfonsogarciacarohttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Falfonsogarciacaro&data=02%7C01%7CKevin.Ransom%40microsoft.com%7Ceca24679b8da4a8fc4ec08d5f89d038b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636688275617247458&sdata=ha53fLIE6w9PDcizyyztcGZH7VFyZFd5dGa%2B4IKxNEA%3D&reserved=0 But I guess the fable story makes this a lot more complicated?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ffsharp%2Ffslang-suggestions%2Fissues%2F542%23issuecomment-410005005&data=02%7C01%7CKevin.Ransom%40microsoft.com%7Ceca24679b8da4a8fc4ec08d5f89d038b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636688275617247458&sdata=LOyMbHd45t3OqPpd7ronoVwu4V8394xlRjzlbJDI7Ew%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAE76FgLlswRO6kSKnOhRJdce8hcQmOh6ks5uMzYmgaJpZM4MJFib&data=02%7C01%7CKevin.Ransom%40microsoft.com%7Ceca24679b8da4a8fc4ec08d5f89d038b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636688275617247458&sdata=e0AKrbSTqW%2FBbPs5RcfICGub1je9dTB%2BTTnZ2iH0iEk%3D&reserved=0.
@KevinRansom oh that's really cool. Is there anywhere tracking progress on it so I can know when it lands?
Not yet. Update to the RFC will be coming, but I've got a few more things on my plate
@jwosty getting this done has been my highest priority for months, and yet all sorts of other stuff has kept nosing ahead of it. I will say that thanks to last night's check-in eliminating our dependence on a super old version of the msbuild libraries. I have nothing higher priority right now.
Phillip also has been occupied by other higher priority items too.
Ok, thanks. Just want to say I really appreciate all the stuff you guys are doing! Keep up the awesome work! :)
@matthid to add to what @KevinRansom was saying, I think for the example you mention, we would rely on package manager to fail with a stable error message / error number.
This would allow VS or other editors to detect well known error and provide a code fix / way for user to install dependency.
If you think more is needed, you might want to propose additional design of things that could be made available in the PackageManager assembly and could be used, but I think it is a can of worm to have that happening inside the FSI process rather than propagating errors handled by the tooling.
If you think more is needed.
Yes indeed. I think error scenarios are almost missing from the spec. In particular we need to design/decide:
I think we should look at all of those from a users and a extension developers perspective and specify those cases a bit. We should learn from type-providers where errors a lot of times are useless for the user AND the developer (which is really the worst of both worlds)
My suggestions would be:
#r
is invalid@matthid Sorry, I haven't followed the discussion. Fable just needs the list of arguments that must be passed to the F# compiler. In the case of dependencies, this means resolving the actual .dll references (like -r:/Users/Anne/.nuget/packages/Fable.Core.2.0.0/lib/netstandard2.0/Fable.Core.dll
). This is what Dotnet.ProjInfo does for us, and what FSharpChecker.GetProjectOptionsFromScript used to do too (currently disabled precisely because it's difficult to add nuget packages to a script and it's not useful to program Fable if you don't have access to Fable.Core).
Not sure how this is supposed to work, but if .GetProjectOptionsFromScript
works the same shouldn't be a problem for Fable.
@alfonsogarciacaro I was more speaking if the #r "manager:"
extension specified here will work in the fable repl for example. I'm not sure the current reflection based code would do something useful in the browser?
No, forget about adding dependencies to the online repl, we cannot even have multiple files there :wink:
We do have some dependencies in the online repl but preparing them is a somewhat complicated process, must be done before hand and it has limitations (only metadata can be read).
Ok, so probably a fun vacation project :P. Anyway, I guess that means we can ignore fable for now and use #if once we know how to do it...
Any progress on this problem?
@reinux Yes, you can follow the PR here: https://github.com/Microsoft/visualfsharp/pull/5850
It's temporarily on hold so that we can ship what we have with FSI and F# 4.6
I think in order for FSI to be useful in larger projects there should be better integration with the project system. This may not be possible with .fsx files which don't appear to work inside projects (don't appear to see definitions in previous files in a project or see definitions in referenced projects).
For example:
File0.fs and File1.fs live inside Project1 which references Project0. Right click on File1.fs or within File1.fs -> Load context to FSI. This:
Is this request separate enough from this dicsussion that it should go in a separate suggestion?
Correct, .fsx files effectively float in space. What you're looking for is something akin to "generate project context" in the FSI instance, which could be a tooling feature but it would be great if there was also an FSI mechanism for if you just launched the REPL in a project's directory.
Any progress here?
Nope.
So the „#r nuget“ in Jupyter will only be possible in Jupyter (since it’s a kernel feature) or how is that related to this issue here?
Right click on File1.fs or within File1.fs -> Load context to FSI. Is this request separate enough from this dicsussion that it should go in a separate suggestion?
@charlesroddie This would be a great feature to have, but being buried in this thread probably doesn't help giving it traction. Back in VS 2015 there was an extension creating #r
references of all references in a project, dumped into a script file. That already made this process much simpler.
But I agree, a single 'send context to FSI' would just be awesome.
@ronaldschlenker correct, until .NET 5, #r nuget
will only work in the jupyter kernel (and previews of .NET 5). You try it out here: https://mybinder.org/v2/gh/dotnet/try/master
Closing as this has been implemented.
Just for interest (if you have time to answer): why .Net5? Will there be a unified scripting base with C# or what’s the deal here?
.NET 5 is the next release train for F#, and since this is in preview today, that is the next available release for the feature (same goes with nameof
and opening of static classes).
I would love for the #project "my.fsproj
support to be added. When doing Haskell and GHCi, I always use its "open an interactive and load the current project's source code" feature. So productive and a brilliant way to explore what functionality you've written.
I would love for the
#project "my.fsproj
support to be added. When doing Haskell and GHCi, I always use its "open an interactive and load the current project's source code" feature. So productive and a brilliant way to explore what functionality you've written.
This would be awesome! 💯 As far as I understand the resolution mechanism for those references will be extendable.
The resolution mechanism will work with a prefix like nuget (at the moment the only implementation) or paket.
#r "project: my.fsproj"
Please correct me if I am wrong.
In my imagination a possible project reference could be:
#r "project: my.fsproj"
Which executes the following steps when evaluated (.NET Core with NuGet references):
I've worked on something during today's live stream - https://github.com/Krzysztof-Cieslak/DependencyManager.FsProj
@ErikSchierboom @toburger @Krzysztof-Cieslak can you file an issue on dotnet/fsharp
tracking this as a feature request? I think it's perfectly reasonable and @Krzysztof-Cieslak has shown a possible way forward.
Valid point! I've created the issue here: https://github.com/dotnet/fsharp/issues/8764
[ edited by @dsyme to be a more comprehensive guide to this design question ]
[ Latest implementation is here: https://github.com/Microsoft/visualfsharp/pull/4042 ]
Package references in F# scripts
There has been a long standing desire to add the ability to reference nuget packages from F# scripts. Originally this was conceived as basic fully qualified nuget references like
packages.config
. Lately this has evolved into integrating "dotnet" or "paket" or "npm" package specifications and dependency management tools as part of the toolchain, or providing generic hooks to allow this.Related links
#r nuget
in RoslynDesign principles
You can add a package reference to a script with a single line using a normal text editor that supports F# - no extra files (e.g. a packages.config) are needed
Package references include version constraints, and dependency resolution is performed a.la. nuget v3 and/or paket
it works "at design time" , i.e. I can open a script containing package specifications and quickly get editing and type checking against a resolved set of packages - without needing to run any of the script or any command line tools
it works universally, i.e. it works on any default install of F# editing + scripting, whether Ionide or the Visual F# Tools or web-hosted delivery of F# scripting such as Azure Functions or Azure Notebooks.
It aligns well with how package management will be dealt with in the future of the .NET toolchain
it works "the same way" on both .NET Framework/Mono and .NET Core, at least to some approximation,.
It considers the needs of F# when used as a Javascript programming language through toolchains such as Fable or WebSharper. Here NPM is a natural package manager, though there are others.
The design and implementation do not induce a dependency on any one specific package manager within the core F# toolchain (i.e. compiler/scripting/editor/Fsharp.Compiler.Service/ProjectCracker). It might be that different package managers have some specific support to make them work, but we remain open to new package managers.
The implementation does not induce bad "layering problems" in the F# toolchain implementation reminiscent of MSBuild, see below.
It works efficiently - package resolution is amortized, for example.
The default settings in editing and execution tools are sufficiently space-efficient, sharing packages between scripts if needed to achieve this.
Possibiity. "dotnet" references in scripts
The way the new dotnet core tooling loads nuget packages and their assemblies is awesome! I've been using its extensibility to build a DotnetCliTool. All the dependencies are downloaded & loaded from the single
%userprofile%\.nuget\packages
directory. I would like to be able to use this same mechanism from scripts. I would like the same types of reference to be supported, package references and project references.Instead of having to use a nuget client to download the package and then reference the assembly like:
I want to be able to do:
Possibiity.
#project "my.fsproj"
references in scriptsPeople have suggested that project references should work the same way as package references to work in Visual Studio 2017 with the dotnet core tooling. e.g. be able to do:
Possibiity.
#r "paket: Foo.Bar.dll"
references in scriptsThe experiment https://github.com/Microsoft/visualfsharp/pull/2483 contains a prototype of integrating paket package management directly into the F# programming model for .NET Framework programming. This includes design-time support. The experiment violates one of the design principles above - it "bakes in" support for Paket only. However that support could be factored into either fsi.exe.
Possibility.
#r "packages"
references in scriptsPeople have suggested that a script simply referencing
#r "packages"
should implicitly pick up the packages from its context, e.g. the containingpackages.config
orsolution
orpaket.dependencies
in the toolchain. In the Experiment adding Paket support to FSI this is#r "paket"
Possibility. implicit generation of load scripts
Paket has a feature to accurately generate
.fsx
and.csx
load scripts containing the#r
references suitable for use with F# and C# scripting. This feature is a natural and simple way to integrate package resolution - simply have the package manager resolve the packages, generate the scripts and load the scriptsPossibility. sharing packages and package caches
A major question is where packages are cached. This is primarily a responsibility of the package manager, but becomes a serious issue for scripts because packages can't be duplicated for 100s of scripts, so some shared caching is needed.
Possibility. align with C
It is possible we should approach this in a similar way that C# repl and scripting want to approach the problem for .NET Core. There are shared concerns here and it is likely we should move forward with the same model for how assemblies and assembly versions are loaded in a scripting session.
Given that C# and F# scripting are also used for Azure Functions, it seems to me that a shared behavior here would be best. However this would mean spec'ing out that behavior all-up and perhaps building out an underlying component that F# and C# could sit atop.
ScriptCS also wants to align with a compatible mechanism: https://github.com/fsharp/fslang-suggestions/issues/542#issuecomment-282497990
Possibility. allow expression of SemVer version constraints
Paket and dotnet both have ways of specifying version constraints. The abiility to include these prior to package resolution is important.
https://github.com/fsharp/fslang-suggestions/issues/542#issuecomment-282498554
Possibility. autocomplete and search
Auto-completing package names gives a great way to search and discover package functionality.
Basic autocomplete is possible already in package specifications like
packages.config
andpaket.dependencies
. e.g. see autocomplete in IonideAdditionally auto-completing on search terms such as
#r "package: statistics
giving search of package description text would be helpful.Challenges
Some things make this tricky for F#
Challenge: Compiler architectural layering (basics)
The basic "natural" layering of the toolchain is
Here
[AssemblyResolution]
is a plugin-point or the mechanism used to resolve assemblies.Note that in this architecture the F# editing tools support the F# scripting programming model via
FSharp.Compiler.Service.dll
.This means that as things stand today the implementation of F# scripting is not "a tool on top of F#" (ala scriptcs) but actually part of
FSharp.Compiler.Service.dll
and thus pretty much universally supported in F# editors. The allows us to deliver scripting into a very wide range of contexts simply, efficiently and consistently, e.g. into Ionide, Azure Functions, Azure Notebooks and many online tools.With integrated package resolution one option is that this becomes
where
[PackageResolution]
indicates a potential plug-in point. An alternatives would be to build a layer "outside and on top" of FSI.EXEHowever, this approach doesn't seem to work particularly well with the incremental addition of package specifications in a REPL session.
Challenge: Compiling scripts and architectural layering
Traditionally the tool
fsc.exe
has supported the ability to compile F# scripts including their references. This has induced a violation of layering in the toolchain: the FSC.EXE tool also included the logic for assembly resolution and quite a lot of logic for processing F# files as scripts. This meant that the FSC.EXE tool and FSharp.Compiler.dll became badly dependent on the MSBuild API simply to resolve assembly references in scripts.Worse still this "leaked out" into the logical specification of the compiler itself. The compiler was now able to accept strange assembly specifications such as
-r:System. FooBar,Version=3.2.10,..
on the command line and resolve them with MSBuild. In the original world of .NET MSBuild shipped as part of the .NET Framework. However when MSBuid became separated this started to cause immense pain, and even more so when ,.NET Core came about. This problem was poisonous to whole toolchains built on FSharp.Compiler.Service, and we only recently did the hard work to "extract the rotten tooth" and make MSBuild optionalChallenge: .NET Core toolchain
The .NET Core toolchain changes some things about how F# compilation is surfaced. In particular it adds a layer
dotnet ...
via which all tools are accessed from the command line.. This means that with .NET Core the architectural layering becomes something like this:The question is where [PackageResolution] happens in this toolchain. But first there are other questions that need to be resolved:
Not all variations of F# running on .NET Core go through the
dotnet
tool. For example, clients of the FSharp.Compiler.Service.dll running on .NET Core such as the Fable compiler do not - they just embed F# parsing and checking directly via the compiler service DLL.Fable assumes an installation of "dotnet". However will it be the case that all clients of F# scripting (editors, engines etc.) can assume an installation of "dotnet"? If I embed F# scripting in an application, do I assume "dotnet" is installed? Do I have to download reference assembly packages from nuget to do simple F# script execution?
It is not clear what the specification of the F# .NET Core scripting engine will with respect to loading "uplevel" versions of the assemblies that are used to implement the scripting tool itself. This is discussed below.
It seems that these questions need to be resolved before integrated package management can be addressed.