Solution-relative paths for Relative TP

[sergey-tihon]

FileSystem and Relative TPs are really useful in Unit Tests. In my case I have a large files that are used from tests with read-only access. It does not make sense to copy these files in target directory. I want to be able to use Relative TP to specify solution-relative or project-relative paths. The only way that I found is

let [<Literal>] dataFilesRoot  = __SOURCE_DIRECTORY__ + @"..\..\data"
type DataFiles = FSharp.Management.FileSystem<dataFilesRoot>

[ReedCopsey]

Look at CommonPaths docs here: https://github.com/forki/FSharp.Management/blob/master/docs/content/CommonFolders.fsx

You should be able to use RelativePath combined with the fshapmanagementlocation to read these files in a way that works no matter where things are installed.

[ReedCopsey]

Another key here... You can't actually make a path relative to the solution, you need to build it relative to the build folder, but using that, it works fine. Problem is that's there's no way to find the solution folder after build/compile when running, at least not without providing more info

[ReedCopsey]

@sergey-tihon Does this latest pull solve your issue? Can this be closed now?

[sergey-tihon]

As I understand I still cannot use paths like this

type DataFiles = FSharp.Management.RelativePath< @"..\data">
let file = DataFiles.``test.simple.utf8``

because in runtime it is equal to \Stanford.NLP.NET\tests\Stanford.NLP.Segmenter.FSharp.Tests\bin\Debug\test.simple.utf8 instead of \Stanford.NLP.NET\tests\data\test.simple.utf8 (this one in design time)

P.S. Actually \Stanford.NLP.NET\tests\Stanford.NLP.Segmenter.FSharp.Tests\bin\Debug\test.simple.utf8 looks strange for me, a could undestand \Stanford.NLP.NET\tests\Stanford.NLP.Segmenter.FSharp.Tests\bin\data\test.simple.utf8 but not the first one ...

[ReedCopsey]

There are two problems this:

1) The type provider has no knowledge of where your build output folder lies, so there's no way to build a relative path to the build folder. This could, potentially, be handled via an optional static parameter (default to "bin\Debug" or "bin\Release"), but would need to be configurable, and would have to be set by the user.

2) At runtime, especially when doing unit testing, we don't know where the solution folder lies. The assemblies potentially get shadow copied, so the current location is not at all what you'd expect. CommonFolders handles this (at runtime) by allowing you to specify the original location, which is why the solution can get there.

I could see a way to handle this, potentially, if we didn't use literals in the provider, but it would change how the type provider needs to work dramatically in order to function. You'd need to construct an instance (instead of using a provided) constant, since the "base path" would need to be constructed at runtime. We'd also need to force the user to specify a build folder path (always) so we could go from the assembly location to the project folder. I suspect the extra complexity wouldn't be worth the benefits, though I do agree it'd be nice to write what you've got above.

-Reed

//cc @forki Steffen - do you have any thoughts on this?

[forki]

Maybe I didn't read the issues here correctly, but my idea was to use the relative path provider but to return absolute paths. So it would always provide the path to the original file.

Would this help?

[ReedCopsey]

@forki There's a fundamental problem here -

The issue is that we have access to a different "root" path at runtime vs. compile time, and there's no (automatic) way to bridge the two. When you create a type provider, you are given the project root folder (the location of the .fsproj), but at runtime, the only "root" you have access to is the assembly location.

Since the assembly build path can be customized in the .fsproj (it's not always bin/debug or bin/release, and the default scaffolding recommendations actually specify having a different output path), we can't just assume we can go up two folders and use that as the root.

There are ways around this - though they'd all require a fairly significant change to how the type providers work.

First, the user would have to, either at compile time or runtime, provide the "relative" location of the build output folder from the project directory. This could be done as another static parameter, and we could default it to "bin\debug" (or use "....", which would work in reverse, and be more clear for bin/release and bin/debug). This would let us "remap" the relative path to the project folder at runtime.

Second, instead of using literals for the values, we'd have to change everything around so the file paths were all properties. This would let us store the info, and compute at runtime the appropriate absolute paths based off the location of the assembly, the relative location to the build folder, and the specified file (property) coming out of the type provider.

At the end of the day, I'm not sure it's worth the effort. It makes the type provider a lot more complicated (I had played with this a while back, and it's nowhere near as clean). The code you can write now (ie: https://github.com/fsprojects/FSharp.Management/blob/master/tests/FSharp.Management.Tests/FileSystemProvider.Tests.fs#L63-L69 ) isn't really too bad.

I could add another method to CommonFolders that built a path from the application location directly, though - which would allow us to shorten that code to:

let ``Can access solution files using RelativePath provider``() =
    let fsDocPath = RelativeToBuild.``..``.``..``.``..``.``..``.docs.content.``FileSystemProvider.fsx``
    // Doesn't exist - could add this
    let path = CommonFolders.BuildApplicationPath fsDocPath

    System.IO.File.Exists(path) |> should equal true

It doesn't save a lot, but it does shorten it a bit. The main "ugliness" here is that you're building the relative path to the build location, then backing out folders.

Another option would be to allow the RelativePath provider to build paths for a folder in the solution, relative to a second folder - this could work similar to how the FileSystem provider works now, and let you do something like:

type Relative = RelativePath<@"..\data", relativeFrom=@"bin\Debug">

// This would end up being a value of @"..\..\..\data\test.simple.utf8"
let relativeFile = DataFiles.``test.simple.utf8``

// This would be the full path to the file, computed at runtime
let file = CommonFolders.BuildApplicationPath DataFiles.``test.simple.utf8``

I'm pretty sure I could make this work (without changing the fundamental nature of the type provider) without breaking anything that is there now. It's also relatively clear in its intent, I think.

@sergey-tihon I know this isn't exactly what you're picturing, but would this work for your scenario? It'd give you the nicer path structure I think you're after, without breaking anything else (though you would still need to pass through a function to turn it into an absolute path).

-Reed

[sergey-tihon]

@ReedCopsey Thanks for options.

According to the latest post, I think that it does not make sense to add additional noise to RelativePath TP. I do not see advantages over initial version with __SOURCE_DIRECTORY__ andFileSystem. I would leave it as it is.

[ReedCopsey]

@sergey-tihon The problem with the original version, btw, is that it will break if you're committing to source control. It relies on the absolute file paths to always be the same. I kind of like the last option I posted above - it would at least work correctly in source control, and be consistent with the other providers. I'm happy enough leaving things as they stand today, though, as the option in the test right now works for me.

[forki]

If I understand correctly everybody is happy with the current solution. I close the issue and if we have trouble or a new nice idea we open a new one.