search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
14.15k stars 4.42k forks source link

Support for FreeBSD #14537

Open ghuntley opened 9 years ago

ghuntley commented 9 years ago

Updated proposal from 2017/9

Proposal (by @karelz - https://github.com/dotnet/corefx/issues/1626#issuecomment-329840518) will be updated in top-post based on further discussion and proposal changes.

We discussed community-driven port for FreeBSD with @RussellHaley (from FreeBSD community) and @wfurt (from .NET Core team) who both expressed interest in the work. Here's a plan proposal we put together (feedback / suggestions are welcome):

  1. Produce binaries in CoreCLR & CoreFX repo targeting FreeBSD - using hacks is fine
    • Hard to parallelize, @wfurt will work on that
    • The build can be mix of builds from other platforms (Mac, Linux) targeting FreeBSD
    • We will need documented steps (on FreeBSD wiki) to reproduce the build with FreeBSD-specific bug fixes
  2. Run & stabilize CoreCLR tests (using corerun)
    • Tests may be built on another platform
    • Goal: Provides basic quality of runtime
  3. Run & stabilize CoreFX tests (using corerun)
    • Tests may be built on another platform
    • Note this requires xunit. We believe, based on our past porting experience, once [2] is done, xunit will just work.
    • This can be in theory parallelized with [2] - it may require shortcutting xunit (e.g. generate static execution recipe on another platform)
    • We can expose new OSPlatform API for FreeBSD when the pass rate is reasonable: see dotnet/corefx#23989
  4. Full stack build on FreeBSD (using corerun as bootstrapper from [1]-[3])
    • We will need all tools (nuget, msbuild, roslyn) to work on boostrapping .NET Core
  5. Installers (FreeBSD ports)
    • First-stage: Using product binaries from nuget feeds
    • Second-stage: Build product from source (blocked on build from source effort)
    • Requires FreeBSD community expertise and guidance on design
    • Note: We can link FreeBSD packages also from official .NET Core download pages as community-support packages
  6. Regular build and test runs on FreeBSD
    • Goal: Make sure changes in .NET Core repos breaking FreeBSD are known early
    • Design needed
    • Requires FreeBSD community expertise and guidance on design

Operation principles:

If anyone is interested in helping, please let us know here. We can easily distribute work items from [2] & [3] above once we are far enough with [1].

Original proposal from @ghuntley from 2015/5

This issue is to discuss unit(s) of work to actually produce FreeBSD assemblies for corefx.

@stephentoub - There's what's likely a more pressing issue, which is actually building for FreeBSD. Today, when we need to specialize an assembly for a particular platform, we effectively have three builds, producing three different managed assemblies: Windows, Linux, OSX. Sounds like at least for now we'll need a fourth, FreeBSD. I suggest you start by modifying the build to support an IsFreeBSD property (or just IsBSD of you think there's a high chance that the implementations across BSDs will be the same even with varied kernels) along with the appropriate OSGroup targets. That can then be used in the csproj files as needed to specialize an assembly with FreeBSD-specific code.

Related issue(s)

/cc: @janhenke @josteink

josteink commented 9 years ago

There seems to be agreement as far as https://github.com/dotnet/corefx/issues/1576 is concerned.

When we also have a decision on https://github.com/dotnet/corefx/issues/1625 we should be able to start shipping some code.

ghuntley commented 8 years ago

Agreement on dotnet/runtime#14536 has been reached by the portteam, unless MSFT chooses otherwise it will be FreeBSD. Issue dotnet/corefx#1999 will potentially be the issue that introduces the definition into the public API.

josteink commented 8 years ago

Agreement on dotnet/runtime#14536 has been reached by the portteam, unless MSFT chooses otherwise it will be FreeBSD

If I read that right, this means that when https://github.com/dotnet/corefx/pull/1999 is merged, we can consider this MSFT approving of the new public API, and can therefore press forward on the remaining issues with regular pull-requests without need for MSFT approval.

If so, that sounds good to me.

ghuntley commented 8 years ago

Next steps as per https://github.com/dotnet/corefx/pull/1999#issuecomment-111279577 are:

  1. The "FreeBSD port team" continues their work to get a FreeBSD version of CoreFX produced (tracked by dotnet/corefx#1626).
  2. The port team brings up enough of the CoreFX and CoreCLR stack on FreeBSD such that we can start running the CoreFX unit tests on FreeBSD.
  3. The tests reach some minimal quality level. I don't know exactly what this looks like yet, but I expect it means something like a majority of the tests pass. Ideally we would not have a bunch of specific tests disabled for only FreeBSD (compared to Linux and OSX, we wouldn't want to hold FreeBSD to a higher standard than the other *NIX platforms we have there).
  4. Working with the FreeBSD port team, the CoreFX team gets the CoreFX tests added to our CI system running on FreeBSD.
  5. Discuss merging a PR based for issue dotnet/runtime#14536, which adds the property.
josteink commented 8 years ago

That sounds like a fully reasonable plan to me.

janhenke commented 8 years ago

Okay, then let's start the work on getting corefx to work.

josteink commented 8 years ago

First obstacle in building corefx on FreeBSD seems to be mono. The build-script insists version 4.1 is required. @ajensenwaud did some work on this on the Frankfurt-host, but I'm not sure how complete it is.

I'll queue a build for now and see what the output looks like.

Edit: The (mono) build crashes with the following kicker at the end:

Making all in mini
make[1]: "/usr/home/josteink/mono/mono/mini/Makefile" line 2906: warning: duplicate script for target "%.exe" ignored
make[1]: "/usr/home/josteink/mono/mono/mini/Makefile" line 2899: warning: using previous script for "%.exe" defined here
  CC       genmdesc-genmdesc.o
In file included from genmdesc.c:9:0:
mini.h:17:34: fatal error: ./mono/metadata/loader.h: Too many levels of symbolic links
 #include <mono/metadata/loader.h>
                                  ^
compilation terminated.
*** Error code 1

Stop.
make[1]: stopped in /usr/home/josteink/mono/mono/mini
*** Error code 1

Stop.
stephentoub commented 8 years ago

First obstacle in building corefx on FreeBSD seems to be mono

FWIW, I personally don't think this is the first obstacle. There are two build related issues:

  1. Building assemblies that work correctly on FreeBSD
  2. Building those assemblies on FreeBSD

(1) is critical, and is I believe what this issue is meant to be about. (2) is very nice to have, but lack of it doesn't prevent the creation of a great system for running managed code on FreeBSD.

You're of course free to prioritize however you see fit, but my recommendation would be to focus on (1) rather than (2).

Note that we still have issues building corefx on Linux and building it on OSX, such that our CI system builds the assemblies for those platforms on Windows; it then shuttles the resulting assemblies over to the target platform to execute the tests.

josteink commented 8 years ago

That's fair enough. I just assumed that it would be easier to get general FreeBSD platform support baked into corefx if we could actually build it ourselves on FreeBSD.

I'll make do with Windows-initiated building for now and attempt to ninja together a build-configuration.

akoeplinger commented 8 years ago

@josteink btw. corefx should now build on Mono 4.0.1.44.

josteink commented 8 years ago

@akoeplinger Nice. That leaves me some hope we can get it running on FreeBSD too :)

ajensenwaud commented 8 years ago

Good points. However if we really want corefx to be part of the FreeBSD environment, we really need it to be able to compile from source to get it into the Ports system.

I did hear that Mono 4.0.1.44 fixes a lot of these issues but have not had time to play with it yet. I know the ports team are updating the port Makefile as well as we speak with a new patch.

On 12 Jun 2015, at 20:21, Stephen Toub notifications@github.com wrote:

First obstacle in building corefx on FreeBSD seems to be mono

FWIW, I personally don't think this is the first obstacle. There are two build related issues:

Building assemblies that work correctly on FreeBSD Building those assemblies on FreeBSD (1) is critical, and is I believe what this issue is meant to be about. (2) is very nice to have, but lack of it doesn't prevent the creation of a great system for running managed code on FreeBSD.

You're of course free to prioritize however you see fit, but my recommendation would be to focus on (1) rather than (2).

Note that we barely have corefx building-on-Linux and building-on-OSX, such that our CI system builds the assemblies for those platforms on Windows; it then shuttles the resulting assemblies over to the target platform to execute the tests.

— Reply to this email directly or view it on GitHub.

stephentoub commented 8 years ago

Yes, I'm in no way disagreeing... being able to build corefx on Linux, OSX, and FreeBSD is important. I'm simply suggesting that from a priority perspective it's more important to be able to actually run corefx on Linux, OSX, and FreeBSD. :wink: If both can be worked on in parallel, all the better.

ghost commented 8 years ago

@ghuntley, would be super :cool: if we have a markdown task checklist outlining what what is remaining:

- [x] task 1
- [ ] task 2
- [ ] task 3

renders as:

This will probably encourage others to score those feats and FreeBSD support will land rather sooner than anticipated! :sunglasses:

janhenke commented 8 years ago

To my knowledge the following pieces of work in CoreFX are required for FreeBSD support:

13 Assemblies do not compile on their own and need FreeBSD specific changes. Mostly the Interop pieces that already exist for Linux/OS X (order by the occurrence in the build output):

I will try to update that list based on PRs opened and merged.

josteink commented 8 years ago

FYI: PR dotnet/corefx#2039 merged

josteink commented 8 years ago

Just trying to be ahead of the curve here... How do we plan to implement System.IO.FileSystem.Watcher ?

Iirc FreeBSD has no inotify such as Linux and Windows does (which is also why there is no Dropbox last time I checked). Will this be a potential source of trouble coming our way? Or does anyone have an idea for how to work around this?

janhenke commented 8 years ago

I suggest we stub that out for the moment and throw a PlatformNotSupportedException as Stephen Toub suggested in the other topic (https://github.com/dotnet/corefx/pull/2021#issuecomment-111602342). Then we have at least a complete set of assemblies and we can continue to work on that particular issue without blocking further steps.

Would you mind opening a separate issue for that?

ghuntley commented 8 years ago

Let's move System.IO.FileSystem.Watcher discussions to dotnet/corefx#2046

ghost commented 8 years ago

Guys is there any such blocker for System.Diagnostics.Process?

ghuntley commented 8 years ago

@jasonwilliams200OK added FreeBSD to S.RT.I.RI early this morning which was merged but the FreeBSD tests within CheckPlatformTests had to be backed out until dotnet/buildtools is updated.

ghuntley commented 8 years ago

@jasonwilliams200OK there were some discussions last night about System.Diagnostics.Process in gitter which have been formalized into https://github.com/dotnet/corefx/issues/2070

ghost commented 8 years ago

@ghuntley, thanks. I actually read those messages. System.Diagnostics.Process is a tricky one. AFAIK, io.js team had similar challenges with FreeBSD process management. Mono team has probably nailed it, so lets hope if @akoeplinger and co. could enlighten us on this matter? :)

josteink commented 8 years ago

System.IO.FileSystem.DriveInfo

As discussed in the gitter, For this one I tried looking into basic usage of getmntinfo:

#include <sys/param.h>
#include <sys/ucred.h>
#include <sys/mount.h>
#include <stdio.h>

int main() {
  struct statfs *mntbuf;
  int mntsize = getmntinfo(&mntbuf, MNT_NOWAIT);

  for( int i = 0; i < mntsize; i++ ) {
    printf("%s\n", mntbuf[i].f_mntonname);
  }
}

Running that sample yielded this output:

$ ./a.out
/
/dev
/tmp
/usr/home
/usr/ports
/usr/src
/var/crash
/var/log
/var/mail
/var/tmp
/dev/fd
/usr/compat/linux/proc
/proc
$

So it seems it does what we need. The question is, should we do any type of filtering on the results?

Looking at the "intent" of the DriveInfo object, coming from the Windows world of .NET it has often been to enumerate the available locations to store or retrieve files (C:, D:, etc). But when using Unix hierarchical file-systems, returning / would be adequate to cover those needs.

So what should we return? What would be useful? Should even consider it being useful or not?

The Linux-version just dumps everything, except things set to be ignored:

https://github.com/dotnet/corefx/blob/master/src/System.IO.FileSystem.DriveInfo/src/System/IO/DriveInfo.Linux.cs#L98-L99

I tried putting in the following filter, but it didn't really change anything in terms of output:

    if ((mntbuf[i].f_flags != MNT_IGNORE)) {
        printf("%s\n", mntbuf[i].f_mntonname);
    }

Any opinions?

ghost commented 8 years ago

@josteink, great diggings! Based on https://github.com/dotnet/corefx/issues/815#issuecomment-113825960 and https://github.com/dotnet/corefx/issues/1729, I think we should collaborate with @sokket to come up with a solution with works across different Unices.

jonmill commented 8 years ago

I have a version running on OSX that uses getmntinfo and statfs to get information about each mount point, which seems like the most logical mapping from the Windows Drive concept. I'll double check that the function and struct definitions on OSX match the FreeBSD definitions and, if so, my commit for OSX will work for BSD as well.

I'll be sure to add you to my PR @josteink

josteink commented 8 years ago

Sounds good. Thanks for the heads up and thanks for giving FreeBSD some love too.

I looked into some basic pinvoke for these functions, and it seems like we need to do all the marshalling and conversions ourselves, so if anyone else has already put in the effort, who am I to say no? ;)

jonmill commented 8 years ago

No problem...looks like the main difference was with the struct declarations; since we'll probably hit this more in the future, I'm doing some refactoring that will allow us to share a lot of the PInvoke signatures. I'll add a bigger description in my PR (today or tomorrow, based on how the test run) but I basically added the PInvoke signatures and struct signatures for FreeBSD (based on the headers I found online) and it compiles. I've tested it on Mac so it should (in theory...) work on FreeBSD since it's just a struct declaration change, but your milage may vary :). If it doesn't, you'll have the DriveInfo class and PInvokes 99% of the way there and will just require some tweaking based on FreeBSD nuances.

ghuntley commented 8 years ago

Excellent news @sokket. I've created you an account on the machine the port-team uses for development, it's european based but it's always on and has heaps of memory and processing power. Hopefully this will help out and remove some of the friction when working w/FreeBSD.

# ssh sokket@freebsd-frankfurt.zbsd.org

Password authentication is disabled, use one of your keys.

@josteink see also issue: https://github.com/dotnet/corefx/issues/815 (System.IO.FileSystem.DriveInfo for Mac/FreeBSD)

janhenke commented 8 years ago

Are there any updates? Did anybody implement the remaining assemblies on FreeBSD?

josteink commented 8 years ago

I've been busy attending my new baby, haven't had time for any coding anywhere.

I've suspected that issues like these have been lying dormant and I guess this confirms it to a certain extent.

janhenke commented 8 years ago

For the assemblies that are still not implemented, I linked the "how to implement"-issue in the list above. I hope that helps coordinating the effort to get these remaining assemblies implemented.

josteink commented 8 years ago

I must admit I was having difficulties keeping track of what we have done and where, so that's definitely a good move. Good job :)

ajensenwaud commented 8 years ago

Where do I find that? Would be grat to get the remaining assemblies implemented.

On 25/07/15 22:10, Jan Henke wrote:

For the assemblies that are still not implemented, I linked the "how to implement"-issue in the list above. I hope that helps coordinating the effort to get these remaining assemblies implemented.

— Reply to this email directly or view it on GitHub https://github.com/dotnet/corefx/issues/1626#issuecomment-124838781.

josteink commented 8 years ago

This comment here: https://github.com/dotnet/corefx/issues/1626#issuecomment-111800540

janhenke commented 8 years ago

I am right now waiting for the native shims to be finished, as these should take over most of the work for getting these assemblies working on FreeBSD.

@nguerrera would be great if you can keep us posted on the progress. :)

ghost commented 8 years ago

Update: @janhenke confirmed that with https://github.com/dotnet/corefx/pull/2974 merged, System.IO.Pipes builds on FreeBSD! :sunglasses:

ghost commented 8 years ago

Update: dotnet/corefx#2527 closed, System.IO.MemoryMappedFiles builds on FreeBSD. Thanks @janhenke for the confirmation!

janhenke commented 8 years ago

Thanks to the shims approach, it just comes down to make sure the shims compile on FreeBSD. Thankfully that makes life a lot easier. :)

janhenke commented 8 years ago

dotnet/corefx#3257 should bring us both System.Diagnostic.Process and System.IO.FileSystem.Watcher leaving just System.Net.NameResolution unresolved. (I will check the mentioned two assemblies once the PR is merged and works on FreeBSD)

janhenke commented 8 years ago

dotnet/corefx#3471 should bring us System.Net.NameResolution and complete the list above.

jonmill commented 8 years ago

dotnet/corefx#3471 was just merged :)

ghost commented 8 years ago

@sokket, thanks for the update. I built master (f467911) on FreeBSD using this guide: https://gist.github.com/jasonwilliams200OK/6efa7907e66275df2d24. Current blocker is https://github.com/dotnet/buildtools/issues/292, which is fixed in upstream but waiting for next buildtools roll-out. :)

ghost commented 8 years ago

Update: new buildtools with fix for dotnet/buildtools#292 has landed in CoreFX master. Next stopper from buildtools is https://github.com/dotnet/buildtools/issues/300: missing OS specific tool to be able to run the tests.

@janhenke, you have marked System.Diagnostics.Process (#2070) and System.IO.FileSystem.Watcher (#2046) as done; but they are neither implemented nor do they compile on FreeBSD. Have you actually verified the list by compiling the managed code?

Based on my recent experience with commit 60c78da3c918b0d256cc1f878de06d351dbe3342 (see msbuild.log), following assemblies do not compile:

janhenke commented 8 years ago

As far as I recall I verified the associated shims compile. Since the managed code should be free of FreeBSD specific code. Those shims you mention should have been shimed out with the PRs linked above. But I have also run a full compile in between. At the very least System.Diagnostics.ThreadInfoand System.IO.FileSystemWatcher did compile. So something must have regressed.

ghost commented 8 years ago

Those shims you mention should have been shimed out with the PRs linked above.

Actually, PR https://github.com/dotnet/corefx/pull/3257 is not related to shim. There is still some PAL code within the managed projects (the old approach), therefore it is required to build managed assemblies to be absolutely sure.

janhenke commented 8 years ago

Actually, PR dotnet/corefx#3257 is not related to shim.

I disagree. It is refactoring the P/Invoke code to the System.Native shim. Also as I edited above, I am recalling at least some of the assemblies compiled in between.

ghost commented 8 years ago

I disagree

https://github.com/dotnet/corefx/pull/3257/files: see the instances of .Unix.cs and .Linux.cs for System.Diagnostics.. Note that .OSX.cs is untouched.

It is refactoring the P/Invoke code to the System.Native shim

Yes it does refactor some common helpers under System.Native, but not System.Diagnostics.* et al.

stephentoub commented 8 years ago

Even when these assemblies are only P/Invoking to System.* libs, there may still be FreeBSD work required for some of them, e.g. System.Diagnostics.Process and System.IO.FileSystem.Watcher. They are using functionality specific to Linux and OS X, and we don't plan to try to abstract that behind native shims. The goal of the shims isn't to end up with a single managed binary for Unix, though that's a very nice property when it comes from the work; the primary goal is to avoid ABI differences that cause fragility. I expect at least a handful of assemblies will continue to have Linux/OS X specific binaries, where a FreeBSD binary would also be needed.

stephentoub commented 8 years ago

FYI, there are no corefx assemblies named System.Diagnostics.ProcessManager, System.Diagnostics.ThreadInfo, System.IO.FileSystemWatcher, or System.Net.SocketAddress. Those are types in other assemblies.