NetBSD buildslave?

[krytarowski]

We are already building 100% CoreCLR, 100% CoreFX, 100% CoreRT via pkgsrc-wip/NetBSD.

We are passing almost 90% of PAL tests (https://github.com/dotnet/coreclr/issues/3123)

I think it's about right time to plan NetBSD build machine. What are the requirements for one?

[ellismg]

Do you know if it's possible to run NetBSD in Azure? I know FreeBSD is support and I've seen a few references to a Google Summer of Code project to get NetBSD in Azure as well, but I'm not sure what the status is.

If this is supported, that makes things a lot easier because we already have a well defined system for using Azure resources in our CI system. If we would need to host something ourselves, it becomes a little more difficult.

It would be also helpful to document getting a NetBSD system up and running and installing all the necessary stuff to have a build environment.

[krytarowski]

At the moment there is no Azure support. I don't know who to talk in Microsoft/Azure-team to get one. I don't know the technology -- it would be nice to get a reference implementation.

I saw a reference to OpenBSD and FreeBSD port.

Regarding packages -- I think it's best to wait for the beginning of April to get pkgsrc-2016Q1 out. I will push there LLDB and LLVM libunwind. Otherwise we will need to use pkgsrc-wip/custom packages for them (and build from sources).

[jack-pappas]

More info on NetBSD on Azure: NetBSD/azure

Porting CoreCLR to NetBSD is one thing, adding NetBSD to the CI build infrastructure is a different story. I think NetBSD will need to port some/all of the Hyper-V drivers over from FreeBSD so it can run on an Azure VM.

[krytarowski]

https://wiki.freebsd.org/HyperV FreeBSD Virtual Machines on Microsoft Hyper-v

[krytarowski]

https://github.com/dotnet/coreclr/pull/3219 just broke NetBSD building...

[krytarowski]

Can we go temporarily with some walk-around for now? Like installing NetBSD in Linux kvm, which is installed in Azure?

[krytarowski]

lldb and libunwind are in pkgsrc-2016Q1, all dependencies are aboard in packages

[ghost]

Probably a good idea to create a repo like https://github.com/FreeBSDonHyper-V/hv-kvp and then port hv-kvp to NetBSD. After that there is integration service: https://github.com/FreeBSDonHyper-V/FreeBSD-Integration-Service and Azure automation https://github.com/FreeBSDonHyper-V/azure-freebsd-automation.

This will probably complete the pre-reqs to build NetBSD VM on Azure adn upload to https://vmdepot.msopentech.com/List/Index?sort=Featured&search=freebsd.

[krytarowski]

CoreCLR builds in the upstream version and passes all PAL tests.

Can I develop HyperV port without Windows? I don't use nonnetbsd machines, I can access remotely a Linux one..

[ghost]

I think at this point NetBSD has better CoreCLR feature coverage than FreeBSD. So :+1: for the NetBSD buildslave. :)

In my understanding:

• To configure the build slave on CI server, we need to first note that the dotnet Jenkins is running on Windows Azure cloud.
• To run NetBSD VM on Windows Azure cloud, NetBSD needs to be Hyper-V compliant.

• To make NetBSD Hyper-V compliant, some tools need to be ported first:

  • hv-kvp
  • Intergration-Service
  • Azure-automation

  (which I believe would be straight forward to port, mostly written in C and couple of shell scripts and those tools available for FreeBSD as are pretty mature)

Otherwise, as you pointed out, NetBSD VM can be configured to run on top of any Azure Linux/FreeBSD VM.

[krytarowski]

I don't want to make lack of Azure support right now blocking. Azure isn't relevant to .Net.

[ghost]

Azure isn't relevant to .Net.

CoreCLR CI server runs on Azure.

[krytarowski]

Anyway someone from the CI team should tell what's the status.

[ellismg]

If you can provide step by step instructions on how to take a FreeBSD system and set up whatever additional software we need to get NetBSD working (and then how in NetBSD to set up the tool chain) we can start to figure out how to get this to work in our CI system.

[ellismg]

Within NetBSD we also need to install Java (since the Jenkins daemon needs it) so if you have instructions on how to do that as well, it would be helpful.

[krytarowski]

I have got prepared a script for Linux and qemu+kvm:

http://mail-index.netbsd.org/netbsd-users/2015/10/05/msg017003.html

ftp://ftp.netbsd.org/pub/NetBSD/misc/kamil/autoinst.sh

NetBSD-7.0 had a bug with virtio so please use the standard driver for networking and disk (not virtio).

[krytarowski]

I had the following scripts:

$ cat install-netbsd-novirtio.sh
#!/bin/sh

qemu-system-x86_64 \
    -enable-kvm \
    -net user \
    -net nic \
    -m 14G \
    -cdrom /home/kamil/netbsd-7.0/boot-com.iso \
    -nographic \
    -smp cores=6 \
    -hda /home/kamil/obrazy-kvm/netbsd7.0_amd64 \
    -boot d
$ cat run-netbsd-novirtio.sh
#!/bin/sh

qemu-system-x86_64 \
    -enable-kvm \
    -net user \
    -net nic \
    -m 14G \
    -nographic \
    -smp cores=6 \
    -hda /home/kamil/obrazy-kvm/netbsd7.0_amd64 \
    -boot c

Once you will get a functional system, I will tell you how to get the dependencies and Jenkins.

[nbyouri]

Jenkins and Java, which Jenkins depends on are available in pkgsrc and should work on NetBSD.

[krytarowski]

My suggested steps to start:

1. Get installed the system with autoinst.sh
2. Get pkgsrc-wip to build coreclr-git with its dependencies:

cd /usr/pkgsrc/devel/git-base && make install
cd /usr/pkgsrc && git clone --depth 1 git://wip.pkgsrc.org/pkgsrc-wip.git wip

1. Build coreclr-git

cd /usr/pkgsrc/wip/coreclr-git && make

1. Run PAL tests

cd /usr/pkgsrc/wip/coreclr-git && make test

1. Install Java and Jenkins @nbyouri might help with it hopefully.

[krytarowski]

I was told that to run Jenkins we need to install:

• Java

cd /usr/pkgsrc/lang/openjdk8 && make install

• Jenkins

cd /usr/pkgsrc/devel/jenkins && make install

I got example commands to run:

/usr/pkg/java/openjdk8/bin/java -Duser.home=/path/to/somewhere -jar /usr/pkg/share/jenkins/jenkins.war --httpPort=10080

/usr/pkg/java/openjdk8/bin/java -jar /usr/pkg/share/jenkins/jenkins.war

The rest is up to you how to configure it.

[nbyouri]

Alternatively, you could use packages to avoid wasting time compiling. Jenkins and Java are available here: ftp://ftp.netbsd.org/pub/pkgsrc/packages/NetBSD/amd64/7.0_2015Q4/All/

[krytarowski]

@nbyouri we need 2016Q1 as there is llvm+lldb+clang+libunwind in proper version (autoinst.sh will fetch this version of pkgsrc and allow to build them from sources). Not sure when the prebuilt packages will be uploaded.

[krytarowski]

Martin Husemann from the releng@ team told me that NetBSD-7.0.1 will be released pretty soon. While it's good to exercise with 7.0, we can setup the bot with 7.0.1 (we can easily upgrade later from 7.0). The 7.0.1 version should have the virtio bug squashed.

[krytarowski]

For the reference, we got a volunteer to work on Azure for NetBSD during the Google Summer of Code (by @retrage).

https://summerofcode.withgoogle.com/organizations/6246531984261120/?sp-page_size=48#4762276204642304

I wish all best to the project and fill the missing Azure gap.

[krytarowski]

Any progress on it? For what I know, NetBSD works on Azure without hardware acceleration.

[XVilka]

@krytarowski has anything changed with this?

[krytarowski]

@XVilka Yes

1. Hyper-V support has been developed but still not merged with mainline NetBSD
2. Hardware-assisted virtualization (other than Xen) for NetBSD has been added. I've ported HAXM ( http://blog.netbsd.org/tnf/entry/the_hardware_assisted_virtualization_challenge ) for this purpose, but we are growing a native NVMM engine for this purpose as well (right now only AMD CPUs are supported, Intel coming soon)
3. LLDB/ptrace(2) porting for debugging crashes of .NET code on NetBSD is ongoing.
4. Robust pthread(3) mutexes are not started.. however we have added pshared support for semaphores.

Work is ongoing towards getting fully support for .NET on NetBSD... help is welcome. CoreCLR certainly rotted a bit without a buildslave.

[krytarowski]

OK, hyperv landed the NetBSD sources!

https://twitter.com/nonakap/status/1096334488945123328

@zaydogan offered help to write a tutorial how to create a NetBSD image for Azure.

[ericstj]

@wfurt do we have any plans to build / support NetBSD

[wfurt]

it seems unlikely at the moment - at least as an official platform. There is no Azure support AFAIK so this will be limited to community support.

[krytarowski]

NetBSD 9.0 included hyper-v support so it should be now easier... however I am still preempted with generic NetBSD enhancements.

[wfurt]

FreeBSD is now cros-compiled using Linux container (to make it easier) That may work for NetBSD as well to avoid hassle with bootstrap cli and Azure support.