search

sec / dotnet-core-freebsd-source-build

Collection of script to build .NET Core under FreeBSD OS (with binary releases)
MIT License
53 stars 4 forks source link

Service index errors on ./build_runtime.sh #8

Closed greggyb closed 2 years ago

greggyb commented 2 years ago

FreeBSD 13.0 host. FreeBSD 13.0-release jail (using BastilleBSD from pkg).

Jail config:

dn {
  devfs_ruleset = 4;
  enforce_statfs = 2;
  exec.clean;
  exec.consolelog = /var/log/bastille/dn_console.log;
  exec.start = '/bin/sh /etc/rc';
  exec.stop = '/bin/sh /etc/rc.shutdown';
  host.hostname = dn;
  mount.devfs;
  mount.fstab = /usr/local/bastille/jails/dn/fstab;
  path = /usr/local/bastille/jails/dn/root;
  securelevel = 2;

  interface = bastille0;
  ip4.addr = 10.0.0.13;
  ip6 = disable;
  allow.mount=1;
  allow.mlock=1;
}
  1. ./install_tools.sh (problem with lttng-ust -- does not exist except as linux compat package 'linux-c7-lttng-ust', updated the script to reference this)
  2. ./init.sh
  3. ./build_runtime.sh: errors
root@dn:~/git/dotnet-core-freebsd-source-build # ./build_runtime.sh                                                                           
__DistroRid: freebsd-x64
##vso[task.setvariable variable=Artifacts;isSecret=false;isOutput=true]/root/git/dotnet-core-freebsd-source-build/runtime/artifacts           
##vso[task.setvariable variable=Artifacts.Toolset;isSecret=false;isOutput=true]/root/git/dotnet-core-freebsd-source-build/runtime/artifacts/toolset
##vso[task.setvariable variable=Artifacts.Log;isSecret=false;isOutput=true]/root/git/dotnet-core-freebsd-source-build/runtime/artifacts/log/Release
##vso[task.setvariable variable=Temp;isSecret=false;isOutput=true]/root/git/dotnet-core-freebsd-source-build/runtime/artifacts/tmp/Release
##vso[task.setvariable variable=TMP;isSecret=false;isOutput=true]/root/git/dotnet-core-freebsd-source-build/runtime/artifacts/tmp/Release     
Attempting to log this clients IP for Azure Package feed telemetry purposes  
##vso[task.prependpath]/root/git/dotnet-core-freebsd-source-build/runtime//.dotnet                                                            
##vso[task.setvariable variable=DOTNET_MULTILEVEL_LOOKUP;isSecret=false;isOutput=true]0
##vso[task.setvariable variable=DOTNET_SKIP_FIRST_TIME_EXPERIENCE;isSecret=false;isOutput=true]1                                              
/root/git/dotnet-core-freebsd-source-build/runtime/.dotnet/sdk/6.0.100/MSBuild.dll /nologo -maxcpucount /m -verbosity:m /v:minimal /bl:/root/git/dotnet-core-freebsd-source-build/runtime/artifacts/log/Release
/ToolsetRestore.binlog /clp:Summary /clp:ErrorsOnly;NoSummary /nr:false /p:TreatWarningsAsErrors=true /p:ContinuousIntegrationBuild=true /p:__ToolsetLocationOutputFile=/root/git/dotnet-core-freebsd-source-bu
ild/runtime/artifacts/toolset/6.0.0-beta.21560.2.txt /t:__WriteToolsetLocation /warnaserror /root/git/dotnet-core-freebsd-source-build/runtime/artifacts/toolset/restore.proj
/root/git/dotnet-core-freebsd-source-build/runtime/artifacts/toolset/restore.proj : error : Unable to load the service index for source https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-public/nuget
/v3/index.json.                                    
/root/git/dotnet-core-freebsd-source-build/runtime/artifacts/toolset/restore.proj : error : Unable to load the service index for source https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/
v3/index.json.                                     
/root/git/dotnet-core-freebsd-source-build/runtime/artifacts/toolset/restore.proj : error : Unable to load the service index for source https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3
/index.json.                                       
/root/git/dotnet-core-freebsd-source-build/runtime/artifacts/toolset/restore.proj : error : Unable to load the service index for source https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet6/nuget/v3/in
dex.json.                                          
/root/git/dotnet-core-freebsd-source-build/runtime/artifacts/toolset/restore.proj : error : Unable to load the service index for source https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet6-transport/n
uget/v3/index.json.                                
/root/git/dotnet-core-freebsd-source-build/runtime/artifacts/toolset/restore.proj : error : Unable to load the service index for source https://pkgs.dev.azure.com/azure-public/vside/_packaging/vs-buildservic
es/nuget/v3/index.json.                            
/root/git/dotnet-core-freebsd-source-build/runtime/artifacts/toolset/restore.proj : error : Unable to find package Microsoft.DotNet.Arcade.Sdk. No packages exist with this id in source(s): dotnet-eng, dotnet
-public, dotnet-tools, dotnet6, dotnet6-transport, richnav                                             
/root/git/dotnet-core-freebsd-source-build/runtime/artifacts/toolset/restore.proj : error MSB4236: The SDK 'Microsoft.DotNet.Arcade.Sdk' specified could not be found.
Build failed with exit code 1. Check errors above.                                                     
##vso[task.complete result=Failed;]msbuild execution failed.
sec commented 2 years ago

Hi, For jail build - I haven't tested that, but it needs live internet connection for sure, so check that first (also there can be some ipc/mmap issues when running dotnet inside jail, so those should be also enabled, I think more details on that can be found in As for lttng, I know it's no longer available in pkg, it's been addressed already in https://github.com/dotnet/runtime/pull/63567 (you can follow the thread here https://github.com/dotnet/runtime/issues/14537#issuecomment-1006309245).

Please let me know if this helps, as this looks like duplicate of #6

greggyb commented 2 years ago

Thanks for the rapid reply. Will take a look at your suggestions today and follow up here. (I'm also trying out some linux jails to see if it's more straightforward to use a packaged version of dotnet with linuxulator).

Quick notes:

6 is addressed by allow.mlock for the jail.

sec commented 2 years ago

Hm, then maybe it's blacklisted certificate error - have you made the fix for that (it's in the README of the repo) - I know that package validation have been disabled for FreeBSD also in main dotnet repo, but don't know if those changes are already live.

btw. last time I checked dotnet using linuxulator it wasn't working because of missing syscall's (initial plan was to use Linux binaries to bootstrap native build, but not possible at the moment, that's why we're using cross-build output).

edit: also lttng-ust issue is now resolved in main branch (added patch and removed from pkg install)

greggyb commented 2 years ago

Some more digging:

Changing jail settings has not made any difference:

I applied the change from the README: 

mv /usr/share/certs/blacklisted/VeriSign_Universal_Root_Certification_Authority.pem /usr/share/certs/trusted
certctl rehash

This also made no difference.

I also get the same error when attempting to build a simple project in a Debian Buster jail environment. I followed the official instructions for installing the dotnet apt package. This yields a dotnet installation that can do some basic things (such as tell you its version or launch an fsi interactive session).

Attempting to build a project with no Nuget dependencies yields an error:

root@bust:~/letterbox# dotnet build
Microsoft (R) Build Engine version 17.0.0+c9eb9dd64 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
/usr/share/dotnet/sdk/6.0.101/NuGet.targets(130,5): error : Unable to load the service index for source https://api.nuget.org/v3/index.json. [/root/letterbox/letterbox.fsproj]
/usr/share/dotnet/sdk/6.0.101/NuGet.targets(130,5): error :   Protocol not supported (api.nuget.org:443) [/root/letterbox/letterbox.fsproj]
/usr/share/dotnet/sdk/6.0.101/NuGet.targets(130,5): error :   Protocol not supported [/root/letterbox/letterbox.fsproj]

Build FAILED.

/usr/share/dotnet/sdk/6.0.101/NuGet.targets(130,5): error : Unable to load the service index for source https://api.nuget.org/v3/index.json. [/root/letterbox/letterbox.fsproj]
/usr/share/dotnet/sdk/6.0.101/NuGet.targets(130,5): error :   Protocol not supported (api.nuget.org:443) [/root/letterbox/letterbox.fsproj]
/usr/share/dotnet/sdk/6.0.101/NuGet.targets(130,5): error :   Protocol not supported [/root/letterbox/letterbox.fsproj]
    0 Warning(s)
    1 Error(s)

Time Elapsed 00:00:00.70

Note this line, the first error from the build:

/usr/share/dotnet/sdk/6.0.101/NuGet.targets(130,5): error : Unable to load the service index for source https://api.nuget.org/v3/index.json. [/root/letterbox/letterbox.fsproj]

This is the same error text as in the build_runtime.sh step:

(error excerpted from original issue error)

/root/dotnet-core-freebsd-source-build/runtime/artifacts/toolset/restore.proj : error : Unable to load the service index for source https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-public/nuget/v3/index.json.

In both cases, we see the build failing when attempting to load the service index. Both seem to be looking for a nuget repository at path .../v3/index.json.

I'm not sure what to do with this, but it appears that the issue is similar across a Linux jail and a FreeBSD jail environment, so I expect there's some issue with the jail config. Unfortunately, I'm not sure where to look next for this.

Note that I have network connectivity, including working DNS resolution and I have allowed raw sockets, so I should be able to do any network operations.

sec commented 2 years ago

Well, if you can build outside of jail - then you know where the issue is :) You could try to check if fetch https://api.nuget.org/v3/index.json works inside jail, but I assume it is, as you wrote you have net/dns working.

greggyb commented 2 years ago

Fetch works fine. This seems to be some dotnet functionality that is failing when running a build.

I hit other, later issues building outside of a jail (in build_installer.sh) that I still need to troubleshoot.

greggyb commented 2 years ago

So, I was able to find a solution here. Short version: the jail must have an ip6 address assigned (even if it's just a loopback address).

More in this issue on the same error: https://github.com/mono/mono/issues/21236

Specific comment with solution: https://github.com/mono/mono/issues/21236#issuecomment-948443375

I would recommend adding a note to this effect in the README. Sample to add an ip6 address with Bastille:

bastille config <jail-name> set ip6.address = fe80::2 # loopback address after the default assigned to host

Sample from jail.conf:

jail-name {
  interface = bastille0;
  ip4.addr = 10.0.0.11;
  ip6.addr = fe80::2;
}

Note that you must make sure you do not also have a line with ip6 = disable;.

greggyb commented 2 years ago

Still troubleshooting the build, but this specific issue is done.