BinaryToolKit fails during ./prep-source-build.sh on arm64

[dviererbe]

Describe the bug

The BinaryToolKit fails during ./prep-source-build.sh on arm64.

To Reproduce

(On a rented Hetzner CAV21 instance (4vCPUs (Ampere), 8GB RAM) with Ubuntu 24.04 LTS (Noble Numbat))

1. git clone https://github.com/dotnet/dotnet.git
2. cd dotnet
3. git checkout v9.0.0-preview.7.24405.7
4. ./prep-source-build.sh

Exceptions (if any)

12:50:15 info: BinaryTool[0] Starting binary tool at 8/22/2024 12:50:15 PM in Clean mode
12:50:15 info: BinaryTool[0] Creating directory '/home/dviererbe/dotnet/artifacts/log/binary-report' for 'OutputReportDirectory'.
12:50:15 info: BinaryTool[0] Detecting binaries in '/home/dviererbe/dotnet' not listed in '/home/dviererbe/dotnet/eng/allowed-sb-binaries.txt'...
Unhandled exception: System.ComponentModel.Win32Exception (2): An error occurred trying to start process 'file' with working directory '/home/dviererbe/dotnet'. No such file or directory
   at System.Diagnostics.Process.ForkAndExecProcess(ProcessStartInfo startInfo, String resolvedFilename, String[] argv, String[] envp, String cwd, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec)
   at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start(ProcessStartInfo startInfo)
   at BinaryToolKit.DetectBinaries.ExecuteProcessAsync(String executable, String arguments) in /home/dviererbe/dotnet/eng/tools/BinaryToolKit/DetectBinaries.cs:line 97
   at BinaryToolKit.DetectBinaries.IsNotUTF16Async(String file) in /home/dviererbe/dotnet/eng/tools/BinaryToolKit/DetectBinaries.cs:line 75
   at BinaryToolKit.DetectBinaries.IsBinaryAsync(String filePath) in /home/dviererbe/dotnet/eng/tools/BinaryToolKit/DetectBinaries.cs:line 64
   at BinaryToolKit.DetectBinaries.<>c__DisplayClass3_0.<<ExecuteAsync>b__0>d.MoveNext() in /home/dviererbe/dotnet/eng/tools/BinaryToolKit/DetectBinaries.cs:line 31
--- End of stack trace from previous location ---
   at BinaryToolKit.DetectBinaries.ExecuteAsync(String targetDirectory, String outputReportDirectory, String allowedBinariesFile) in /home/dviererbe/dotnet/eng/tools/BinaryToolKit/DetectBinaries.cs:line 34
   at BinaryToolKit.BinaryTool.ExecuteAsync(String targetDirectory, String outputReportDirectory, String allowedBinariesFile, Modes mode) in /home/dviererbe/dotnet/eng/tools/BinaryToolKit/BinaryTool.cs:line 39
   at BinaryToolKit.Program.<>c__DisplayClass7_0.<<SetCommandAction>b__0>d.MoveNext() in /home/dviererbe/dotnet/eng/tools/BinaryToolKit/Program.cs:line 79
--- End of stack trace from previous location ---
   at System.CommandLine.CliCommand.<>c__DisplayClass32_0.<<SetAction>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Invocation.InvocationPipeline.InvokeAsync(ParseResult parseResult, CancellationToken cancellationToken)

Further technical details

$ .dotnet/dotnet --info
.NET SDK:
 Version:           9.0.100-preview.7.24380.2
 Commit:            1e16d3702a
 Workload version:  9.0.100-manifests.28aae763
 MSBuild version:   17.12.0-preview-24374-02+48e81c6f1

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  24.04
 OS Platform: Linux
 RID:         linux-arm64
 Base Path:   /home/dviererbe/dotnet/.dotnet/sdk/9.0.100-preview.7.24380.2/

.NET workloads installed:
Configured to use loose manifests when installing new manifests.
There are no installed workloads to display.

Host:
  Version:      9.0.0-preview.7.24376.15
  Architecture: arm64
  Commit:       static

.NET SDKs installed:
  9.0.100-preview.7.24380.2 [/home/dviererbe/dotnet/.dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 9.0.0-preview.7.24379.2 [/home/dviererbe/dotnet/.dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 9.0.0-preview.7.24376.15 [/home/dviererbe/dotnet/.dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
  None

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

$ neofetch --off
dviererbe@ubuntu-8gb-hel1-1 
--------------------------- 
OS: Ubuntu 24.04 LTS aarch64 
Host: vServer 20171111 
Kernel: 6.8.0-38-generic 
Uptime: 2 hours, 53 mins 
Packages: 672 (dpkg) 
Shell: bash 5.2.21 
Resolution: 1024x768 
Terminal: /dev/pts/0 
CPU: (4) 
GPU: 00:01.0 Red Hat, Inc. Virtio 1.0 GPU 
Memory: 403MiB / 7729MiB

[dviererbe]

Running this on the same instance in an lxc container seems to work. This must have to do with the hetzner Ubuntu image.

[dotnet-issue-labeler[bot]]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[dotnet-issue-labeler[bot]]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[MichaelSimons]

@ellahathaway - Can you investigate? TIA

[omajid]

@dviererbe what does command -v file say on this machine?

[ellahathaway]

what does command -v file say on this machine?

I'm also wondering this. The part of the code that's failing is here, when the BinaryToolKit runs file <filename> as a process. The failure makes me think that the file command is unavailable.

[dviererbe]

what does command -v file say on this machine?

I'm also wondering this. The part of the code that's failing is here, when the BinaryToolKit runs file <filename> as a process. The failure makes me think that the file command is unavailable.

@omajid @ellahathaway Yes that was it:

$ command -v file
$ echo $?
1

After apt install file, the ./prep-source-build.sh script worked as expected.

Thanks and sorry for wasting your time a little. I should have noticed that too.