search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.46k stars 4.76k forks source link

.NET runtime not found on Ubuntu #79237

Closed ptr727 closed 1 year ago

ptr727 commented 1 year ago

Description

My weekly GitHub actions build started failing. I suspect some change in Ubuntu .NET 6 results in the apps no longer running, and complaining no runtime found.

Docker:

# Build externally
# dotnet publish ./PlexCleaner/PlexCleaner.csproj --runtime linux-x64 --self-contained false --output ./Docker/PlexCleaner
COPY PlexCleaner /PlexCleaner
RUN dotnet --info \
    && /PlexCleaner/PlexCleaner --version

Actions error:

#15 [10/10] RUN dotnet --info     && /PlexCleaner/PlexCleaner --version
#15 0.222 .NET SDK (reflecting any global.json):
#15 0.222  Version:   6.0.111
#15 0.222  Commit:    b3bb659a9d
#15 0.222 
#15 0.222 Runtime Environment:
#15 0.224  OS Name:     ubuntu
#15 0.224  OS Version:  22.04
#15 0.227  OS Platform: Linux
#15 0.251  RID:         ubuntu.22.04-x64
#15 0.251  Base Path:   /usr/lib/dotnet/sdk/6.0.111/
#15 0.254 
#15 0.254 global.json file:
#15 0.254   Not found
#15 0.254 
#15 0.254 Host:
#15 0.254   Version:      6.0.11
#15 0.254   Architecture: x64
#15 0.254   Commit:       943474ca16
#15 0.254 
#15 0.254 .NET SDKs installed:
#15 0.254   6.0.111 [/usr/lib/dotnet/sdk]
#15 0.254 
#15 0.254 .NET runtimes installed:
#15 0.254   Microsoft.AspNetCore.App 6.0.11 [/usr/lib/dotnet/shared/Microsoft.AspNetCore.App]
#15 0.254   Microsoft.NETCore.App 6.0.11 [/usr/lib/dotnet/shared/Microsoft.NETCore.App]
#15 0.254 
#15 0.254 Download .NET:
#15 0.254   https://aka.ms/dotnet-download
#15 0.254 
#15 0.254 Learn about .NET Runtimes and SDKs:
#15 0.254   https://aka.ms/dotnet/runtimes-sdk-info
#15 0.262 You must install .NET to run this application.
#15 0.262 
#15 0.262 App: /PlexCleaner/PlexCleaner
#15 0.262 Architecture: x64
#15 0.262 App host version: 6.0.11
#15 0.262 .NET location: Not found
#15 0.262 
#15 0.262 Learn about runtime installation:
#15 0.262 https://aka.ms/dotnet/app-launch-failed
#15 0.262 
#15 0.262 Download the .NET runtime:
#15 0.262 https://aka.ms/dotnet-core-applaunch?missing_runtime=true&arch=x64&rid=ubuntu.22.04-x64&apphost_version=6.0.11
#15 ERROR: process "/bin/sh -c dotnet --info     && /PlexCleaner/PlexCleaner --version" did not complete successfully: exit code: 131

Full error: https://github.com/ptr727/PlexCleaner/actions/runs/3622494725/jobs/6107272543

Reproduction Steps

Install .NET 6 on ubuntu:latest. Compile .NET core app in github actions step, copy .NET app to docker ditnet --info works. Run .NET app, complains no .NET found?

Expected behavior

Work as before.

Actual behavior

.NET location: Not found

Regression?

Yes, this worked in previous ubuntu:latest release, no code changes, github actions build just started failing.

Known Workarounds

No response

Configuration

https://github.com/ptr727/PlexCleaner/actions/runs/3616398576

Other information

No response

ghost commented 1 year ago

Tagging subscribers to this area: @vitek-karas, @agocke, @vsadov See info in area-owners.md if you want to be subscribed.

Issue Details
### Description My weekly GitHub actions build started failing. I suspect some change in Ubuntu .NET 6 results in the apps no longer running, and complaining no runtime found. Docker: ``` # Build externally # dotnet publish ./PlexCleaner/PlexCleaner.csproj --runtime linux-x64 --self-contained false --output ./Docker/PlexCleaner COPY PlexCleaner /PlexCleaner RUN dotnet --info \ && /PlexCleaner/PlexCleaner --version ``` Actions error: ``` #15 [10/10] RUN dotnet --info && /PlexCleaner/PlexCleaner --version #15 0.222 .NET SDK (reflecting any global.json): #15 0.222 Version: 6.0.111 #15 0.222 Commit: b3bb659a9d #15 0.222 #15 0.222 Runtime Environment: #15 0.224 OS Name: ubuntu #15 0.224 OS Version: 22.04 #15 0.227 OS Platform: Linux #15 0.251 RID: ubuntu.22.04-x64 #15 0.251 Base Path: /usr/lib/dotnet/sdk/6.0.111/ #15 0.254 #15 0.254 global.json file: #15 0.254 Not found #15 0.254 #15 0.254 Host: #15 0.254 Version: 6.0.11 #15 0.254 Architecture: x64 #15 0.254 Commit: 943474ca16 #15 0.254 #15 0.254 .NET SDKs installed: #15 0.254 6.0.111 [/usr/lib/dotnet/sdk] #15 0.254 #15 0.254 .NET runtimes installed: #15 0.254 Microsoft.AspNetCore.App 6.0.11 [/usr/lib/dotnet/shared/Microsoft.AspNetCore.App] #15 0.254 Microsoft.NETCore.App 6.0.11 [/usr/lib/dotnet/shared/Microsoft.NETCore.App] #15 0.254 #15 0.254 Download .NET: #15 0.254 https://aka.ms/dotnet-download #15 0.254 #15 0.254 Learn about .NET Runtimes and SDKs: #15 0.254 https://aka.ms/dotnet/runtimes-sdk-info #15 0.262 You must install .NET to run this application. #15 0.262 #15 0.262 App: /PlexCleaner/PlexCleaner #15 0.262 Architecture: x64 #15 0.262 App host version: 6.0.11 #15 0.262 .NET location: Not found #15 0.262 #15 0.262 Learn about runtime installation: #15 0.262 https://aka.ms/dotnet/app-launch-failed #15 0.262 #15 0.262 Download the .NET runtime: #15 0.262 https://aka.ms/dotnet-core-applaunch?missing_runtime=true&arch=x64&rid=ubuntu.22.04-x64&apphost_version=6.0.11 #15 ERROR: process "/bin/sh -c dotnet --info && /PlexCleaner/PlexCleaner --version" did not complete successfully: exit code: 131 ``` Full error: https://github.com/ptr727/PlexCleaner/actions/runs/3622494725/jobs/6107272543 ### Reproduction Steps Install .NET 6 on ubuntu:latest. Compile .NET core app in github actions step, copy .NET app to docker `ditnet --info` works. Run .NET app, complains no .NET found? ### Expected behavior Work as before. ### Actual behavior `.NET location: Not found` ### Regression? Yes, this worked in previous ubuntu:latest release, no code changes, github actions build just started failing. ### Known Workarounds _No response_ ### Configuration https://github.com/ptr727/PlexCleaner/actions/runs/3616398576 ### Other information _No response_
Author: ptr727
Assignees: -
Labels: `area-Host`, `untriaged`
Milestone: -
Clockwork-Muse commented 1 year ago

Install .NET 6 on ubuntu:latest.

Side note: Is there some reason you're doing this yourself, instead of using one of the pre-built .net images?

ptr727 commented 1 year ago

Install .NET 6 on ubuntu:latest.

Side note: Is there some reason you're doing this yourself, instead of using one of the pre-built .net images?

Pre-built images did not exist when I started my project circa .NET Core 3 (at least I don't think they existed?).
I'll try switching when I switch my project to .NET 7.

As for this issue, I found several other reports of the .11 .NET build that is included in Ubuntu being broken, and to switch to installing hardcoded .8 builds.

As for me, I just reverted to installing using PMC and not relying on Ubuntu native support.

# Install .NET runtime using PMC
# Ubuntu 22.04 / Jammy includes .NET, set correct order when using packages-microsoft-prod.deb
# https://github.com/dotnet/core/issues/7699
RUN wget https://packages.microsoft.com/config/ubuntu/$(lsb_release -sr)/packages-microsoft-prod.deb \
    && dpkg -i packages-microsoft-prod.deb \
    && rm packages-microsoft-prod.deb \
    && touch /etc/apt/preferences \
    && echo "Package: *" >> /etc/apt/preferences \
    && echo "Pin: origin \"packages.microsoft.com\"" >> /etc/apt/preferences \
    && echo Pin-Priority: 1001 >> /etc/apt/preferences \
    && cat /etc/apt/preferences \
    && cat /etc/apt/sources.list.d/microsoft-prod.list \
    && apt-get update \
    && apt-get install -y dotnet-runtime-6.0 \
    && dotnet --info
agocke commented 1 year ago

This combo looks suspicious:

15 0.254 .NET SDKs installed:

15 0.254 6.0.111 [/usr/lib/dotnet/sdk]

15 0.262 App: /PlexCleaner/PlexCleaner

15 0.262 Architecture: x64

15 0.262 App host version: 6.0.11

15 0.262 .NET location: Not found

The default dotnet install location on Linux is /usr/share/dotnet, but this is /usr/lib/dotnet.

You can change this by putting a different path in /etc/dotnet/install_location, but I'm not sure if that was created by this installer.

vitek-karas commented 1 year ago

As far as I can tell this is likely due to wrong installation. The fact that dotnet --info works means that the installation itself is fine and can run (that command runs managed code, so runtime works). But then it fails when running PlexCleaner executable, so apphost. So I would guess that:

If the installation of the runtime is supposed to be global, then it needs to be registered in /etc/dotnet/install_location. It would also work if it was in the default location which is /usr/share/dotnet, but since it's not there it needs to be registered.

If the installation is supposed to be only for running the app and not global to the machine, then it needs to set DOTNET_ROOT environment variable - since otherwise the executable has no way of finding the runtime.

If you want to be 100% about this, you can set COREHOST_TRACE=1 before running the app, it should print out detailed log of where it looked for the runtime and why.

The key question is - how did you install the SDK/runtime?

ptr727 commented 1 year ago

The key question is - how did you install the SDK/runtime?

Using apt-get install -y dotnet-sdk-6.0 on a vanilla Ubuntu image, see

Clockwork-Muse commented 1 year ago

Pre-built images did not exist when I started my project circa .NET Core 3 (at least I don't think they existed?). I'll try switching when I switch my project to .NET 7.

3.1 images exist, although I'm not sure when they were first created.

ghost commented 1 year ago

DOTNET_ROOT=/usr/lib/dotnet solved it for me. I was setting DOTNET_ROOT to something else in my bashrc.

vitek-karas commented 1 year ago

@markhiveit thanks - DOTNET_ROOT will override any other configuration, so if it points to the wrong location, the app will simply not work at all.

ghost commented 1 year ago

@markhiveit thanks - DOTNET_ROOT will override any other configuration, so if it points to the wrong location, the app will simply not work at all.

The odd thing was that it was working up until very recently with DOTNET_ROOT set in my bashrc - it was pointing to ${HOME}/.config/dotnet. It's possible I changed something else that stopped it working, but then others seem to be bumping into the same issue at the same time. I'm also on Ubuntu 22.04, so wonder if it's something specific to that -- an apt upgrade deleted a symlink or something?

Anyway, sounds like my config was wrong in the first place and needed correcting.

agocke commented 1 year ago

Confirmed: the default dotnet install location is [/usr/lib/dotnet/shared/Microsoft.NETCore.App] for ubuntu:latest, but it doesn't have the /etc/dotnet/install_location file. So this deb package is broken.

Not sure who owns creating deb packages. @jkoritzinsky would you happen to know?

Daan4 commented 1 year ago

I experienced the same issue on ubuntu 22.04 going from 6.0.9 to 6.0.11. Resolved it by doing mv /lib/dotnet/install_location /etc/dotnet/install_location

agocke commented 1 year ago

@MichaelSimons do you know who the contact would be for the Ubuntu packages?

MichaelSimons commented 1 year ago

@mirespace, is the primary dev of the source-built .NET packages in the Ubuntu feed. @mirespace, can you help out with this issue? TIA

mirespace commented 1 year ago

Hi all,

Thanks a lot Michael for the mention! We detected that DOTNET_ROOT was not properly updated when upgrading from previous versions to latest 6.0.111 (we did a change in the install layout as some of you could notice), affecting only to amd64 architectures:

https://bugs.launchpad.net/ubuntu/+source/dotnet6/+bug/1997746

We were monitoring any possible issues related to this, but we didn't get any on Launchpad so we couldn't estimate if this was causing a real impact or not. @ptr727 or anyone else affected, whenever you face an issue with the package, feel free to file a issue about dotnet package in ubuntu at https://bugs.launchpad.net/ubuntu/+source/dotnet6/+filebug ... thankyou!. I'll track this to the above bug already opened. Anyway, happy to help here, of course!

The fix is prepared for next version .112 next week, sorry for the inconvenience.

The issue arises in the file /etc/profile.d/dotnet.sh, where the DOTNET_ROOT variable is set:

# Set location for AppHost lookup
[ -z "$DOTNET_ROOT" ] && export DOTNET_ROOT=/usr/lib/dotnet

There are a couple of workarounds in the meantime (I already saw some of you applying it, thanks!):

We didn't notice the missing location files! Thanks @agocke for the reporting here and @Daan4 for the workaround. I already filed a bug for that:

https://bugs.launchpad.net/ubuntu/+source/dotnet6/+bug/1999266

I'll update this issue when the fix would be released. Thanks in advance for your patience.

agocke commented 1 year ago

@mirespace Just to give a little more context where this matters:

If you use dotnet directly, you probably won't notice much difference. In these cases the runtime is always next to the dotnet launcher so there's extra to find.

The scenario that's broken is what we call the "apphost" scenario, where a native exe is bundled with the user application. In that case the apphost needs to launch the runtime, but doesn't necessarily know where to look. The behavior is defined as either

1) the default location in /usr/share 2) the location as specified in /etc/dotnet/install_location

The fixes for this would generally be to either use the default install location, or place the install_location file in the expected /etc/dotnet location. No real preference on either choice.

vitek-karas commented 1 year ago

The fixes for this would generally be to either use the default install location, or place the install_location file in the expected /etc/dotnet location. No real preference on either choice.

I would say we prefer to always write the install_location file, even if the install is into the default location.

The design docs for this area are here: https://github.com/dotnet/designs/blob/main/accepted/2020/install-locations.md https://github.com/dotnet/designs/blob/main/accepted/2021/install-location-per-architecture.md

Ideally the installer would also write the install_location_<arch> file, also pointing to the same path.

mirespace commented 1 year ago

Hi! To clarify, both fixes will be introduced in the next version:

coc0a commented 1 year ago

FWIW, was having this issue on a freshly spun up arm64 (NOT amd64) VPS instance, and before I found this issue, I tried rebooting the VPS, after which it worked fine. Investigating further, I found /etc/dotnet to be empty but DOTNET_ROOT set to the correct value of /usr/lib/dotnet, no idea if it was set before the reboot as this is my first time using .NET and didn't know about it before hand. No idea what happened after the reboot.

Clockwork-Muse commented 1 year ago

FWIW, was having this issue on a freshly spun up arm64 (NOT amd64) VPS instance, and before I found this issue, I tried rebooting the VPS, after which it worked fine. Investigating further, I found /etc/dotnet to be empty but DOTNET_ROOT set to the correct value of /usr/lib/dotnet, no idea if it was set before the reboot as this is my first time using .NET and didn't know about it before hand. No idea what happened after the reboot.

Environment variables usually require session refresh (the common case being PATH), although there's various mechanisms to manually do so.

masto commented 1 year ago

FYI: I'm using zsh instead of bash, and as a result /etc/profile is not read and DOTNET_ROOT remains unset. Obviously I can work around this in any number of ways now that I know what's happening, but I'm dropping a comment here in case there's a better answer to make it work out of the box for non-bash users.

I don't know what that answer is. TBH I only learned today that zsh doesn't read /etc/profile. It seems like a bummer. On my system, there are 10 scripts in /etc/profile.d and I'm now wondering if any of them contain important things I've been missing.

Realizing now how much hander the generic problem of "we need to modify your shell environment on installation" is than I knew, I won't feel bad if the decision here is "you're on your own if you don't use the default shell". That said, I think zsh and fish are pretty popular these days. I think zsh is even the default on macOS so lots more people are aware of it.

jkotas commented 1 year ago

The proper way is for the package installer to update /etc/dotnet/install_location with the path. This step is missing in the recent Ubuntu package installer. It is a bug in the Ubuntu package installer.

Setting DOTNET_ROOT is a quick workaround. It comes with number of issues as you have pointed out.

mirespace commented 1 year ago

Hi all! Happy New year btw.

(OFYI) Sorry for the delay in commenting here: the fixes were released this week with version 6.0.113 (maybe some of you noticed them also in the 112 version that was in proposed pocket in December). I hope this help, and please, let me know if something else needs to be changed for you.

@masto Thanks for bringing up the zsh user experience.... would you mind submitting a bug at https://bugs.launchpad.net/ubuntu/+source/dotnet6/+filebug ? I will mark it as Wishlist at first, but it will help us to be aware of it. We will work on it as soon as possible when time and resources allow.

elinor-fung commented 1 year ago

Thanks for the update, @mirespace! I'm going to close this issue since the fix has been released.