AppImage could not find QtWebEngineProcess

vaibhavpandeyvpz commented 6 years ago

I tried to run my AppImage build using following commands:

mkdir -p AppDir/usr/lib/
cp -r /usr/lib/x86_64-linux-gnu/nss AppDir/usr/lib/
./linuxdeploy-x86_64.AppImage --appdir AppDir
./linuxdeploy-plugin-qt-x86_64.AppImage --appdir AppDir
./linuxdeploy-x86_64.AppImage --appdir AppDir --output appimage

I get below error:

Could not find QtWebEngineProcess
[12038:12038:0826/074846.252467:FATAL:zygote_host_impl_linux.cc(182)] Check failed: ReceiveFixedMessage(fds[0], kZygoteBootMessage, sizeof(kZygoteBootMessage), &boot_pid). 
#0 0x7f6bb0c224e5 <unknown>
#1 0x7f6bb0c38d9a <unknown>
#2 0x7f6bb029057b <unknown>
#3 0x7f6bb028f427 <unknown>
#4 0x7f6bb028fa6c <unknown>
#5 0x7f6bafed5a4d <unknown>
#6 0x7f6bafedaea4 <unknown>
#7 0x7f6baf9fd20b <unknown>
#8 0x7f6baf9fe205 <unknown>
#9 0x7f6baf9931a1 QtWebEngineCore::BrowserContextAdapter::defaultContext()
#10 0x7f6bb749fde0 QWebEngineProfile::defaultProfile()
#11 0x000000427674 <unknown>
#12 0x00000041038f <unknown>
#13 0x7f6bb560e830 __libc_start_main
#14 0x0000004104c9 <unknown>

Received signal 6
#0 0x7f6bb0c224e5 <unknown>
#1 0x7f6baf9811c2 <unknown>
#2 0x7f6bb0c2287d <unknown>
#3 0x7f6bb56234a0 <unknown>
#4 0x7f6bb5623418 gsignal
#5 0x7f6bb562501a abort
#6 0x7f6bb0c22225 <unknown>
#7 0x7f6bb0c38e95 <unknown>
#8 0x7f6bb029057b <unknown>
#9 0x7f6bb028f427 <unknown>
#10 0x7f6bb028fa6c <unknown>
#11 0x7f6bafed5a4d <unknown>
#12 0x7f6bafedaea4 <unknown>
#13 0x7f6baf9fd20b <unknown>
#14 0x7f6baf9fe205 <unknown>
#15 0x7f6baf9931a1 QtWebEngineCore::BrowserContextAdapter::defaultContext()
#16 0x7f6bb749fde0 QWebEngineProfile::defaultProfile()
#17 0x000000427674 <unknown>
#18 0x00000041038f <unknown>
#19 0x7f6bb560e830 __libc_start_main
#20 0x0000004104c9 <unknown>
  r8: ffff8000ed4f4ce0  r9: ffff8000ed4f4cd0 r10: 0000000000000008 r11: 0000000000000206
 r12: 00007fff15196730 r13: 00000000000000ac r14: 00007fff15196740 r15: 00007fff15196bd0
  di: 0000000000002f06  si: 0000000000002f06  bp: 00007fff15196710  bx: 00007fff15196cb0
  dx: 0000000000000006  ax: 0000000000000000  cx: 00007f6bb5623418  sp: 00007fff15196388
  ip: 00007f6bb5623418 efl: 0000000000000206 cgf: 0000000000000033 erf: 0000000000000000
 trp: 0000000000000000 msk: 0000000000000000 cr2: 0000000000000000
[end of stack trace]
Calling _exit(1). Core file will not be generated.

However, if I run the same AppImage as sudo, it works fine.

TheAssassin commented 6 years ago

Can you provide your AppImage, or if that's not possible (actually, it'd be even better), an MWE?

vaibhavpandeyvpz commented 6 years ago

Here you go:

Source: https://github.com/vaibhavpandeyvpz/mwe-qtwebengine/
Build script: https://github.com/vaibhavpandeyvpz/mwe-qtwebengine/blob/master/build-linux.sh
AppImage: https://transfer.sh/KZYfd/MWE-x86_64.AppImage
Environment: Ubuntu 16.04 (64-Bit) + ppa:beineri/opt-qt-5.10.1-xenial

AppImage works on the machine it has been built upon. On a clean/stock Ubuntu 16.04 machine, it only works in sudo mode (as root). Otherwise, it dumps above error into terminal.

TheAssassin commented 6 years ago

Thanks @vaibhavpandeyvpz. On my Ubuntu 18.04 with KDE, the AppImage works perfectly fine. To make sure that this doesn't just use my system Qt's QtWebEngineProcess, I re-ran the AppImage with strace (sudo strace to be precise, sudo is required due to the FUSE handling in the runtime, if you run --appimage-extract first, you can also just run strace squashfs-root/AppRun).

> sudo strace ./MWE-x86_64.AppImage 2>&1 | ag QtWebEngineProcess
access("/tmp/.mount_MWE-x8pPXVSF/usr/libexec/QtWebEngineProcess", F_OK) = 0
lstat("/tmp/.mount_MWE-x8pPXVSF/usr/libexec/QtWebEngineProcess", {st_mode=S_IFREG|0755, st_size=6304, ...}) = 0

Looks like the bundled process is used, as expected.

Can you please run my strace command posted above (s/ag/grep/) and post the result?

vaibhavpandeyvpz commented 6 years ago

Same as yours.

access("/tmp/.mount_MWE-x8Mjgh15/usr/libexec/QtWebEngineProcess", F_OK) = 0
lstat("/tmp/.mount_MWE-x8Mjgh15/usr/libexec/QtWebEngineProcess", {st_mode=S_IFREG|0755, st_size=6304, ...}) = 0

But be informed this was when it was run as sudo and the AppImage does run fine when run under sudo. The problem is when it is run as normal user.

vaibhavpandeyvpz commented 6 years ago

I was trying to not believe https://discourse.appimage.org/t/running-appimage-app-on-ubuntu-16-04/200/22. Can you please check on Ubuntu 16.04 if you have a machine or VM (like me)? The above results I posted were from 16.04 only. Now I am starting to think if AppImage or better say fuse is supported in Ubuntu 16.04 for non-root users or not.

vaibhavpandeyvpz commented 6 years ago

I have downloaded official Qt 5.11.1 from Qt website and built the app and AppImage with it. It runs fine on the other stock/new Ubuntu 16.04 VM as normal user. To my findings now, I assume 2 points below:

The packages in Stephan's PPA are not PIE i.e., position independent executable (as it is with official Qt distribution) and looks for QtWebEngineProcess in system location.
~There might have been some bug in QtWebEngine in 5.10.1 which is now fixed 5.11.1.~

I do not feel it is an issue with linuxdeploy or linuxdeployqt now. Thus closing this.

TheAssassin commented 6 years ago

@vaibhavpandeyvpz I wonder why this bug hit you, though. I will reopen this issue until I got to testing the MWE on more distributions.

Re. the PPAs, I experienced them as high-quality distributions of Qt, they never let us down, and usually work fine. If there's a bug in one of the packages, we should talk to the author, who I'd say is quite responsive, and probably interested in fixing the issue.

TheAssassin commented 6 years ago

The original MWE also works fine in a Debian stretch (current stable) XFCE environment.

@azubieta do you have any ideas what could've caused the original bug? I hate to close bugs without knowing their cause...

vaibhavpandeyvpz commented 6 years ago

I now even tried with official Qt 5.10.1 gcc_64 toolchain from the installer on Qt website and the compiled MWE works under normal (without root) user on Ubuntu 16.04, 17.04 & 18.04 (all versions I checked). There is some issue with QtWebEngine from Stephen's PPA looking for QtWebEngineProcess in wrong location when run as a normal user.

vaibhavpandeyvpz commented 6 years ago

I tried replacing the QtWebEngineProcess in MWE (built with Qt libs from PPA) with the QtWebEngineProcess from official Qt distribution, it then works under normal user. So I doubt its only an issue with how QtWebEngineProcess is being built.

EDIT: I ran diff against ldd of both executables and seen no differences.

TheAssassin commented 6 years ago

There is some issue with QtWebEngine from Stephen's PPA looking for QtWebEngineProcess in wrong location when run as a normal user.

This is rather unlike. Why should it behave differently with and without root? And why does it work fine on all the machines I tried so far (my own workstation and a couple of VMs)?

To be honest, it looks more and more like the opposite of "works on my machine": "does not work on your machine". I doubt there's an issue within Stephan's PPAs. There is no reason I could imagine to use different paths when looking for that executable, unless the user's environment is non-default (e.g., some environment variable).

But let's no longer make assumptions, but look for some evidence. Please try two things:

Call the AppImage with --appimage-extract (required to run strace on the payload without root permissions)
run the contents:
- call squashfs-root/AppRun normally, and check whether it works
- call strace squashfs-root/AppRun 2>&1 | grep QtWebEngineProcess (now it's extracted, no more sudo required, so please run without)

The latter call will allow you to see what path it loads QtWebEngineProcess from.

Also, please try to run on other machines, e.g., some virtual machines, to make sure some configuration on your computer is not causing the bug.

vaibhavpandeyvpz commented 6 years ago

Running squashfs-root/AppRun returned:

Could not find QtWebEngineProcess
[6666:6666:0828/191824.763345:FATAL:zygote_host_impl_linux.cc(182)] Check failed: ReceiveFixedMessage(fds[0], kZygoteBootMessage, sizeof(kZygoteBootMessage), &boot_pid). 
#0 0x7f3e52dbf4e5 <unknown>
#1 0x7f3e52dd5d9a <unknown>
#2 0x7f3e5242d57b <unknown>
#3 0x7f3e5242c427 <unknown>
#4 0x7f3e5242ca6c <unknown>
#5 0x7f3e52072a4d <unknown>
#6 0x7f3e52077ea4 <unknown>
#7 0x7f3e51b9a20b <unknown>
#8 0x7f3e51b9b205 <unknown>
#9 0x7f3e51b8a5b9 <unknown>
#10 0x7f3e51b8a78e QtWebEngineCore::WebContentsAdapter::WebContentsAdapter()
#11 0x7f3e58f0b7e1 <unknown>
#12 0x7f3e58f0c364 QWebEnginePage::QWebEnginePage()
#13 0x7f3e58f1af8b QWebEngineView::page()
#14 0x7f3e58f1b029 QWebEngineView::load()
#15 0x0000004026ef <unknown>
#16 0x0000004024b7 <unknown>
#17 0x7f3e577ab830 __libc_start_main
#18 0x000000402589 <unknown>

Received signal 6
#0 0x7f3e52dbf4e5 <unknown>
#1 0x7f3e51b1e1c2 <unknown>
#2 0x7f3e52dbf87d <unknown>
#3 0x7f3e577c04a0 <unknown>
#4 0x7f3e577c0418 gsignal
#5 0x7f3e577c201a abort
#6 0x7f3e52dbf225 <unknown>
#7 0x7f3e52dd5e95 <unknown>
#8 0x7f3e5242d57b <unknown>
#9 0x7f3e5242c427 <unknown>
#10 0x7f3e5242ca6c <unknown>
#11 0x7f3e52072a4d <unknown>
#12 0x7f3e52077ea4 <unknown>
#13 0x7f3e51b9a20b <unknown>
#14 0x7f3e51b9b205 <unknown>
#15 0x7f3e51b8a5b9 <unknown>
#16 0x7f3e51b8a78e QtWebEngineCore::WebContentsAdapter::WebContentsAdapter()
#17 0x7f3e58f0b7e1 <unknown>
#18 0x7f3e58f0c364 QWebEnginePage::QWebEnginePage()
#19 0x7f3e58f1af8b QWebEngineView::page()
#20 0x7f3e58f1b029 QWebEngineView::load()
#21 0x0000004026ef <unknown>
#22 0x0000004024b7 <unknown>
#23 0x7f3e577ab830 __libc_start_main
#24 0x000000402589 <unknown>
  r8: ffff8000d1d8d6a0  r9: ffff8000d1d8d690 r10: 0000000000000008 r11: 0000000000000206
 r12: 00007fff30217950 r13: 00000000000000aa r14: 00007fff30217960 r15: 00007fff30217df0
  di: 0000000000001a0a  si: 0000000000001a0a  bp: 00007fff30217930  bx: 00007fff30217ed0
  dx: 0000000000000006  ax: 0000000000000000  cx: 00007f3e577c0418  sp: 00007fff302175a8
  ip: 00007f3e577c0418 efl: 0000000000000206 cgf: 0000000000000033 erf: 0000000000000000
 trp: 0000000000000000 msk: 0000000000000000 cr2: 0000000000000000
[end of stack trace]
Calling _exit(1). Core file will not be generated.

This is the strace squashfs-root/AppRun 2>&1 | grep QtWebEngineProcess output:

access("/home/vaibhav/Desktop/squashfs-root/usr/libexec/QtWebEngineProcess", F_OK) = 0
lstat("/home/vaibhav/Desktop/squashfs-root/usr/libexec/QtWebEngineProcess", {st_mode=S_IFREG|0755, st_size=6304, ...}) = 0
recvmsg(135, Could not find QtWebEngineProcess

The code in question: https://github.com/qt/qtwebengine/blob/5.11/src/core/web_engine_library_info.cpp#L144

vaibhavpandeyvpz commented 6 years ago

According to line https://github.com/qt/qtwebengine/blob/5.11/src/core/web_engine_library_info.cpp#L164, it seems to be looking in QLibraryInfo::LibraryExecutablesPath.

Upon adding below simple line to main entry-point of MWE source, after QApplication creation:

std::cout << "libexec path: " << QLibraryInfo::location(QLibraryInfo::LibraryExecutablesPath).toStdString();

If built with Stephan's libraries, it outputs:

libexec path: /tmp/.mount_MWE-x8CSgEv8/usr/libexec

If built with official Qt libraries, it outputs:

libexec path: /tmp/.mount_MWE-x8zEQuRj/usr/libexec

Seems fine though it still can't find QtWebEngineProcess. Weird.

vaibhavpandeyvpz commented 6 years ago

@TheAssassin Don't know about Debian stretch, but since KDE is based on Qt itself, it might be having Qt lib(exec)s already in-place (or in PATH) to be found. Can you please try the AppImage I linked above on a clean distro like Ubuntu 16.04 without any Qt install?

TheAssassin commented 6 years ago

I wonder whether the file is not executable by non-root users in that MWE AppImage...

valdmann commented 4 years ago

WebEngine developer here. The message "Could not find QtWebEngineProcess" is actually coming from the QtWebEngineProcess itself... This however is not a sign of existential crisis, but merely an issue with sandboxing: the QtWebEngineProcess runs inside a separate user namespace and it can't access the FUSE-mounted squashfs image. Passing the --disable-namespace-sandbox flag to the app works around the issue. I've not much clue about the internals of FUSE, but adding the allow_other option to the FUSE mount might perhaps also help.

TheAssassin commented 4 years ago

@valdmann thanks a lot for the detailed insights. I'll have to check whether adding that FUSE option implies any security problems. If not, I'll propose adding it in our AppImage runtime.

CC @probonopd

TheAssassin commented 4 years ago

Seems like on Ubuntu and a lot of other systems, this mount option is unavailable without changing the system configuration (for security reasons, but that doesn't really matter).

I guess we'll just have to document this flag

probonopd commented 4 years ago

What are the security implications of adding the allow_other option to the FUSE mount ?

The way I read it: User B can access the contents of an AppImage that has been mounted by user A. May not be desirable?

@valdmann, thanks for chiming in. Could something be set in qt.conf that would have the same effect as passing the --disable-namespace-sandbox option?

TheAssassin commented 4 years ago

What are the security implications of adding the allow_other option to the FUSE mount ?

Does that matter? As long as you can't use it, it's the same as if there was no such option at all.

@valdmann, thanks for chiming in. Could something be set in qt.conf that would have the same effect as passing the --disable-namespace-sandbox option?

What's the security implications? Disabling a sandbox sounds like a horrible idea...

probonopd commented 4 years ago

As long as you can't use it, it's the same as if there was no such option at all.

What do you mean by this, especially by "as long as you can't use it"?

TheAssassin commented 4 years ago

Seems like on Ubuntu and a lot of other systems, this mount option is unavailable without changing the system configuration (for security reasons, but that doesn't really matter).

https://manpages.ubuntu.com/manpages/bionic/en/man8/mount.fuse.8.html, section CONFIGURATION:

user_allow_other Allow non-root users to specify the allow_other or allow_root mount options (see below).

probonopd commented 4 years ago

What's the right thing to do then?

TheAssassin commented 4 years ago

I don't know. Disabling the sandbox is a bad idea. The sandbox is there for a reason. The problem is that it's based on cgroups. I wouldn't mind adding that FUSE option, having read about it a bit, but there is a reason it's not available by default I guess.

Perhaps you can test a few distros whether allow_other is available there? As far as I understand the docs I read so far, that's not the case. But I might be wrong.

Edit:

> sshfs ... -o allow_other
fusermount: option allow_other only allowed if 'user_allow_other' is set in /etc/fuse.conf

As said before, in a standard installation, this option cannot be used.

valdmann commented 4 years ago

@valdmann, thanks for chiming in. Could something be set in qt.conf that would have the same effect as passing the --disable-namespace-sandbox option?

No, the only alternative way is to use the QTWEBENGINE_CHROMIUM_FLAGS environment variable.

xiaoyifang commented 2 years ago

encounter same error .

https://github.com/xiaoyifang/goldendict/releases/download/v22.2.alpha-a32ebfa2/6.2.3-GoldenDict-a32ebfa2-x86_64.AppImage

linuxdeploy / linuxdeploy-plugin-qt

AppImage could not find QtWebEngineProcess #13