search

AppImageCrafters / appimage-builder

GNU/Linux packaging solution using the AppImage format
MIT License
316 stars 58 forks source link

fails when using sshfs filesystem #128

Closed baryluk closed 3 years ago

baryluk commented 3 years ago 
+ mkdir -p AppDir/usr/bin
+ cp -v multimonitor_ldc AppDir/usr/bin/multimonitor
'multimonitor_ldc' -> 'AppDir/usr/bin/multimonitor'
+ mkdir -p AppDir/usr/share/icons
+ cp -v /usr/share/icons/desktop-base/scalable/emblems/emblem-debian.svg AppDir/usr/share/icons/multimonitor.svg
'/usr/share/icons/desktop-base/scalable/emblems/emblem-debian.svg' -> 'AppDir/usr/share/icons/multimonitor.svg'
+ appimage-builder --skip-test
INFO:root:======
INFO:root:Script
INFO:root:======
INFO:root:=================
INFO:root:Generating AppDir
INFO:root:=================
INFO:root:
INFO:root:Bundling dependencies
INFO:root:---------------------
INFO:apt:apt-get update
Get:1 http://deb.debian.org/debian sid InRelease [157 kB]
Ign:1 http://deb.debian.org/debian sid InRelease
Get:2 http://deb.debian.org/debian sid/main amd64 Packages [8,626 kB]
Get:3 http://deb.debian.org/debian sid/contrib amd64 Packages [61.6 kB]
Get:4 http://deb.debian.org/debian sid/non-free amd64 Packages [107 kB]
Fetched 8,952 kB in 5s (1,698 kB/s)                       
Reading package lists... Done
W: GPG error: http://deb.debian.org/debian sid InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 04EE7237B7D453EC NO_PUBKEY 648ACFD622F3D138
W: The repository 'http://deb.debian.org/debian sid InRelease' is not signed.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
Reading package lists... Done
Building dependency tree... Done
The following additional packages will be installed:
  gcc-10-base libbsd0 libc6 libcrypt1 libedit2 libffi7 libgcc-s1 libllvm11 libmd0 libstdc++6
  libtinfo6 libz3-4 zlib1g
Suggested packages:
  glibc-doc libc-l10n locales
Recommended packages:
  libidn2-0 libnss-nis libnss-nisplus
The following NEW packages will be installed:
  gcc-10-base libbsd0 libc6 libcrypt1 libedit2 libffi7 libgcc-s1 libllvm11 libmd0
  libphobos2-ldc-shared94 libstdc++6 libtinfo6 libz3-4 zlib1g
0 upgraded, 14 newly installed, 0 to remove and 121 not upgraded.
Need to get 30.5 MB of archives.
After this operation, 131 MB of additional disk space will be used.
WARNING: The following packages cannot be authenticated!
  gcc-10-base libgcc-s1 libcrypt1 libc6 libmd0 libbsd0 libtinfo6 libedit2 libffi7 libstdc++6
  libz3-4 zlib1g libllvm11 libphobos2-ldc-shared94
Authentication warning overridden.
Get:1 http://deb.debian.org/debian sid/main amd64 gcc-10-base amd64 10.2.1-6 [201 kB]
Get:2 http://deb.debian.org/debian sid/main amd64 libgcc-s1 amd64 10.2.1-6 [41.4 kB]
Get:3 http://deb.debian.org/debian sid/main amd64 libcrypt1 amd64 1:4.4.18-4 [87.5 kB]
Get:4 http://deb.debian.org/debian sid/main amd64 libc6 amd64 2.31-12 [2,814 kB]
Get:5 http://deb.debian.org/debian sid/main amd64 libmd0 amd64 1.0.3-3 [28.0 kB]
Get:6 http://deb.debian.org/debian sid/main amd64 libbsd0 amd64 0.11.3-1 [108 kB]
Get:7 http://deb.debian.org/debian sid/main amd64 libtinfo6 amd64 6.2+20201114-2 [341 kB]
Get:8 http://deb.debian.org/debian sid/main amd64 libedit2 amd64 3.1-20191231-2+b1 [96.7 kB]
Get:9 http://deb.debian.org/debian sid/main amd64 libffi7 amd64 3.3-6 [23.1 kB]
Get:10 http://deb.debian.org/debian sid/main amd64 libstdc++6 amd64 10.2.1-6 [493 kB]
Get:11 http://deb.debian.org/debian sid/main amd64 libz3-4 amd64 4.8.10-1 [6,949 kB]
Get:12 http://deb.debian.org/debian sid/main amd64 zlib1g amd64 1:1.2.11.dfsg-2 [90.4 kB]
Get:13 http://deb.debian.org/debian sid/main amd64 libllvm11 amd64 1:11.0.1-2 [17.9 MB]
Get:14 http://deb.debian.org/debian sid/main amd64 libphobos2-ldc-shared94 amd64 1:1.24.0-2 [1,280 kB]
Fetched 30.5 MB in 12s (2,499 kB/s)                                                          
Download complete and in download only mode
Reading package lists... Done
Building dependency tree... Done
The following additional packages will be installed:
  gcc-10-base libcrypt1 libgcc-s1
Suggested packages:
  glibc-doc libc-l10n locales
Recommended packages:
  libidn2-0 libnss-nis libnss-nisplus
The following NEW packages will be installed:
  gcc-10-base libc6 libcrypt1 libgcc-s1 libstdc++6 zlib1g
0 upgraded, 6 newly installed, 0 to remove and 121 not upgraded.
Need to get 0 B/3,727 kB of archives.
After this operation, 16.3 MB of additional disk space will be used.
WARNING: The following packages cannot be authenticated!
  gcc-10-base libgcc-s1 libcrypt1 libc6 libstdc++6 zlib1g
Authentication warning overridden.
Download complete and in download only mode
INFO:AptPackageDeploy:Deploying libllvm11_1%3a11.0.1-2_amd64.deb to /home/user/vps1/home/baryluk/Projekty/multimonitor/AppDir
INFO:AptPackageDeploy:Deploying libstdc++6_10.2.1-6_amd64.deb to /home/user/vps1/home/baryluk/Projekty/multimonitor/AppDir/opt/libc
tar: ./usr/share/doc/libstdc++6: Cannot utime: No such file or directory
tar: Exiting with failure status due to previous errors
dpkg-deb: error: tar subprocess returned error exit status 2
ERROR:root:"/usr/bin/dpkg-deb -x /home/user/vps1/home/baryluk/Projekty/multimonitor/appimage-builder-cache/apt/archives/libstdc++6_10.2.1-6_amd64.deb /home/user/vps1/home/baryluk/Projekty/multimonitor/AppDir/opt/libc" execution failed
Traceback (most recent call last):
  File "/usr/local/bin/appimage-builder", line 8, in <module>
    sys.exit(__main__())
  File "/usr/local/lib/python3.9/dist-packages/appimagebuilder/__main__.py", line 94, in __main__
    creator.build()
  File "/usr/local/lib/python3.9/dist-packages/appimagebuilder/builder/builder.py", line 62, in build
    self._bundle_dependencies()
  File "/usr/local/lib/python3.9/dist-packages/appimagebuilder/builder/builder.py", line 83, in _bundle_dependencies
    deployed_packages = apt_deploy.deploy(
  File "/usr/local/lib/python3.9/dist-packages/appimagebuilder/builder/deploy/apt/deploy.py", line 40, in deploy
    extracted_packages = self._extract_packages(appdir_root, deploy_list)
  File "/usr/local/lib/python3.9/dist-packages/appimagebuilder/builder/deploy/apt/deploy.py", line 89, in _extract_packages
    self.apt_venv.extract_package(package, final_target)
  File "/usr/local/lib/python3.9/dist-packages/appimagebuilder/builder/deploy/apt/venv.py", line 224, in extract_package
    shell.assert_successful_result(output)
  File "/usr/local/lib/python3.9/dist-packages/appimagebuilder/common/shell.py", line 30, in assert_successful_result
    raise RuntimeError(
RuntimeError: "/usr/bin/dpkg-deb -x /home/user/vps1/home/baryluk/Projekty/multimonitor/appimage-builder-cache/apt/archives/libstdc++6_10.2.1-6_amd64.deb /home/user/vps1/home/baryluk/Projekty/multimonitor/AppDir/opt/libc" execution failed with code 2
$ mount | grep vps1
baryluk@server:/ on /home/user/vps1 type fuse.sshfs (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
$

Moving, the entire directory ~/tmp (on tmpfs), makes it not fail and successfully build the AppImage

azubieta commented 3 years ago

According to the logs dpkg-deb -x is failing to run

INFO:AptPackageDeploy:Deploying libstdc++6_10.2.1-6_amd64.deb to /home/user/vps1/home/baryluk/Projekty/multimonitor/AppDir/opt/libc tar: ./usr/share/doc/libstdc++6: Cannot utime: No such file or directory tar: Exiting with failure status due to previous errors dpkg-deb: error: tar subprocess returned error exit status 2

could you try to run it manually like this dpkg-deb -x < path to libstdc++6_10.2.1-6_amd64.deb> <path to AppDir/opt/libc> I would like to discard that it's a dpkg-deb issue

baryluk commented 3 years ago

@azubieta yes, dpkg-deb -x fails with the same error.

strace shows this near the end:

...
[pid 2387865] mkdirat(AT_FDCWD, "./usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu", 0700) = -1 EEXIST (File exists)
[pid 2387865] newfstatat(AT_FDCWD, "./usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu", {st_mode=S_IFDIR|0755, st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0
[pid 2387865] openat(AT_FDCWD, "./usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28-gdb.py", O_WRONLY|O_CREAT|O_EXCL|O_NOCTTY|O_NONBLOCK|O_CLOEXEC, 0644) = -1 EEXIST (File exists)
[pid 2387865] unlinkat(AT_FDCWD, "./usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28-gdb.py", 0) = 0
[pid 2387865] openat(AT_FDCWD, "./usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28-gdb.py", O_WRONLY|O_CREAT|O_EXCL|O_NOCTTY|O_NONBLOCK|O_CLOEXEC, 0644) = 3
[pid 2387865] write(3, "# -*- python -*-\n# Copyright (C)"..., 2387) = 2387
[pid 2387865] utimensat(3, NULL, [UTIME_OMIT, {tv_sec=1610278539, tv_nsec=0} /* 2021-01-10T11:35:39+0000 */], 0) = 0
[pid 2387865] close(3)                  = 0
[pid 2387865] utimensat(AT_FDCWD, "./usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu", [UTIME_OMIT, {tv_sec=1610278539, tv_nsec=0} /* 2021-01-10T11:35:39+0000 */], 0) = 0
[pid 2387865] utimensat(AT_FDCWD, "./usr/share/gdb/auto-load/usr/lib", [UTIME_OMIT, {tv_sec=1610278539, tv_nsec=0} /* 2021-01-10T11:35:39+0000 */], 0) = 0
[pid 2387865] utimensat(AT_FDCWD, "./usr/share/gdb/auto-load/usr", [UTIME_OMIT, {tv_sec=1610278539, tv_nsec=0} /* 2021-01-10T11:35:39+0000 */], 0) = 0
[pid 2387865] utimensat(AT_FDCWD, "./usr/share/gdb/auto-load", [UTIME_OMIT, {tv_sec=1610278539, tv_nsec=0} /* 2021-01-10T11:35:39+0000 */], 0) = 0
[pid 2387865] utimensat(AT_FDCWD, "./usr/share/gdb", [UTIME_OMIT, {tv_sec=1610278539, tv_nsec=0} /* 2021-01-10T11:35:39+0000 */], 0) = 0
[pid 2387865] utimensat(AT_FDCWD, "./usr/share", [UTIME_OMIT, {tv_sec=1610278539, tv_nsec=0} /* 2021-01-10T11:35:39+0000 */], 0) = 0
[pid 2387865] symlinkat("libstdc++.so.6.0.28", AT_FDCWD, "./usr/lib/x86_64-linux-gnu/libstdc++.so.6") = -1 EEXIST (File exists)
[pid 2387865] unlinkat(AT_FDCWD, "./usr/lib/x86_64-linux-gnu/libstdc++.so.6", 0) = 0
[pid 2387865] symlinkat("libstdc++.so.6.0.28", AT_FDCWD, "./usr/lib/x86_64-linux-gnu/libstdc++.so.6") = 0
[pid 2387865] utimensat(AT_FDCWD, "./usr/lib/x86_64-linux-gnu/libstdc++.so.6", [UTIME_OMIT, {tv_sec=1610278539, tv_nsec=0} /* 2021-01-10T11:35:39+0000 */], AT_SYMLINK_NOFOLLOW) = 0
[pid 2387865] openat(AT_FDCWD, "./usr/lib/x86_64-linux-gnu/libstdc++.so.6", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 3
[pid 2387865] newfstatat(3, "", {st_mode=S_IFLNK|0777, st_size=19, ...}, AT_EMPTY_PATH) = 0
[pid 2387865] close(3)                  = 0
[pid 2387865] symlinkat("gcc-10-base", AT_FDCWD, "./usr/share/doc/libstdc++6") = -1 EEXIST (File exists)
[pid 2387865] unlinkat(AT_FDCWD, "./usr/share/doc/libstdc++6", 0) = 0
[pid 2387865] symlinkat("gcc-10-base", AT_FDCWD, "./usr/share/doc/libstdc++6") = 0
[pid 2387865] utimensat(AT_FDCWD, "./usr/share/doc/libstdc++6", [UTIME_OMIT, {tv_sec=1610278539, tv_nsec=0} /* 2021-01-10T11:35:39+0000 */], AT_SYMLINK_NOFOLLOW) = -1 ENOENT (No such file or directory)
...
azubieta commented 3 years ago

It seems that we have to fill a bug on the dpkg bug tracker then. If you found any workaround please leave it here for future users.

baryluk commented 3 years ago

It looks like the bug is actually in the tar. tar -x -p -f data.tar.xz (after extracting manually data.tar.xz using ar x libc6*.deb), shows same issue.

tar correctly passes AT_SYMLINK_NOFOLLOW. And the file it is trying to update utime exists:

lrwxrwxrwx 1 user user 32 May  7 17:00 ./lib64/ld-linux-x86-64.so.2 -> /lib/x86_64-linux-gnu/ld-2.31.so

Maybe it is a bug in the FUSE. Anyway, closing, as this is not appimage-builder issue.

baryluk commented 3 years ago

It is sshfs bug. https://github.com/libfuse/sshfs/issues/250

Sorry for the noise here. I just never experienced this issue despite me using sshfs daily for last few years, until I tried appimage-builder.