How to run ykman bundled with Yubikey Manager AppImage?

sunknudsen commented 3 years ago

YubiKey Manager (ykman) version: YubiKey Manager 1.2.3
How was it installed?: YubiKey Manager AppImage downloaded from https://www.yubico.com/support/download/yubikey-manager/
Operating system and version: Tails 4.20
YubiKey model and version: YubiKey 5 NFC
Bug description summary: Cannot run ykman bundled with YubiKey Manager AppImage.

Steps to reproduce

On macOS, I simply run /Applications/YubiKey\ Manager.app/Contents/MacOS/ykman.

On Tails, I see ykman executable is bundled, but I am not able to run executable (perhaps AppImage content can be extracted using --appimage-extract and patched).

That said, process is involved.

Expected result

I was expecting to be able to run ykman somehow using AppImage arguments (perhaps a naive expectation).

sunknudsen commented 3 years ago

Btw, stumbled upon https://support.yubico.com/hc/en-us/articles/360016649039-Installing-Yubico-Software-on-Linux, but only Ubuntu is supported.

Wish Debian was added to the list of supported operating systems which would cover Kali, Parrot OS and Tails.

fdennis commented 3 years ago

Good catch! We want to bundle ykman cli with ykman gui. It works well on Windows and macOS but as you managed to find out, it does not appear to work as expected with AppImage. This is something we will try to improve for the next release of ykman gui. Thank you for reporting this!

As a workaround you can install ykman cli through other alternatives (e.g. pip) as documented here

sunknudsen commented 3 years ago

Hey @fdennis,

This is something we will try to improve for the next release of ykman gui. Thank you for reporting this!

Amazing! I see @emlun is working on a PR. Thanks guys!

As a workaround you can install ykman cli through other alternatives (e.g. pip) as documented here

I tried alternatives and wasn’t able to get them working on Tails without “hacks”.

I am planning on publishing a YubiKey PGP episode shortly on YouTube. If timing works, I would love to remove patch from guide.

Would you happen to have a release date target?

emlun commented 3 years ago

Sorry, the next release date is not yet planned, so probably not in time for your guide.

But is there some reason you can't install ykman from the Debian repositories? apt-get install yubikey-manager is working just fine for me in Tails. Installing via pip works too, although it's a bit more involved:

$ sudo apt-get install python3-pip python3-setuptools swig gcc libpcsclite-dev python3-dev
$ torsocks pip3 install yubikey-manager
$ ~/.local/bin/ykman info

sunknudsen commented 3 years ago

Sorry, the next release date is not yet planned, so probably not in time for your guide.

👍

apt-get install yubikey-manager is working just fine for me in Tails.

I tried using APT, but version on APT repo is pretty old (version 2.1.0).

I also really like how AppImage version is self-contained therefore Tails-friendly (it can be stored in “Persistent” folder vs installed at boot using “Additional Software” feature).

Installing via pip works too

This option installs recent version, but requires patching to be made persistent (I also ran into errors when running commands, see log bellow).

Thanks for helping out btw! Can’t wait for next AppImage release of Yubikey Manager. 🤓

$ sudo apt update

We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:

    #1) Respect the privacy of others.
    #2) Think before you type.
    #3) With great power comes great responsibility.

[sudo] password for amnesia:       
Hit:1 tor+https://cdn-fastly.deb.debian.org/debian buster InRelease            
Hit:2 tor+https://cdn-fastly.deb.debian.org/debian-security buster/updates InRelease
Hit:3 tor+http://umjqavufhoix3smyq6az2sx4istmuvsgmz4bq5u5x56rnayejoo6l2qd.onion 4.20 InRelease
Hit:4 tor+https://cdn-fastly.deb.debian.org/debian bullseye InRelease
Hit:5 tor+http://apow7mjfryruh65chtdydfmqfpj5btws7nbocgtaovhvezgccyjazpqd.onion/torproject.org buster InRelease
Hit:6 tor+https://cdn-fastly.deb.debian.org/debian buster-backports InRelease
Hit:7 tor+https://cdn-fastly.deb.debian.org/debian sid InRelease
Reading package lists... Done
Building dependency tree       
Reading state information... Done
20 packages can be upgraded. Run 'apt list --upgradable' to see them.

$ sudo apt install python3-pip python3-setuptools swig gcc libpcsclite-dev python3-dev
[sudo] password for amnesia:       
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following package was automatically installed and is no longer required:
  libpcre2-posix0
Use 'sudo apt autoremove' to remove it.
The following additional packages will be installed:
  binutils binutils-common binutils-x86-64-linux-gnu dh-python gcc-8 libasan5
  libbinutils libc-dev-bin libc6-dev libcc1-0 libexpat1-dev libgcc-8-dev
  libitm1 liblsan0 libmpx2 libpython3-dev libpython3.7-dev libtsan0 libubsan1
  linux-libc-dev python-pip-whl python3.7-dev swig3.0
Suggested packages:
  binutils-doc dpkg-dev gcc-multilib make manpages-dev autoconf automake
  libtool flex bison gdb gcc-doc gcc-8-multilib gcc-8-doc gcc-8-locales
  libgcc1-dbg libgomp1-dbg libitm1-dbg libatomic1-dbg libasan5-dbg
  liblsan0-dbg libtsan0-dbg libubsan1-dbg libmpx2-dbg libquadmath0-dbg
  glibc-doc python-setuptools-doc swig-doc swig-examples swig3.0-examples
  swig3.0-doc
Recommended packages:
  manpages-dev build-essential python3-wheel
The following NEW packages will be installed:
  binutils binutils-common binutils-x86-64-linux-gnu dh-python gcc gcc-8
  libasan5 libbinutils libc-dev-bin libc6-dev libcc1-0 libexpat1-dev
  libgcc-8-dev libitm1 liblsan0 libmpx2 libpcsclite-dev libpython3-dev
  libpython3.7-dev libtsan0 libubsan1 linux-libc-dev python-pip-whl
  python3-dev python3-pip python3-setuptools python3.7-dev swig swig3.0
0 upgraded, 29 newly installed, 0 to remove and 20 not upgraded.
Need to get 0 B/74.6 MB of archives.
After this operation, 196 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
[INFO] Saving package changes
Selecting previously unselected package binutils-common:amd64.
(Reading database ... 130878 files and directories currently installed.)
Preparing to unpack .../00-binutils-common_2.31.1-16_amd64.deb ...
Unpacking binutils-common:amd64 (2.31.1-16) ...
Selecting previously unselected package libbinutils:amd64.
Preparing to unpack .../01-libbinutils_2.31.1-16_amd64.deb ...
Unpacking libbinutils:amd64 (2.31.1-16) ...
Selecting previously unselected package binutils-x86-64-linux-gnu.
Preparing to unpack .../02-binutils-x86-64-linux-gnu_2.31.1-16_amd64.deb ...
Unpacking binutils-x86-64-linux-gnu (2.31.1-16) ...
Selecting previously unselected package binutils.
Preparing to unpack .../03-binutils_2.31.1-16_amd64.deb ...
Unpacking binutils (2.31.1-16) ...
Selecting previously unselected package dh-python.
Preparing to unpack .../04-dh-python_3.20190308_all.deb ...
Unpacking dh-python (3.20190308) ...
Selecting previously unselected package libcc1-0:amd64.
Preparing to unpack .../05-libcc1-0_8.3.0-6_amd64.deb ...
Unpacking libcc1-0:amd64 (8.3.0-6) ...
Selecting previously unselected package libitm1:amd64.
Preparing to unpack .../06-libitm1_8.3.0-6_amd64.deb ...
Unpacking libitm1:amd64 (8.3.0-6) ...
Selecting previously unselected package libasan5:amd64.
Preparing to unpack .../07-libasan5_8.3.0-6_amd64.deb ...
Unpacking libasan5:amd64 (8.3.0-6) ...
Selecting previously unselected package liblsan0:amd64.
Preparing to unpack .../08-liblsan0_8.3.0-6_amd64.deb ...
Unpacking liblsan0:amd64 (8.3.0-6) ...
Selecting previously unselected package libtsan0:amd64.
Preparing to unpack .../09-libtsan0_8.3.0-6_amd64.deb ...
Unpacking libtsan0:amd64 (8.3.0-6) ...
Selecting previously unselected package libubsan1:amd64.
Preparing to unpack .../10-libubsan1_8.3.0-6_amd64.deb ...
Unpacking libubsan1:amd64 (8.3.0-6) ...
Selecting previously unselected package libmpx2:amd64.
Preparing to unpack .../11-libmpx2_8.3.0-6_amd64.deb ...
Unpacking libmpx2:amd64 (8.3.0-6) ...
Selecting previously unselected package libgcc-8-dev:amd64.
Preparing to unpack .../12-libgcc-8-dev_8.3.0-6_amd64.deb ...
Unpacking libgcc-8-dev:amd64 (8.3.0-6) ...
Selecting previously unselected package gcc-8.
Preparing to unpack .../13-gcc-8_8.3.0-6_amd64.deb ...
Unpacking gcc-8 (8.3.0-6) ...
Selecting previously unselected package gcc.
Preparing to unpack .../14-gcc_4%3a8.3.0-1_amd64.deb ...
Unpacking gcc (4:8.3.0-1) ...
Selecting previously unselected package libc-dev-bin.
Preparing to unpack .../15-libc-dev-bin_2.28-10_amd64.deb ...
Unpacking libc-dev-bin (2.28-10) ...
Selecting previously unselected package linux-libc-dev:amd64.
Preparing to unpack .../16-linux-libc-dev_4.19.194-3_amd64.deb ...
Unpacking linux-libc-dev:amd64 (4.19.194-3) ...
Selecting previously unselected package libc6-dev:amd64.
Preparing to unpack .../17-libc6-dev_2.28-10_amd64.deb ...
Unpacking libc6-dev:amd64 (2.28-10) ...
Selecting previously unselected package libexpat1-dev:amd64.
Preparing to unpack .../18-libexpat1-dev_2.2.6-2+deb10u1_amd64.deb ...
Unpacking libexpat1-dev:amd64 (2.2.6-2+deb10u1) ...
Selecting previously unselected package libpcsclite-dev.
Preparing to unpack .../19-libpcsclite-dev_1.8.24-1_amd64.deb ...
Unpacking libpcsclite-dev (1.8.24-1) ...
Selecting previously unselected package libpython3.7-dev:amd64.
Preparing to unpack .../20-libpython3.7-dev_3.7.3-2+deb10u3_amd64.deb ...
Unpacking libpython3.7-dev:amd64 (3.7.3-2+deb10u3) ...
Selecting previously unselected package libpython3-dev:amd64.
Preparing to unpack .../21-libpython3-dev_3.7.3-1_amd64.deb ...
Unpacking libpython3-dev:amd64 (3.7.3-1) ...
Selecting previously unselected package python-pip-whl.
Preparing to unpack .../22-python-pip-whl_18.1-5_all.deb ...
Unpacking python-pip-whl (18.1-5) ...
Selecting previously unselected package python3.7-dev.
Preparing to unpack .../23-python3.7-dev_3.7.3-2+deb10u3_amd64.deb ...
Unpacking python3.7-dev (3.7.3-2+deb10u3) ...
Selecting previously unselected package python3-dev.
Preparing to unpack .../24-python3-dev_3.7.3-1_amd64.deb ...
Unpacking python3-dev (3.7.3-1) ...
Selecting previously unselected package python3-pip.
Preparing to unpack .../25-python3-pip_18.1-5_all.deb ...
Unpacking python3-pip (18.1-5) ...
Selecting previously unselected package python3-setuptools.
Preparing to unpack .../26-python3-setuptools_40.8.0-1_all.deb ...
Unpacking python3-setuptools (40.8.0-1) ...
Selecting previously unselected package swig3.0.
Preparing to unpack .../27-swig3.0_3.0.12-2_amd64.deb ...
Unpacking swig3.0 (3.0.12-2) ...
Selecting previously unselected package swig.
Preparing to unpack .../28-swig_3.0.12-2_amd64.deb ...
Unpacking swig (3.0.12-2) ...
Setting up dh-python (3.20190308) ...
Setting up python3-setuptools (40.8.0-1) ...
Setting up binutils-common:amd64 (2.31.1-16) ...
Setting up linux-libc-dev:amd64 (4.19.194-3) ...
Setting up libasan5:amd64 (8.3.0-6) ...
Setting up swig3.0 (3.0.12-2) ...
Setting up libpcsclite-dev (1.8.24-1) ...
Setting up libmpx2:amd64 (8.3.0-6) ...
Setting up libubsan1:amd64 (8.3.0-6) ...
Setting up python-pip-whl (18.1-5) ...
Setting up libbinutils:amd64 (2.31.1-16) ...
Setting up libc-dev-bin (2.28-10) ...
Setting up libcc1-0:amd64 (8.3.0-6) ...
Setting up liblsan0:amd64 (8.3.0-6) ...
Setting up libitm1:amd64 (8.3.0-6) ...
Setting up binutils-x86-64-linux-gnu (2.31.1-16) ...
Setting up libtsan0:amd64 (8.3.0-6) ...
Setting up swig (3.0.12-2) ...
Setting up binutils (2.31.1-16) ...
Setting up python3-pip (18.1-5) ...
Setting up libgcc-8-dev:amd64 (8.3.0-6) ...
Setting up libc6-dev:amd64 (2.28-10) ...
Setting up gcc-8 (8.3.0-6) ...
Setting up gcc (4:8.3.0-1) ...
Setting up libexpat1-dev:amd64 (2.2.6-2+deb10u1) ...
Setting up libpython3.7-dev:amd64 (3.7.3-2+deb10u3) ...
Setting up python3.7-dev (3.7.3-2+deb10u3) ...
Setting up libpython3-dev:amd64 (3.7.3-1) ...
Setting up python3-dev (3.7.3-1) ...
Processing triggers for man-db (2.8.5-2) ...
Processing triggers for libc-bin (2.28-10) ...
[INFO] Examining package changes
[INFO] New packages manually installed: {'python3-dev', 'python3-pip', 'swig', 'libpcsclite-dev', 'gcc', 'python3-setuptools'}

$ torsocks pip3 install --user yubikey-manager
Collecting yubikey-manager
  Downloading https://files.pythonhosted.org/packages/5c/04/f72e4fc281dcdd96c55f3d1cecfa0e4fc450168ca16e4767ad150e5392f3/yubikey_manager-4.0.5-py3-none-any.whl (153kB)
    100% |████████████████████████████████| 163kB 434kB/s 
Collecting pyOpenSSL>=0.15.1 (from yubikey-manager)
  Downloading https://files.pythonhosted.org/packages/b2/5e/06351ede29fd4899782ad335c2e02f1f862a887c20a3541f17c3fa1a3525/pyOpenSSL-20.0.1-py2.py3-none-any.whl (54kB)
    100% |████████████████████████████████| 61kB 243kB/s 
Requirement already satisfied: click<9.0,>=6.0 in /usr/lib/python3/dist-packages (from yubikey-manager) (7.0)
Collecting pyscard<3.0,>=1.9 (from yubikey-manager)
  Downloading https://files.pythonhosted.org/packages/23/e2/42e3de90edfe9a7a0bde2d0a303aac447a4022778e8e552965db5a74ea8f/pyscard-2.0.1.tar.gz (149kB)
    100% |████████████████████████████████| 153kB 520kB/s 
Collecting fido2<1.0,>=0.9 (from yubikey-manager)
  Downloading https://files.pythonhosted.org/packages/80/c3/5077ee98edd23ee00b9f5f889fd65e8dd8dbe7717d663d3b5137e31f07e6/fido2-0.9.1.tar.gz (206kB)
    100% |████████████████████████████████| 215kB 873kB/s 
Requirement already satisfied: cryptography<4.0,>=2.1 in /usr/lib/python3/dist-packages (from yubikey-manager) (2.6.1)
Requirement already satisfied: six>=1.5.2 in /usr/lib/python3/dist-packages (from pyOpenSSL>=0.15.1->yubikey-manager) (1.12.0)
Building wheels for collected packages: pyscard, fido2
  Running setup.py bdist_wheel for pyscard ... error
  Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-n94ho61c/pyscard/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-e__4ffwy --python-tag cp37:
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help

  error: invalid command 'bdist_wheel'

  ----------------------------------------
  Failed building wheel for pyscard
  Running setup.py clean for pyscard
  Running setup.py bdist_wheel for fido2 ... error
  Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-n94ho61c/fido2/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-fjk16eby --python-tag cp37:
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help

  error: invalid command 'bdist_wheel'

  ----------------------------------------
  Failed building wheel for fido2
  Running setup.py clean for fido2
Failed to build pyscard fido2
pyopenssl 20.0.1 has requirement cryptography>=3.2, but you'll have cryptography 2.6.1 which is incompatible.
Installing collected packages: pyOpenSSL, pyscard, fido2, yubikey-manager
  Running setup.py install for pyscard ... done
  Running setup.py install for fido2 ... done
  The script ykman is installed in '/home/amnesia/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed fido2-0.9.1 pyOpenSSL-20.0.1 pyscard-2.0.1 yubikey-manager-4.0.5

emlun commented 3 years ago

Ah, I see. By "patching" I assume you mean the Tails environment? Or did you need to patch ykman itself?

For what it's worth, I saw the same errors about bdist_wheel but they don't seem to prevent ykman from working. I didn't test it exhaustively, but at least ykman oath accounts code still worked.

sunknudsen commented 3 years ago

Ah, I see. By "patching" I assume you mean the Tails environment?

Yes. I am putting together a guide that recommends using Tails as an air-gapped and hardened OS to provision YubiKeys that, once provisioned securely, can be used on less secure environments.

sunknudsen commented 3 years ago

Hey guys, can’t wait for feature! Any update on ETA?

emlun commented 3 years ago

Sorry to keep you waiting! We've built a repackaged release of 1.2.3 with the entrypoint patch from #293. It's not up on the releases page yet, but I'll update here when it is.

sunknudsen commented 3 years ago

Amazing @emlun! Thanks for update. 🤓

emlun commented 3 years ago

Alright, 1.2.3b is up now on https://developers.yubico.com/yubikey-manager-qt/Releases/ . With that, the AppImage will run the CLI instead of the GUI if called with ykman as the first argument. But we consider this an experimental, not-recommended way to use the CLI, so we're leaving it undocumented for now.

sunknudsen commented 3 years ago

Thanks so much for putting together release.

But we consider this an experimental, not-recommended way to use the CLI, so we're leaving it undocumented for now.

👍

Will “feature” be included in future releases?

Asking because I am referring to https://developers.yubico.com/yubikey-manager-qt/Releases/yubikey-manager-qt-latest-linux.AppImage in my “How to generate and air gap PGP private keys using GnuPG, Tails and YubiKey” guide.

Is there anything I can do to help feature become “non-experimental”?

sunknudsen commented 3 years ago

Btw, noticed release was signed by “Emil Lundberg (Software Developer) emil@yubico.com” vs “Dennis Fokin dennis.fokin@yubico.com”.

Will future releases be signed by Emil as well? Asking because guide currently refers to Dennis’ pub key.

Should users import both keys? Perhaps others?

sunknudsen commented 3 years ago

Just tested release on Tails. So far, everything works as expected. Thanks guys!

emlun commented 3 years ago

Yes, we intend to keep this feature in future releases.

Is there anything I can do to help feature become “non-experimental”?

Not at this time (but thank you for the offer!). Maybe we'll warm up to this being a recommended usage of the CLI in the future, but it'll need some time to mature first.

Will future releases be signed by Emil as well?

Maybe - some past releases were signed by me and by yet other developers. I'd recommend referring your readers to our Software Signing page instead of any individual key. That page includes instructions for importing and verifying new keys.

sunknudsen commented 3 years ago

@fdennis @emlun Thanks so much for helping out.

I just updated How to generate and air gap PGP private keys using GnuPG, Tails and YubiKey guide.

One can now run ykman persistently thanks to yubikey-manager-qt.AppImage ykman.

sunknudsen commented 3 years ago

Btw, I am starting work on a Bash script to automate guide in the context of enterprise GnuPG/YubiKey employee PGP private key provisioning (and cold storage).

Before reinventing the wheel, are you guys aware of tools that can be used to create and move employee PGP private keys to YubiKeys effortlessly while signing employee pub keys using organization master private key?

emlun commented 3 years ago

I don't know of one, but I haven't gone looking either.

Yubico / yubikey-manager-qt

How to run ykman bundled with Yubikey Manager AppImage? #292

Steps to reproduce

Expected result