Container opens fine with 1.5.3 but not with 1.6.x version

[alphazo]

On Archlinux, I created a container with Tomb 1.5.3 but I can no longer open it with the latest git version of Tomb.

Tomb 1.5.3

# tomb open /home/alpha/test.tomb -k /home/alpha/test.tomb.key

tomb  .  Commanded to open tomb /home/alpha/test.tomb
tomb [W] An active swap partition is detected, this poses security risks.
tomb (*) All your swaps are belong to crypt.  Good.
tomb  .  Valid key file found: /home/alpha/test.tomb.key
tomb  .  Mountpoint not specified, using default: /media/test.tomb
tomb (*) Opening test.tomb on /media/test.tomb
tomb  .  This tomb is a valid LUKS encrypted device.
tomb  .  Cipher is "aes" mode "xts-plain64:sha256" hash "sha1"
tomb  .  A password is required to use key test.tomb.key
tomb  .  Password OK.
tomb (*) Success unlocking tomb test
tomb  .  Checking filesystem via /dev/loop0
fsck from util-linux 2.25.1
test: clean, 38/12288 files, 20359/49152 blocks
tomb (*) Success opening test.tomb on /media/test.tomb
tomb  .  Last visit by alpha(1000) from /dev/pts/0 on fatfly
tomb  .  on date dim. 26 oct. 2014 21:23:08 CET

# tomb close
tomb  .  Tomb close 
tomb  .  Closing tomb [test] mounted on /media/test.tomb
tomb (*) Tomb [test] closed: your bones will rest in peace.

Tomb git version (1.6.x)

# tomb open /home/alpha/test.tomb -k /home/alpha/test.tomb.key
tomb  .  Commanded to open tomb /home/alpha/test.tomb
tomb [W] An active swap partition is detected, this poses security risks.
tomb (*) All your swaps are belong to crypt.  Good.
tomb  .  Key is valid.
tomb  .  Mountpoint not specified, using default: /media/test.tomb
tomb (*) Opening test.tomb on /media/test.tomb
tomb  .  Valid tomb file found: /home/alpha/test.tomb
losetup: write error: Bad file descriptor
tomb [W] Loop mount of volumes is not possible on this machine, this error
tomb [W] often occurs on VPS and kernels that don't provide the loop module.
tomb [W] It is impossible to use Tomb on this machine at this conditions.
tomb [E] Operation aborted.

[hellekin]

RHAAA Hello @alphazo, can you try the version in the cleanup branch? [sorry for the spam]

[jaromil]

Cleanup shouldn't be changing anything in this, but good to try since we are currently working on integrating it and later will look into this.

Looks like loopback handling has probs here... Not sure yet about the origin of prob

[alphazo]

I tried the cleanup branch but it doesn't help. Forgot to mention that I'm using GnuPG 2.1 (dev branch) though.

 tomb --version
 Tomb 1.7 - a strong and gentle undertaker for your secrets

  Copyright (C) 2007-2014 Dyne.org Foundation, License GNU GPL v3+
  This is free software: you are free to change and redistribute it
  The latest Tomb sourcecode is published on <http://tomb.dyne.org>

  This source code is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
  Please refer to the GNU Public License for more details.

 System utils:

 Sudo version 1.8.11p1
 cryptsetup 1.6.6
 pinentry-gtk2 0.8.3
 gpg (GnuPG) 2.1.0-beta890 - key forging algorithms (GnuPG symmetric ciphers):

[jaromil]

ACK. Gpg2, that must be it.

We need a solution to this.

Thanks for reporting!

[alphazo]

Gnupg2 is the default gpg implementation on ArchLinux (gpg is actually an alias to gpg2). I went back to the official gpg 2.0.26 instead of the 2.1 dev branch and still can't open the container with Tomb 1.6.x version while it opens fine with Tomb 1.5.3.

[jaromil]

Yes. Current git HEAD (1.6) has refactored key usage and this must be a bug introduced.

We need to look into this before releasing it stable of course.

Using status-fd may be a fix.

[jaromil]

Can you try again with current git HEAD? We are done with the cleanup and it passes all tests on my systems.

[alphazo]

I just compiled the latest git HEAD but unfortunately I get the exact same error message. For information I'm using gnupg 2.0.26 found in ArchLinux [core].

[jaromil]

Can you tell us utilities versions printed out by tomb -v and then also can you run the command reporting error with tomb -D ... so to have debugging messages? From the first log you pasted looks more like there is a problem with loopfiles. Still strange that 1.5.3 works and latest not.

[jaromil]

I have tested GPG2 and it seems unrelated to the problem, simply works. Please check the loopback device limit on your computer and that isn't reached.

[alphazo]

Here is what I get:

  tomb -v
  Tomb 1.7 - a strong and gentle undertaker for your secrets

  Copyright (C) 2007-2014 Dyne.org Foundation, License GNU GPL v3+
  This is free software: you are free to change and redistribute it
  The latest Tomb sourcecode is published on <http://tomb.dyne.org>

  This source code is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
  Please refer to the GNU Public License for more details.

 System utils:

Sudo version 1.8.11p2
cryptsetup 1.6.6
pinentry-gtk2 0.8.4
gpg (GnuPG) 2.0.26 - key forging algorithms (GnuPG symmetric ciphers):

And the command with debug option set:

 tomb -D open /home/alpha/test.tomb -k /home/alpha/test.tomb.key
 tomb [D] Tomb command: open /home/alpha/test.tomb
 tomb [D] Caller: uid[1000], gid[100], tty[/dev/pts/1].
 tomb [D] Using sudo for root execution of '/usr/bin/tomb -D open /home/alpha/test.tomb -k /home/alpha/test.tomb.key'.
 tomb [D] Tomb command: open /home/alpha/test.tomb
 tomb [D] Caller: uid[1000], gid[100], tty[/dev/pts/1].
 tomb  .  Commanded to open tomb /home/alpha/test.tomb
 tomb [W] An active swap partition is detected, this poses security risks.
 tomb (*) All your swaps are belong to crypt.  Good.
 tomb [D] is_valid_tomb /home/alpha/test.tomb
 tomb  .  Valid tomb file found: /home/alpha/test.tomb
 tomb [D] Tomb found: /home/alpha/test.tomb
 tomb [D] load_key argument: /home/alpha/test.tomb.key
 tomb [D] load_key: /home/alpha/test.tomb.key
 tomb [D] is_valid_key
 tomb  .  Key is valid.
 tomb  .  Mountpoint not specified, using default: /media/test.tomb
 tomb (*) Opening test.tomb on /media/test.tomb
 tomb [D] is_valid_tomb /home/alpha/test.tomb
 tomb  .  Valid tomb file found: /home/alpha/test.tomb
 losetup: write error: Bad file descriptor
 tomb [W] Loop mount of volumes is not possible on this machine, this error
 tomb [W] often occurs on VPS and kernels that don't provide the loop module.
 tomb [W] It is impossible to use Tomb on this machine at this conditions.
 tomb [E] Operation aborted.

[alphazo]

My system is setup for 8 loopback devices but none of them is used. Just to make sure I created a dummy configuration file for loop module

 tee /etc/modprobe.d/moreloop.conf << "options loop max_loop=64"
 rmmod loop
 modprobe loop

And I can see that the number of potential loopback devices listed in /dev/loop* went from 8 to 64. Tried again then Tomb but still no luck.

[jaromil]

The problem is in lo_mount() on line 471.

Try see what can be fixed? The line reporting a problem is:

    # check if we have support for loop mounting
    losetup -f >& -

I suspect the stdout redirection tricks may be provoking problems?

[melon3r]

I think so.

% sudo losetup -f >& -
losetup: write error: Bad file descriptor

[jaromil]

I wrote 'should' since I couldn't reproduce the problem. @alphazo please let us know if this commit fixes.

[alphazo]

Well, before testing I need to figure out why I can no longer package it under ArchLinux:

 ==> Building and installing package
 ==> Making package: tomb-git 431.58f7248-1 (dim. nov. 16 12:33:34 CET 2014)
 ==> Checking runtime dependencies...
 ==> Checking buildtime dependencies...
 ==> Retrieving sources...
   -> Cloning tomb git repo...
 Cloning into bare repository '/tmp/yaourt-tmp-alpha/aur-tomb-git/tomb'...
 remote: Counting objects: 2940, done.
 remote: Compressing objects: 100% (245/245), done.
 remote: Total 2940 (delta 155), reused 133 (delta 46)
 Receiving objects: 100% (2940/2940), 5.88 MiB | 1.17 MiB/s, done.
 Resolving deltas: 100% (1747/1747), done.
 Checking connectivity... done.
   -> Found tomb.install
   -> Found _tomb
 ==> Validating source files with sha256sums...
     tomb ... Skipped
     tomb.install ... Passed
     _tomb ... Passed
 ==> Extracting sources...
   -> Creating working copy of Tomb git repo...
 Cloning into 'tomb'...
 done.
 ==> Starting pkgver()...
 ==> Updated version: tomb-git 589.da46cbc-1
 ==> Starting build()...
 gcc -O2 -o tomb-kdb-pbkdf2 pbkdf2.c -lgcrypt
 gcc -O2 -o tomb-kdb-pbkdf2-getiter benchmark.c -lgcrypt
 gcc -O2 -o tomb-kdb-pbkdf2-gensalt gen_salt.c -lgcrypt
 gcc -O2 -o hexencode hexencode.c
 ==> Entering fakeroot environment...
 ==> Starting package()...
 make[1]: Entering directory '/tmp/yaourt-tmp-alpha/aur-tomb-git/src/tomb/extras/po'
 msgfmt -o es.mo es.po
 msgfmt -o ru.mo ru.po
 msgfmt -o fr.mo fr.po
 msgfmt -o de.mo de.po
 install: cannot create regular file ‘/usr/share/locale/es/LC_MESSAGES/tomb.mo’: Permission denied
 Makefile:11: recipe for target 'install' failed
 make[1]: *** [install] Error 1
 make[1]: Leaving directory '/tmp/yaourt-tmp-alpha/aur-tomb-git/src/tomb/extras/po'
 Makefile:16: recipe for target 'install' failed
 make: *** [install] Error 2
 ==> ERROR: A failure occurred in package().
     Aborting...
 ==> ERROR: Makepkg was unable to build tomb-git.
 ==> Restart building tomb-git ? [y/N]

[jaromil]

make install now executes the extras/po Makefile too, to install translations.

I'd rather leave it so. Not sure you want to have a tomb-intl separated package.

However we can choose now, since gettext is no more a requirement.

I think it should be easy to solve this for your package.

[melon3r]

I don't know arch, but if you can't install files to /usr/share, will it be able to install the tomb executable (/usr/local) or the man page (/usr/share)?

[jaromil]

Last commit should fix the situation, also for the manpage.

[alphazo]

Thanks for fixing the makefile. New version installed and compiled just fine. We are gettting close since the proposed fix for the losetup also seems to be working. Now I can enter my password and it gets accepted but something goes wrong toward the end.

  tomb -D open /home/alpha/test.tomb -k /home/alpha/test.tomb.key
  tomb [D] Tomb command: open /home/alpha/test.tomb
  tomb [D] Caller: uid[1000], gid[100], tty[/dev/pts/0].
  tomb [D] Using sudo for root execution of '/usr/bin/tomb -D open /home/alpha/test.tomb -k /home/alpha/test.tomb.key'.
  tomb [D] Tomb command: open /home/alpha/test.tomb
  tomb [D] Caller: uid[1000], gid[100], tty[/dev/pts/0].
  tomb  .  Commanded to open tomb /home/alpha/test.tomb
  tomb [W] An active swap partition is detected, this poses security risks.
  tomb (*) All your swaps are belong to crypt.  Good.
  tomb [D] is_valid_tomb /home/alpha/test.tomb
  tomb  .  Valid tomb file found: /home/alpha/test.tomb
  tomb [D] Tomb found: /home/alpha/test.tomb
  tomb [D] load_key argument: /home/alpha/test.tomb.key
  tomb [D] load_key: /home/alpha/test.tomb.key
  tomb [D] is_valid_key
  tomb  .  Key is valid.
  tomb  .  Mountpoint not specified, using default: /media/test.tomb
  tomb (*) Opening test.tomb on /media/test.tomb
  tomb  .  This tomb is a valid LUKS encrypted device.
  tomb  .  Cipher is "aes" mode "xts-plain64:sha256" hash "sha1"
  tomb [D] dev mapper device: tomb.test.1416143988.loop0
  tomb [D] Tomb key: /home/alpha/test.tomb.key
  tomb [D] Tomb name: test (to be engraved)
  tomb  .  A password is required to use key /home/alpha/test.tomb.key
  tomb [D] exec_as_user 'alpha': /usr/bin/tomb askpass Insert password to use key: /home/alpha/test.tomb.key
  tomb [D] get_lukskey
  tomb [D] Created tempfile: /dev/shm/1000/2349419651.22155
  tomb [D] get_lukskey returns 0
  tomb  .  Password OK.
  No key available with this passphrase.
  tomb [E] Failure mounting the encrypted file.

[jaromil]

This is very tricky to debug like this. It seems to me that the decrypted output by GPG2 does not corresponds to the original key created: maybe some status lines are added or so. Yet, I cannot reproduce this bug on any of my machines. I hope you can investigate this further: decrypt the key by hand using gpg and inspect its contents, try to manually run luksOpen and such?

[jaromil]

For instance try gpg -d test.tomb.key | hexdump -c and paste the output.

Here a sequence of commands (to be run as root) to open a tomb without using Tomb, should help backtrace the problem:

 lo=$(losetup -f)
 losetup -f secret.tomb
 pass=$(gpg -d secret.key)
 echo -ne "$pass" | cryptsetup --key-file - luksOpen $lo secret
 mount /dev/mapper/secret $HOME/secret-contents

Please note that $pass must be printed without a trailing newline (-n) and interpreting all escape sequences (-e). The latter should be the default in Zsh, but if that is not the case in Arch then changing it in print -R -e -n on line 1697 should succeed in opening your tomb.

Please let us know if that is the case.

[jaromil]

Ok found an Arch box to test at the local hacklab :^) the problem you have does not occurs. I'm guessing is something on your side (old keys used with new tomb? any strange shell setting?). I hope you can debug it using the information above.

[alphazo]

I couldn't compile the latest git version and found out that there is a missing "tomb-kdb-' in /extras/kdf-keys/Makefile on the line containing "install -Dm755 hexencode ${DESTDIR}${PREFIX}/bin/tomb-kdb-hexencode"

So I restarted from scratch with Tomb 1.5.3 and created a brand new container, opened it and copied a file to it then closed it. I then compiled the latest git version with the above fix and tried to open the generated container and it fails the exact same way after the password has been accepted.

You can find both the container and keyfile here: http://natzo.com/files/test4.tomb-issue.zip Password for this non-sensitve container is "test" without the quotes.

Systems tools are based upon the following versions: Sudo version 1.8.11p2 cryptsetup 1.6.6 pinentry-gtk2 0.8.4 gpg (GnuPG) 2.0.26

Here is the result of the commands you asked me to try:

  [root@fatfly alpha]# gpg -d test4.tomb.key | hexdump -c
  gpg: données chiffrées avec AES256
  gpg: chiffré avec 1 phrase de passe
  0000000   n   < 204 220   e 341 202  \b 370 216   Y 362 222 221  \v 270
  0000010 017   t 017 021 267 034   b 032 332   R 216 354 243 254   (   `
  0000020 352   B 255   q 211   .   t 371 022 224   U   _   z   b 034   ~
  0000030   {  \r   5  \b   g 232  \b 271   t   t   ? 275   p 373  \t 220
  0000040 214   o 274   w 202   B 323   9   - 274 253 025   6   L   r   9
  0000050 367 036 210 027   q 270   f   5   f 033   "   @   F 235   4 002
  0000060 270 006   R   J 367 320   2   }   |   u   K   K 254 006 253 307
  0000070   C   ]   X   [ 211 215   "   d 336 273   X 336 031 004   l 025
  0000080 006 034   \   z 220 036 237 201 245   >   @ 353   B   B 353 256
  0000090 257   f 323 035 365 235   4 326 327 035 316 004 257   s 250 243
  00000a0 330 325 032 024 326 320 021 346   W 331 352 261   0 210  \n 306
  00000b0  \0   P   V 271 024 017 025 333   R 274   ^   ?   y 362 206   _
  00000c0   5 312   S 347 251 023   E 222 210 206 301 223 200 364   \   =
  00000d0 035 236 016 033 265 375   L 257   r 276   ;   J 202   l   _   0
  00000e0 270   [   s 276 275   f   C 355 303   x   #   &   u 227 365 317
  00000f0   h 261   G  \t   D 317  \f   R 200 262   _ 002   }   m   p 274
  0000100

And the cryptsetup output (with added --debug option):

  [root@fatfly alpha]# echo -ne "$pass" | cryptsetup --debug --key-file - luksOpen $lo test4
  # cryptsetup 1.6.6 processing "cryptsetup --debug --key-file - luksOpen /dev/loop2 test4"
  # Running command open.
  # Locking memory.
  # Installing SIGINT/SIGTERM handler.
  # Unblocking interruption on signal.
  # Allocating crypt device /dev/loop2 context.
  # Trying to open and read device /dev/loop2.
  # Initialising device-mapper backend library.
  # Trying to load LUKS1 crypt type from device /dev/loop2.
  # Crypto backend (gcrypt 1.6.2) initialized.
  # Detected kernel Linux 3.17.2-1-ARCH x86_64.
  # Reading LUKS header of size 1024 from device /dev/loop2
  # Key length 32, device size 20480 sectors, header size 2050 sectors.
  # Timeout set to 0 miliseconds.
  # Password retry count set to 3.
  # Password verification disabled.
  # Iteration time set to 1000 miliseconds.
  # Password retry count set to 1.
  # Activating volume test4 [keyslot -1] using keyfile -.
  # dm version   OF   [16384] (*1)
  # dm versions   OF   [16384] (*1)
  # Detected dm-crypt version 1.13.0, dm-ioctl version 4.27.0.
  # Device-mapper backend running with UDEV support enabled.
  # dm status test4  OF   [16384] (*1)
  # STDIN descriptor passphrase entry requested.
  # Trying to open key slot 0 [ACTIVE_LAST].
  # Reading key slot 0 area.
  # Userspace crypto wrapper cannot use aes-xts-plain64:sha256 (-2).
  # Using dmcrypt to access keyslot area.
  # Calculated device size is 250 sectors (RW), offset 8.
  # DM-UUID is CRYPT-TEMP-temporary-cryptsetup-28959
  # Udev cookie 0xd4d08c8 (semid 1638400) created
  # Udev cookie 0xd4d08c8 (semid 1638400) incremented to 1
  # Udev cookie 0xd4d08c8 (semid 1638400) incremented to 2
  # Udev cookie 0xd4d08c8 (semid 1638400) assigned to CREATE task(0) with flags DISABLE_SUBSYSTEM_RULES DISABLE_DISK_RULES DISABLE_OTHER_RULES         (0xe)
  # dm create temporary-cryptsetup-28959 CRYPT-TEMP-temporary-cryptsetup-28959 OF   [16384] (*1)
  # dm reload temporary-cryptsetup-28959  OFRW    [16384] (*1)
  # dm resume temporary-cryptsetup-28959  OFRW    [16384] (*1)
  # temporary-cryptsetup-28959: Stacking NODE_ADD (254,4) 0:0 0600 [verify_udev]
  # temporary-cryptsetup-28959: Stacking NODE_READ_AHEAD 256 (flags=1)
  # Udev cookie 0xd4d08c8 (semid 1638400) decremented to 1
  # Udev cookie 0xd4d08c8 (semid 1638400) waiting for zero
  # Udev cookie 0xd4d08c8 (semid 1638400) destroyed
  # temporary-cryptsetup-28959: Processing NODE_ADD (254,4) 0:0 0600 [verify_udev]
  # temporary-cryptsetup-28959: Processing NODE_READ_AHEAD 256 (flags=1)
  # temporary-cryptsetup-28959 (254:4): read ahead is 256
  # temporary-cryptsetup-28959: retaining kernel read ahead of 256 (requested 256)
  # Udev cookie 0xd4de475 (semid 1671168) created
  # Udev cookie 0xd4de475 (semid 1671168) incremented to 1
  # Udev cookie 0xd4de475 (semid 1671168) incremented to 2
  # Udev cookie 0xd4de475 (semid 1671168) assigned to REMOVE task(2) with flags         (0x0)
  # dm remove temporary-cryptsetup-28959  OFT    [16384] (*1)
  # temporary-cryptsetup-28959: Stacking NODE_DEL [verify_udev]
  # Udev cookie 0xd4de475 (semid 1671168) decremented to 0
  # Udev cookie 0xd4de475 (semid 1671168) waiting for zero
  # Udev cookie 0xd4de475 (semid 1671168) destroyed
  # temporary-cryptsetup-28959: Processing NODE_DEL [verify_udev]
  # Trying to open key slot 1 [INACTIVE].
  # Trying to open key slot 2 [INACTIVE].
  # Trying to open key slot 3 [INACTIVE].
  # Trying to open key slot 4 [INACTIVE].
  # Trying to open key slot 5 [INACTIVE].
  # Trying to open key slot 6 [INACTIVE].
  # Trying to open key slot 7 [INACTIVE].
  No key available with this passphrase.
  # Releasing crypt device /dev/loop2 context.
  # Releasing device-mapper backend.
  # Unlocking memory.
  Command failed with code 1: No key available with this passphrase.

[alphazo]

Some more food for thoughts. This time I created the container using the git version and could open it just fine with Tomb but...I could NOT open it using the manual cryptsetup method!

  lo=$(losetup -f)
  [root@fatfly alpha]# losetup -f test5-v2.tomb
  [root@fatfly alpha]# pass=$(gpg -d test5-v2.tomb.key)
  gpg: AES256 encrypted data
  gpg: encrypted with 1 passphrase
  [root@fatfly alpha]# echo -ne "$pass" | cryptsetup --debug --key-file - luksOpen $lo test5-v2
  # cryptsetup 1.6.6 processing "cryptsetup --debug --key-file - luksOpen /dev/loop3 test5-v2"
  # Running command open.
  # Locking memory.
  # Installing SIGINT/SIGTERM handler.
  # Unblocking interruption on signal.
  # Allocating crypt device /dev/loop3 context.
  # Trying to open and read device /dev/loop3.
  # Initialising device-mapper backend library.
  # Trying to load LUKS1 crypt type from device /dev/loop3.
  # Crypto backend (gcrypt 1.6.2) initialized.
  # Detected kernel Linux 3.17.2-1-ARCH x86_64.
  # Reading LUKS header of size 1024 from device /dev/loop3
  # Key length 32, device size 20480 sectors, header size 2050 sectors.
  # Timeout set to 0 miliseconds.
  # Password retry count set to 3.
  # Password verification disabled.
  # Iteration time set to 1000 miliseconds.
  # Password retry count set to 1.
  # Activating volume test5-v2 [keyslot -1] using keyfile -.
  # dm version   OF   [16384] (*1)
  # dm versions   OF   [16384] (*1)
  # Detected dm-crypt version 1.13.0, dm-ioctl version 4.27.0.
  # Device-mapper backend running with UDEV support enabled.
  # dm status test5-v2  OF   [16384] (*1)
  # STDIN descriptor passphrase entry requested.
  # Trying to open key slot 0 [ACTIVE_LAST].
  # Reading key slot 0 area.
  # Userspace crypto wrapper cannot use aes-xts-plain64:sha256 (-2).
  # Using dmcrypt to access keyslot area.
  # Calculated device size is 250 sectors (RW), offset 8.
  # DM-UUID is CRYPT-TEMP-temporary-cryptsetup-30853
  # Udev cookie 0xd4d973a (semid 2097152) created
  # Udev cookie 0xd4d973a (semid 2097152) incremented to 1
  # Udev cookie 0xd4d973a (semid 2097152) incremented to 2
  # Udev cookie 0xd4d973a (semid 2097152) assigned to CREATE task(0) with flags DISABLE_SUBSYSTEM_RULES DISABLE_DISK_RULES DISABLE_OTHER_RULES         (0xe)
  # dm create temporary-cryptsetup-30853 CRYPT-TEMP-temporary-cryptsetup-30853 OF   [16384] (*1)
  # dm reload temporary-cryptsetup-30853  OFRW    [16384] (*1)
  # dm resume temporary-cryptsetup-30853  OFRW    [16384] (*1)
  # temporary-cryptsetup-30853: Stacking NODE_ADD (254,4) 0:0 0600 [verify_udev]
  # temporary-cryptsetup-30853: Stacking NODE_READ_AHEAD 256 (flags=1)
  # Udev cookie 0xd4d973a (semid 2097152) decremented to 1
  # Udev cookie 0xd4d973a (semid 2097152) waiting for zero
  # Udev cookie 0xd4d973a (semid 2097152) destroyed
  # temporary-cryptsetup-30853: Processing NODE_ADD (254,4) 0:0 0600 [verify_udev]
  # temporary-cryptsetup-30853: Processing NODE_READ_AHEAD 256 (flags=1)
  # temporary-cryptsetup-30853 (254:4): read ahead is 256
  # temporary-cryptsetup-30853: retaining kernel read ahead of 256 (requested 256)
  # Udev cookie 0xd4dab01 (semid 2129920) created
  # Udev cookie 0xd4dab01 (semid 2129920) incremented to 1
  # Udev cookie 0xd4dab01 (semid 2129920) incremented to 2
  # Udev cookie 0xd4dab01 (semid 2129920) assigned to REMOVE task(2) with flags         (0x0)
  # dm remove temporary-cryptsetup-30853  OFT    [16384] (*1)
  # temporary-cryptsetup-30853: Stacking NODE_DEL [verify_udev]
  # Udev cookie 0xd4dab01 (semid 2129920) decremented to 0
  # Udev cookie 0xd4dab01 (semid 2129920) waiting for zero
  # Udev cookie 0xd4dab01 (semid 2129920) destroyed
  # temporary-cryptsetup-30853: Processing NODE_DEL [verify_udev]
  # Trying to open key slot 1 [INACTIVE].
  # Trying to open key slot 2 [INACTIVE].
  # Trying to open key slot 3 [INACTIVE].
  # Trying to open key slot 4 [INACTIVE].
  # Trying to open key slot 5 [INACTIVE].
  # Trying to open key slot 6 [INACTIVE].
  # Trying to open key slot 7 [INACTIVE].
  No key available with this passphrase.
  # Releasing crypt device /dev/loop3 context.
  # Releasing device-mapper backend.
  # Unlocking memory.
  Command failed with code 1: No key available with this passphrase.

BTW, I don't know it this is related but in both cases there is intriguing message using cryptsetup:

  Userspace crypto wrapper cannot use aes-xts-plain64:sha256 (-2).

[jaromil]

If tomb now works, consider nothing has changed in the code regarding this issue, so either we have a regression problem either there was some irregularity on creation at your side. The manual code snippet is not throughly tested, I look fwd to more feedback about it, it is now also in the readme.

On regression I did tests on old tombs, but will run them again.

[jaromil]

I did reproduce a compatibility problem with tombs created with 1.5.3 and opend with current HEAD.

Investigating this further. Holding back the release until solved.

[jaromil]

Found! the culprit are escape chars in secret key material: whenever they are present they are discarded by print. I have the fix working and will push it whenever I find a proper Internet connection.

[alphazo]

Thanks for the fix. I can now open the test container I posted above, test4.tomb, generated with 1.5.3. Now I still have my original test.tomb that was created on August 28 with I believe 1.5.3 and for which I'm still getting the "No key available with this passphrase". I have sanitized its content and might be able to send it over.

[jaromil]

Are you sure you didn't use the Git version to create that? can you check?

The key handling refactoring introduced this bug and was committed on August 27.

[alphazo]

Hi Jaromil, is there a simple way to check the version of Tomb that was used to generate a given container?

[jaromil]

Not yet. Its a good idea to have it. Obviously users may remove that info and it would be removed from keys when steghide is used. Yet we don't have it to solve this problem now.

Having run all kinds of tests now I'd rather skip over this one. You may try to 'bruteforce' opening the tomb with a key that has a '\' shifting through it in every possible position: it is highly probable that it was one of those (and just one) and it was omitted by this bug when saved into the key. This can be also debugged using hexdump -c and diff on keys printed with different techniques. But its all rather tricky.

[alphazo]

Would you like to get the test.tomb and associated test.tomb.key.

BTW, here is the hexdump for this particular container.

  gpg -d test.tomb.key | hexdump -c
  gpg: AES256 encrypted data
  gpg: encrypted with 1 passphrase
  0000000 340   / 327   4 365 200 304 360   ~ 253   { 201 225 211   3 360
  0000010   _ 030 213   %   _  \b   <   i   O 232 260 360   H   0 365   9
  0000020   p   ]   ' 271 343 216   " 217   g   I   m 305 177 030 254 212
  0000030 005   S 265 037 317   O 210   l 313   H 217   L   > 033   \ 362
  0000040 205  \a   S   + 311   E 224 227 364   G   ` 231   !   N 363 225
  0000050 257   @   1 354  \t 356 236 247   4   @ 210 001 232 260 205   [
  0000060   _ 374 317   [   N   o 264 345   ^   B 016   X   *  \a 021 375
  0000070   ] 364 273   M 026 316 255 263 200   b 372   +   \ 225   ` 244
  0000080 360 354 240 253 200 270 323   N 337 237   f 272   p       5 022
  0000090 350 234   Q 271   6 272   d   u 361   <   v 001   7 224 032   M
  00000a0   P   r 363   k 334 271 323 304   f 362   " 302 001 210 325 004
  00000b0 364 271 220   0 322   D 257 210   3 020   ]   t   +   } 034   <
  00000c0   { 350  \r   M 256 377 256 373   g 245 361   " 351   '   f   }
  00000d0 321 215 302   S   # 367 231 355 032 317   < 205 006   G 301   -
  00000e0 303   u 262 237   [   7   Z   7   ; 023   O 031   7   0   n 004
  00000f0 225   " 310  \v   m 235 241 302 267 363 037   D 320 377 354  \n
  0000100

[jaromil]

The key material contains more than one escape. Ok please upload it and I'll try to figure out what's the problem with this one, good to make sure we are not leaving any of such bugs unsquashed.

[jaromil]

Please also specify with what version of Tomb (or commit in history) you are able to open this particular tomb.

[alphazo]

Just sent you a PM with a link to the tomb.

[jaromil]

Thanks. Loved the GNU logo :^)

I have tested it and in this case the difference is just the final newline. You can verify this yourself removing the -n from the print in _cryptsetup() hence changing the whole line in:

    print -R - "$TOMBSECRET" | cryptsetup --key-file - ${=@}

your test tomb will open. This simply includes a final newline. I guess this error is only present in tombs created during an interim and recent period of development.

[alphazo]

Good news and thanks for the analysis. Shall we close that thread then?