${#*} → Failure: lexing: empty token.

Niols commented 5 years ago

When running morbig --as simple on the file at the end of this issue, I get

Failure: lexing: empty token.

as an output from Morbig. The Shell script seems to be accepted by other shells. I'll try to narrow this script down to the place where the "bug" occurs.

if [ "${0#/usr/share/initramfs-tools/hooks/}" != "$0" ] ||
        [ "${0#/etc/initramfs-tools/hooks/}" != "$0" ]; then
    # called from an initramfs-tools hook script
    TABFILE="$DESTDIR/cryptroot/crypttab"
elif [ "${0#/scripts/}" != "$0" ]; then
    # called at initramfs stage from a boot script
    TABFILE="/cryptroot/crypttab"
else
    TABFILE="${TABFILE-/etc/crypttab}"
fi
export DM_DEFAULT_NAME_MANGLING_MODE=hex # for dmsetup(8)

# Logging helpers. Send the argument list to plymouth(1), or fold it
# and print it to the standard error.
cryptsetup_message() {
    local IFS=' '
    if [ "${0#/scripts/}" != "$0" ] && [ -x /bin/plymouth ] && plymouth --ping; then
        plymouth message --text="cryptsetup: $*"
    elif [ ${#*} -lt 70 ]; then
        echo "cryptsetup: $*" >&2
    else
        # use busybox's fold(1) and sed(1) at initramfs stage
        echo "cryptsetup: $*" | fold -s | sed '1! s/^/    /' >&2
    fi
    return 0
}

# crypttab_parse_options([--export], [--quiet])
#   Parse $_CRYPTTAB_OPTIONS, a comma-separated option string from the
#   crypttab(5) 4th column, and sets corresponding variables
#   CRYPTTAB_OPTION_<option>=<value> (which are added to the environment
#   if --export is set).
#   For error and warning messages, CRYPTTAB_NAME, (resp. CRYPTTAB_KEY)
#   should be set to the (unmangled) mapped device name (resp. key
#   file).
#   Return 1 on parsing error, 0 otherwise (incl. if unknown options
#   were encountered).
crypttab_parse_options() {
    local quiet="n" export="n"
    while [ $# -gt 0 ]; do
        case "$1" in
            --quiet) quiet="y";;
            --export) export="y";;
            *) cryptsetup_message "WARNING: crypttab_parse_options(): unknown option $1"
        esac
        shift
    done

    local IFS=',' x OPTION VALUE
    unset -v CRYPTTAB_OPTION_cipher \
             CRYPTTAB_OPTION_size \
             CRYPTTAB_OPTION_hash \
             CRYPTTAB_OPTION_offset \
             CRYPTTAB_OPTION_skip \
             CRYPTTAB_OPTION_verify \
             CRYPTTAB_OPTION_readonly \
             CRYPTTAB_OPTION_discard \
             CRYPTTAB_OPTION_plain \
             CRYPTTAB_OPTION_luks \
             CRYPTTAB_OPTION_tcrypt \
             CRYPTTAB_OPTION_veracrypt \
             CRYPTTAB_OPTION_swap \
             CRYPTTAB_OPTION_tmp \
             CRYPTTAB_OPTION_check \
             CRYPTTAB_OPTION_checkargs \
             CRYPTTAB_OPTION_tries \
             CRYPTTAB_OPTION_initramfs \
             CRYPTTAB_OPTION_noearly \
             CRYPTTAB_OPTION_noauto \
             CRYPTTAB_OPTION_loud \
             CRYPTTAB_OPTION_quiet \
             CRYPTTAB_OPTION_keyscript \
             CRYPTTAB_OPTION_keyslot \
             CRYPTTAB_OPTION_header \
             CRYPTTAB_OPTION_tcrypthidden
    # use $_CRYPTTAB_OPTIONS not $CRYPTTAB_OPTIONS as options values may
    # contain '\054' which is decoded to ',' in the latter
    for x in $_CRYPTTAB_OPTIONS; do
        OPTION="${x%%=*}"
        VALUE="${x#*=}"
        if [ "$x" = "$OPTION" ]; then
            unset -v VALUE
        else
            VALUE="$(printf '%b' "$VALUE")"
        fi
        if ! crypttab_validate_option; then
            if [ "$quiet" = "n" ]; then
                cryptsetup_message "ERROR: $CRYPTTAB_NAME: invalid value for '${x%%=*}' option, skipping"
            fi
            return 1
        elif [ -z "${OPTION+x}" ]; then
            continue
        fi
        if [ "$export" = "y" ]; then
            export "CRYPTTAB_OPTION_$OPTION"="${VALUE-yes}"
        else
            eval "CRYPTTAB_OPTION_$OPTION"='${VALUE-yes}'
        fi
    done
    IFS=" "

    if [ "$quiet" = "n" ] &&
            [ -z "${CRYPTTAB_OPTION_luks+x}" ] && [ -n "${CRYPTTAB_OPTION_header+x}" ]; then
        cryptsetup_message "WARNING: Option 'luks' missing in crypttab for target $CRYPTTAB_NAME." \
                           "Headers are only supported for LUKS devices."
    fi
    if [ -z "${CRYPTTAB_OPTION_luks+x}" ] && [ -z "${CRYPTTAB_OPTION_tcrypt+x}" ]; then
        # the compiled-in default for these are subject to change
        options='cipher size'
        if [ -n "${CRYPTTAB_OPTION_keyscript+x}" ] || [ "$CRYPTTAB_KEY" = "none" ]; then
            options="$options hash" # --hash is being ignored in plain mode with keyfile specified
        fi
        for o in $options; do
            if [ "$quiet" = "n" ] && eval [ -z "\${CRYPTTAB_OPTION_$o+x}" ]; then
                cryptsetup_message "WARNING: Option '$o' missing in crypttab for plain dm-crypt" \
                    "mapping $CRYPTTAB_NAME. Please read /usr/share/doc/cryptsetup/README.initramfs and" \
                    "add the correct '$o' option to your crypttab(5)."
            fi
        done
    fi
}

# crypttab_validate_option()
#   Validate $OPTION=$VALUE (or flag $OPTION if VALUE is unset).  return
#   1 on error, unsets OPTION for unknown or useless options.
crypttab_validate_option() {
    # option aliases
    case "$OPTION" in
        read-only) OPTION="readonly";;
        key-slot) OPTION="keyslot";;
        tcrypt-hidden) OPTION="tcrypthidden";;
        tcrypt-veracrypt) OPTION="veracrypt";;
    esac

    # sanitize the option name so CRYPTTAB_OPTION_$OPTION is a valid variable name
    local o="$OPTION"
    case "$o" in
        keyfile-offset) OPTION="keyfile_offset";;
        keyfile-size) OPTION="keyfile_size";;
    esac

    case "$o" in
        # value must be a non-empty string
        cipher|hash|header)
            [ -n "${VALUE:+x}" ] || return 1
        ;;
        # numeric options >0
        size|keyfile-size)
            if ! printf '%s' "${VALUE-}" | grep -Exq "0*[1-9][0-9]*"; then
                return 1
            fi
        ;;
        # numeric options >=0
        offset|skip|tries|keyslot|keyfile-offset)
            if ! printf '%s' "${VALUE-}" | grep -Exq "[0-9]+"; then
                return 1
            fi
        ;;
        tmp)
            if [ -z "${VALUE+x}" ]; then
                VALUE="ext4" # 'tmp flag'
            elif [ -z "$VALUE" ]; then
                return 1
            fi
        ;;
        check)
            if [ -z "${VALUE+x}" ]; then
                if [ -n "${CRYPTDISKS_CHECK-}" ]; then
                    VALUE="$CRYPTDISKS_CHECK"
                else
                    unset -v OPTION
                    return 0
                fi
            fi
            if [ -x "/lib/cryptsetup/checks/$VALUE" ] && [ -f "/lib/cryptsetup/checks/$VALUE" ]; then
                VALUE="/lib/cryptsetup/checks/$VALUE"
            elif [ ! -x "$VALUE" ] || [ ! -f "$VALUE" ]; then
                return 1
            fi
        ;;
        checkargs)
            [ -n "${VALUE+x}" ] || return 1 # must have a value (possibly empty)
        ;;
        keyscript)
            [ -n "${VALUE:+x}" ] || return 1 # must have a value
            if [ "${VALUE#/}" = "$VALUE" ]; then
                VALUE="/lib/cryptsetup/scripts/$VALUE"
            fi
            if [ ! -x "$VALUE" ] || [ ! -f "$VALUE" ]; then
                return 1
            fi
        ;;
        # and now the flags
        verify) ;;
        loud) ;;
        quiet) ;;
        initramfs) ;;
        noearly) ;;
        noauto) ;;
        readonly) ;;
        discard) ;;
        plain) ;;
        luks) ;;
        swap) ;;
        tcrypt) ;;
        veracrypt) ;;
        tcrypthidden) ;;
        *)
            if [ "${quiet:-n}" = "n" ]; then
                cryptsetup_message "WARNING: $CRYPTTAB_NAME: ignoring unknown option '$o'";
            fi
            unset -v OPTION
        ;;
    esac
}

# crypttab_resolve_source()
#   Resolve the CRYPTTAB_SOURCE variable, containing value of the second
#   field of a crypttab(5)-like file.
#   On error (non-existing source), CRYPTTAB_SOURCE is not changed and 1
#   is returned.
crypttab_resolve_source() {
    # return immediately if source is a regular file
    [ ! -f "$CRYPTTAB_SOURCE" ] || return 0
    # otherwise resolve the block device specification
    local dev="$CRYPTTAB_SOURCE"
    dev="$(resolve_device_spec "$dev")" && CRYPTTAB_SOURCE="$dev" || return 1
}

# run_keyscript($keyscriptarg, $tried_count)
#   exec()'ute `$CRYPTTAB_OPTION_keyscript $keyscriptarg`.
#   If $CRYPTTAB_OPTION_keyscript is unset or null and $keyscriptarg is
#   "none" (meaning the passphrase is to be read interactively from the
#   console), use `/lib/cryptsetup/askpass` as keyscript with a suitable
#   prompt message.
#   Since the shell process is replaced with the $CRYPTTAB_OPTION_keyscript
#   program, run_keyscript() must be used on the left-hand side of a
#   pipe, or similar.
run_keyscript() {
    local keyscriptarg="$1" CRYPTTAB_TRIED="$2" keyscript;
    export CRYPTTAB_NAME CRYPTTAB_SOURCE CRYPTTAB_OPTIONS
    export CRYPTTAB_TRIED

    if [ -n "${CRYPTTAB_OPTION_keyscript+x}" ] && \
            [ "$CRYPTTAB_OPTION_keyscript" != "/lib/cryptsetup/askpass" ]; then
        # 'keyscript' option is present: export its argument as
        # $CRYPTTAB_KEY
        export CRYPTTAB_KEY="$keyscriptarg"
        keyscript="$CRYPTTAB_OPTION_keyscript"
    elif [ "$keyscriptarg" = none ]; then
        # don't export the prompt message as CRYPTTAB_KEY
        keyscript="/lib/cryptsetup/askpass"
        keyscriptarg="Please unlock disk $CRYPTTAB_NAME: "
    fi

    exec "$keyscript" "$keyscriptarg"
}

# get_crypt_type()
#    Return the mapping's type, depending on its
#    $CRYPTTAB_OPTION_<option> values
get_crypt_type() {
    local type="plain" # assume plain dm-crypt device by default
    if [ "${CRYPTTAB_OPTION_tcrypt-}" = "yes" ]; then
        type="tcrypt"
    elif [ "${CRYPTTAB_OPTION_plain-}" = "yes" ]; then
        type="plain"
    elif [ "${CRYPTTAB_OPTION_luks-}" = "yes" ] ||
            /sbin/cryptsetup isLuks -- "${CRYPTTAB_OPTION_header-$CRYPTTAB_SOURCE}"; then
        type="luks"
    fi
    echo "$type"
}

# unlock_mapping([$keyfile])
#   Run cryptsetup(8) with suitable options and arguments to unlock
#   $CRYPTTAB_SOURCE and setup dm-crypt managed device-mapper mapping
#   $CRYPTTAB_NAME.
unlock_mapping() {
    local keyfile="${1:--}"

    if [ -n "${CRYPTTAB_OPTION_header+x}" ] && [ ! -f "$CRYPTTAB_OPTION_header" ]; then
        cryptsetup_message "ERROR: $CRYPTTAB_NAME: LUKS header '$CRYPTTAB_OPTION_header' missing"
        return 1
    fi

    local type="$(get_crypt_type)"
    if [ "$type" = "luks" ] || [ "$type" = "tcrypt" ]; then
        # ignored for LUKS and TCRYPT devices
        unset -v CRYPTTAB_OPTION_cipher \
                 CRYPTTAB_OPTION_size \
                 CRYPTTAB_OPTION_hash \
                 CRYPTTAB_OPTION_offset \
                 CRYPTTAB_OPTION_skip
    fi
    if [ "$type" = "plain" ] || [ "$type" = "tcrypt" ]; then
        unset -v CRYPTTAB_OPTION_keyfile_size
    fi
    if [ "$type" = "tcrypt" ]; then
        # ignored for TCRYPT devices
        unset -v CRYPTTAB_OPTION_keyfile_offset
    else
        # ignored for non-TCRYPT devices
        unset -v CRYPTTAB_OPTION_veracrypt CRYPTTAB_OPTION_tcrypthidden
    fi

    if [ "$type" != "luks" ]; then
        # ignored for non-LUKS devices
        unset -v CRYPTTAB_OPTION_keyslot
    fi

    /sbin/cryptsetup -T1 \
        ${CRYPTTAB_OPTION_header:+--header="$CRYPTTAB_OPTION_header"} \
        ${CRYPTTAB_OPTION_cipher:+--cipher="$CRYPTTAB_OPTION_cipher"} \
        ${CRYPTTAB_OPTION_size:+--key-size="$CRYPTTAB_OPTION_size"} \
        ${CRYPTTAB_OPTION_hash:+--hash="$CRYPTTAB_OPTION_hash"} \
        ${CRYPTTAB_OPTION_offset:+--offset="$CRYPTTAB_OPTION_offset"} \
        ${CRYPTTAB_OPTION_skip:+--skip="$CRYPTTAB_OPTION_skip"} \
        ${CRYPTTAB_OPTION_verify:+--verify-passphrase} \
        ${CRYPTTAB_OPTION_readonly:+--readonly} \
        ${CRYPTTAB_OPTION_discard:+--allow-discards} \
        ${CRYPTTAB_OPTION_veracrypt:+--veracrypt} \
        ${CRYPTTAB_OPTION_keyslot:+--key-slot="$CRYPTTAB_OPTION_keyslot"} \
        ${CRYPTTAB_OPTION_tcrypthidden:+--tcrypt-hidden} \
        ${CRYPTTAB_OPTION_keyfile_size:+--keyfile-size="$CRYPTTAB_OPTION_keyfile_size"} \
        ${CRYPTTAB_OPTION_keyfile_offset:+--keyfile-offset="$CRYPTTAB_OPTION_keyfile_offset"} \
        --type="$type" --key-file="$keyfile" \
        open -- "$CRYPTTAB_SOURCE" "$CRYPTTAB_NAME"
}

# crypttab_key_check()
#   Sanity checks for keyfile $CRYPTTAB_KEY.  CRYPTTAB_NAME and
#   CRYPTTAB_OPTION_<option> must be set appropriately.
crypttab_key_check() {
    if [ ! -f "$CRYPTTAB_KEY" ] && [ ! -b "$CRYPTTAB_KEY" ] && [ ! -c "$CRYPTTAB_KEY" ] ; then
        cryptsetup_message "WARNING: $CRYPTTAB_NAME: keyfile '$CRYPTTAB_KEY' not found"
        return 0
    fi

    if [ "$CRYPTTAB_KEY" = "/dev/random" ] || [ "$CRYPTTAB_KEY" = "/dev/urandom" ]; then
        if [ -n "${CRYPTTAB_OPTION_luks+x}" ] || [ -n "${CRYPTTAB_OPTION_tcrypt+x}" ]; then
            cryptsetup_message "WARNING: $CRYPTTAB_NAME: has random data as key"
            return 1
        else
            return 0
        fi
    fi

    local mode="$(stat -L -c"%04a" -- "$CRYPTTAB_KEY")"
    if [ $(stat -L -c"%u" -- "$CRYPTTAB_KEY") -ne 0 ] || [ "${mode%00}" = "$mode" ]; then
        cryptsetup_message "WARNING: $CRYPTTAB_NAME: key file $CRYPTTAB_KEY has" \
            "insecure ownership, see /usr/share/doc/cryptsetup/README.Debian."
    fi
}

# resolve_device_spec($spec)
#   Resolve LABEL=<label>, UUID=<uuid>, PARTUUID=<partuuid> and
#   PARTLABEL=<partlabel> to a block special device.  If $spec is
#   already a (link to a block special device) then it is echoed as is.
#   Return 1 if $spec doesn't correspond to a block special device.
resolve_device_spec() {
    local spec="$1"
    case "$spec" in
        UUID=*|LABEL=*|PARTUUID=*|PARTLABEL=*)
            # don't use /dev/disk/by-label/... to avoid gessing udev mangling
            spec="$(blkid -l -t "$spec" -o device)" || spec=
        ;;
    esac
    [ -b "$spec" ] && printf '%s\n' "$spec" || return 1
}

# dm_blkdevname($name)
#   Print the mapped device name, or return 1 if the the device doesn't exist.
dm_blkdevname() {
    local name="$1" dev
    # /dev/mapper/$name isn't reliable due to udev mangling
    if dev="$(dmsetup info -c --noheadings -o blkdevname -- "$name" 2>/dev/null)" &&
            [ -n "$dev" ] && [ -b "/dev/$dev" ]; then
        echo "/dev/$dev"
        return 0
    else
        return 1
    fi
}

# crypttab_find_entry([--quiet], $target)
#   Search in the crypttab(5) for the given $target, and sets the
#   variables CRYPTTAB_NAME, CRYPTTAB_SOURCE, CRYPTTAB_KEY and
#   CRYPTTAB_OPTIONS accordingly.  (In addition _CRYPTTAB_NAME,
#   _CRYPTTAB_SOURCE, _CRYPTTAB_KEY and _CRYPTTAB_OPTIONS are set to the
#   unmangled values before decoding the escape sequence.)  If there are
#   duplicates then only the first match is considered.
#   Return 0 if a match is found, and 1 otherwise.
crypttab_find_entry() {
    local target="$1" quiet="n" IFS
    if [ "$target" = "--quiet" ] && [ $# -eq 2 ]; then
        quiet="y"
        target="$2"
    fi

    if [ -f "$TABFILE" ]; then
        while IFS="     " read -r _CRYPTTAB_NAME _CRYPTTAB_SOURCE _CRYPTTAB_KEY _CRYPTTAB_OPTIONS; do
            if [ "${_CRYPTTAB_NAME#\#}" != "$_CRYPTTAB_NAME" ] || [ -z "$_CRYPTTAB_NAME" ]; then
                # ignore comments and empty lines
                continue
            fi

            # unmangle names
            CRYPTTAB_NAME="$(printf '%b' "$_CRYPTTAB_NAME")"
            if [ -z "$_CRYPTTAB_SOURCE" ] || [ -z "$_CRYPTTAB_KEY" ] || [ -z "$_CRYPTTAB_OPTIONS" ]; then
                cryptsetup_message "WARNING: '$CRYPTTAB_NAME' is missing some arguments, see crypttab(5)"
                continue
            elif [ "$CRYPTTAB_NAME" = "$target" ]; then
                CRYPTTAB_SOURCE="$( printf '%b' "$_CRYPTTAB_SOURCE" )"
                CRYPTTAB_KEY="$(    printf '%b' "$_CRYPTTAB_KEY"    )"
                CRYPTTAB_OPTIONS="$(printf '%b' "$_CRYPTTAB_OPTIONS")"
                return 0
            fi
        done <"$TABFILE"
    fi

    if [ "$quiet" = "n" ]; then
        cryptsetup_message "WARNING: target '$target' not found in $TABFILE"
    fi
    return 1
}

# crypttab_foreach_entry($callback)
#   Iterate through the crypttab(5) and run the given $callback for each
#   entry found. Variables CRYPTTAB_NAME, CRYPTTAB_SOURCE, CRYPTTAB_KEY
#   and CRYPTTAB_OPTIONS are set accordingly and available to the
#   $callback.  (In addition _CRYPTTAB_NAME, _CRYPTTAB_SOURCE,
#   _CRYPTTAB_KEY and _CRYPTTAB_OPTIONS are set to the unmangled values
#   before decoding the escape sequence.)
#   Return 0 if a match is found, and 1 otherwise.
crypttab_foreach_entry() {
    local callback="$1" IFS
    local _CRYPTTAB_NAME _CRYPTTAB_SOURCE _CRYPTTAB_KEY _CRYPTTAB_OPTIONS \
           CRYPTTAB_NAME  CRYPTTAB_SOURCE  CRYPTTAB_KEY  CRYPTTAB_OPTIONS

    [ -f "$TABFILE" ] || return
    while IFS="     " read -r _CRYPTTAB_NAME _CRYPTTAB_SOURCE _CRYPTTAB_KEY _CRYPTTAB_OPTIONS <&9; do
        if [ "${_CRYPTTAB_NAME#\#}" != "$_CRYPTTAB_NAME" ] || [ -z "$_CRYPTTAB_NAME" ]; then
            # ignore comments and empty lines
            continue
        fi

        # unmangle names
        CRYPTTAB_NAME="$(   printf '%b' "$_CRYPTTAB_NAME"   )"
        CRYPTTAB_SOURCE="$( printf '%b' "$_CRYPTTAB_SOURCE" )"
        CRYPTTAB_KEY="$(    printf '%b' "$_CRYPTTAB_KEY"    )"
        CRYPTTAB_OPTIONS="$(printf '%b' "$_CRYPTTAB_OPTIONS")"

        if [ -z "$CRYPTTAB_SOURCE" ] || [ -z "$CRYPTTAB_KEY" ] || [ -z "$CRYPTTAB_OPTIONS" ]; then
            cryptsetup_message "WARNING: '$CRYPTTAB_NAME' is missing some arguments, see crypttab(5)"
            continue
        fi

        "$callback" 9<&-
    done 9<"$TABFILE"
}

# vim: set filetype=sh :

Niols commented 5 years ago

OK, got it: it's the token ${#*}. We should check if it's in the POSIX standard, but it's accepted by both bash and dash.

treinen commented 5 years ago

What is the meaning of ${#*} in dash or bash? I can't find it in the bash manual, and in the Posix standard neither.

Niols commented 5 years ago

In 2.6.2 Parameter Expansion:

${#parameter} String Length. The length in characters of the value of parameter shall be substituted. If parameter is * or @, the result of the expansion is unspecified. If parameter is unset and set -u is in effect, the expansion shall fail.

Although the result is unspecified, this is syntactically correct. And it could probably mean "the length of what $* would expand to".

treinen commented 5 years ago

OK, so morbig should probably accept it. However, I would say that ${#*} should not be used in a /bin/sh maintainer script when the behaviour is unspecified.

That is which package ?

We should add a check for this in lintshell.

Niols commented 5 years ago

In the included file lib/cryptsetup/functions

yurug commented 5 years ago

Closed by #66

colis-anr / morbig

${#*} → Failure: lexing: empty token. #57