Installer fixes for OpenBSD 5.5

[carletes]

On OpenBSD 5.5, the installer asks for the time zone after the software sets have been installed.

[tmatilai]

Thanks, interesting. I didn't spot this, but not even sure if I run the installer with GUI on. @zigg, any comments? Not sure when I get to verify this.

[mattieb]

I believe I can take a look in the next few days. @silentbicycle mentioned something to me in passing about this as well, but I hadn't heard details since.

[silentbicycle]

I've been meaning to look at the OpenBSD installer source, since that should be authoritative, but I haven't got to it yet. It seems like there may be some variation in when / how it prompts about the time zone, but I don't get why it would be nondeterministic.

[mattieb]

It might be NTP-related, actually.

[carletes]

@silentbicycle, @zigg: It looks like the time when the time zone question is asked depends on whether a certain time zone list file is available, and on whther some HTTP- or CGI-related environment variables are set:

From the installation script src/distrib/miniroot/install.sh:

# [..]
. install.sub
# [..]
install_sets

# If we did not succeed at setting TZ yet, we try again
# using the timezone names extracted from the base set
if [[ -z $TZ ]]; then
    (cd /mnt/usr/share/zoneinfo
        ls -1dF $(tar cvf /dev/null [A-Za-y]*) >/mnt/tmp/tzlist )
    echo
    set_timezone /mnt/tmp/tzlist
# [..]

The installation script begins by including install.sub. If the time zone has not been set after installing the basic sets, the function set_timezone is called (this is the behaviour I see). That function is defined in src/distrib/miniroot/install.sub:

. install.md
# [..]
ask_yn "Use DUIDs rather than device names in fstab?" yes && FSTABFLAG=-F
# [..]
set_timezone() {
    local _zonefile=$1 _zonepath _zsed _tz _zoneroot=/usr/share/zoneinfo

    # If the timezone file is not available,
    # return immediately.

    [[ ! -f $_zonefile ]] && return

    # If configured in a previous call, return immediately
    [[ -n $TZ ]] && return

    if [[ -h /mnt/etc/localtime ]]; then
        TZ=$(ls -l /mnt/etc/localtime 2>/dev/null)
        TZ=${TZ#*${_zoneroot#/mnt}/}
    fi

    waitcgiinfo
    if [[ -n $CGI_TZ ]]; then
        _tz=$CGI_TZ
        [[ -n $_tz ]] && isin "$_tz" `cat $_zonefile` && TZ=$_tz
    fi

    # If neither the base or HTTP_LIST gave a hint, and this is the
    # early question, give up, and ask after the sets are installed
    [[ $_zonefile = /var/tzlist && -z $TZ ]] && return

    while :; do
        ask "What timezone are you in? ('?' for list)" "$TZ"
        _zonepath=${resp%%*(/)}
        case $_zonepath in
        "") continue;;
        "?")    grep -v /. $_zonefile | showcols
            continue;;
        esac

        while isin "$_zonepath/" $(cat $_zonefile); do
            ask "What sub-timezone of '$_zonepath' are you in? ('?' for list)"
            _zsed=$(echo $_zonepath/ | sed 's,/,\\/,g')
            resp=${resp%%*(/)}
            case $resp in
            "") ;;
            "?")    sed -n "/^$_zsed/{s/$_zsed//;/\/./!p;}" $_zonefile | showcols;;
            *)  _zonepath=$_zonepath/$resp;;
            esac
        done

        if isin "$_zonepath" $(cat $_zonefile); then
            TZ=${_zonepath#$_zoneroot}
            return
        fi

        echo -n "'${_zonepath}'"
        echo " is not a valid timezone on this system."
    done
}

# [..]

if [[ $MODE == install ]]; then
    ask_until "System hostname? (short form, e.g. 'foo')" "$(hostname -s)"
    [[ ${resp%%.*} != $(hostname -s) ]] && hostname $resp
    THESETS="$THESETS site$VERSION-$(hostname -s).tgz"

    echo
    donetconfig

    ((NIFS != 0)) && startcgiinfo

    echo
    while :; do
        askpassword "Password for root account?"
        _rootpass="$_password"
        [[ -n "$_password" ]] && break
        echo "The root password must be set."
    done

    rootkey=
    $auto && ask "Public ssh key for root account?" none &&
        [[ $resp != none ]] && rootkey=$resp

    questions
    user_setup

    set_timezone /var/tzlist
    echo
fi

So it seems that set_timezone is called for the first time right after setting up users, with /var/tzlist as an argument. That call may return without settings the time zone (the behaviour I see) if:

• The file (/var/tzlist) does not exist, or
• The variable $TZ is set already, or
• The file (/var/tzlist) exists, it is equal to /var/tzlist and:
  • The file /mnt/etc/localtime does not exist, and
  • The variable $CGI_TZ is not set

The second time it gets called (after the base sets have been installed), it is called like this:

set_timezone /mnt/tmp/tzlist

This time, given the conditions above, the time zone is successfuly set.

[tmatilai]

Sorry folks, I'm a bit lost should we merge this or not. Or are there better ways to ensure the correct behavior?

[mattieb]

CGI_TZ is guessed by the project's ftplist.cgi script. For example, my output for snapshots/i386 looks like this:

http://ftp5.usa.openbsd.org/pub/OpenBSD                 Redwood City, CA, USA
http://ftp3.usa.openbsd.org/pub/OpenBSD                      Boulder, CO, USA
http://mirror.team-cymru.org/pub/OpenBSD                     Chicago, IL, USA
…
TZ=US/Eastern
NSA_ID=0x978d44093f302eb56faa8d3063006a4c376f9ac7c2520a560471d3cd24217fc2
TIME=1413163471
RND_BYTES=0x4529d247145e69a02687a3e76dec2375829745cb1ee968c0d53050b21f13d9c81ac5f2eba7426fc22f78a35e6fa34ca946b704e00b2e6d04001e488d1173131a

(There's a fun discussion about some of this here.)

Thinking on this, I would prefer to address the issue by blocking network access to the box during the installation process, ensuring a consistent build environment. I'm not 100% sure how or if I can do this with Packer, though.

[mattieb]

Based on #6, I started a new branch based on autoinstall(8) support. This should resolve this issue here—I tested lightly by switching a VM to a non-Internet-capable network—as well as work more quickly and be more repeatable overall. I'd appreciate testers!

https://github.com/tmatilai/packer-openbsd/tree/openbsd56-autoinstall

[mattieb]

Hey @carletes, sorry to bother you, but have you had a chance to look at my branch? I think it might solve this for you. I'd love to hear if it does.

[mattieb]

I forward-ported the proxy settings from this PR in 7fdb195920c7c7e2c1159c0270dd070d8a6ede74. The rest was replaced by the work from the autoinstall branch, which I'm confident works in any network situation (though, of course, packages aren't fetchable without a package source.)

[carletes]

Thanks, @zigg!