search

freedomofpress / securedrop

GitHub repository for the SecureDrop whistleblower platform. Do not submit tips here!
https://securedrop.org/
Other
3.62k stars 686 forks source link

Release SecureDrop 2.1.0 #6103

Closed eloquence closed 2 years ago

eloquence commented 3 years ago

This is a tracking issue for the release of SecureDrop 2.1.0

Scheduled as follows:

Feature / string freeze: 2021-09-28 Pre-release announcement: 2021-10-12 Release date: 2021-10-19 Release manager: @zenmonkeykstop Deputy release manager: @conorsch Communications manager: @eloquence Localization manager: @conorsch Deputy LM: @cfm [tentative]

QA team: @creviera @tesitura @cfm @conorsch @zenmonkeykstop

SecureDrop maintainers and testers: As you QA 2.1.0, please report back your testing results as comments on this ticket. File GitHub issues for any problems found, tag them "QA: Release", and associate them with the 2.1.0 milestone for tracking (or ask a maintainer to do so).

Test debian packages will be posted on https://apt-test.freedom.press signed with the test key

QA Matrix for 2.1.0

Test Plan for 2.1.0

Prepare release candidate (2.1.0~rc1)

Prepare release candidate (2.1.0~rc2)

After each test, please update the QA matrix and post details for Basic Server Testing, Application Acceptance Testing and release-specific testing below in comments to this ticket.

Final release

Post release

eloquence commented 3 years ago

Draft release comms are ready for initial review. As always, it's a bit of a judgment call which changes should go into the blog post and which ones are only in the changelog -- if there's stuff you feel warrants more or less visibility, please don't hesitate to comment.

cfm commented 2 years ago

NB. An upgrade scenario newly provisioned from a tag (i.e., without #6120) will fail in securedrop-admin install role install-fpf-repo with apt cache update failed until both VMs are brought up to date via (e.g.):

root@sd-staging:~/securedrop# molecule login -s libvirt-prod-focal -h app-prod
vagrant@app-prod:~$ sudo apt-get update
vagrant@app-prod:~$ sudo apt-get upgrade -y
vagrant@app-prod:~$ exit
root@sd-staging:~/securedrop# molecule login -s libvirt-prod-focal -h mon-prod
vagrant@mon-prod:~$ sudo apt-get update
vagrant@mon-prod:~$ sudo apt-get upgrade -y
vagrant@mon-prod:~$ exit
zenmonkeykstop commented 2 years ago

Yup, that's the same root cause as captured in #6119. If you have more up-to-date bento/20.04 boxes it shouldn't be a problem, but it doesn't look like those are available yet in a provider format that we can use.

cfm commented 2 years ago

Summary

Test plan and results

Environment

  1. [x] Take a backup of your existing SecureDrop installation using the securedrop-admin backup command. You will need enough free space on your Admin Workstation USB to complete this backup.

  2. [x] ~Either~ run the ansible QA playbook from qa-update-playbook (pending #6123):
    https://github.com/freedomofpress/securedrop/blob/ed008a0e49738ab8695fe438255b9ee06362831f/install_files/ansible-base/securedrop-qa.yml#L7-L18

[...]

  1. [x] Complete steps 6 and 7 of clean install QA instructions (kernel testing, QA Matrix and Github release ticket testing).

Basic Server Testing

Tests failing: (see also: #6127)

Command Line User Generation

Administration

Application Acceptance Testing

Source Interface

Landing page base cases
First submission base cases
Returning source base cases

Journalist Interface

Login base cases
Index base cases
Individual source page

Basic Tails Testing

After updating to this release candidate and running securedrop-admin tailsconfig

2.1.0 release-specific changes

Web Applications

Tor Browser "Page Info" says (my emphasis):

Connection Encrypted (Onion Service, TLS_AES_128GCM_SHA256, 128 bit keys, TLS 1.3)

After rerunning make self-signed-https-certs && ./securedrop-admin install, Tor Browser "Page Info" says (my emphasis):

Connection Encrypted (Onion Service, TLS_AES_256GCM_SHA384, 256 bit keys, TLS 1.3)

This suggests that an explicit ./securedrop-admin install is necessary for #5988 to go into effect on upgrading an existing SecureDrop installation.

amnesia@amnesia:~/Persistent/securedrop$ git branch --points-at HEAD
* release/2.1.0
amnesia@amnesia:~/Persistent/securedrop$ ssh app "grep 'SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1 -TLSv1.2' /etc/apache2/sites-enabled/source.conf"
SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1 -TLSv1.2
amnesia@amnesia:~/Persistent/securedrop$ git branch --points-at HEAD
* release/2.1.0
amnesia@amnesia:~/Persistent/securedrop$ for s in app mon; do ssh $s sudo apt-key list | grep -b2 -a1 "2224"; done
Warning: apt-key output should not be parsed (stdout is not a terminal)
587-pub   rsa4096 2016-10-20 [SC] [expired: 2021-06-30]
639:      2224 5C81 E3BA EB41 38B3  6061 310F 5612 00F4 AD77
696-uid           [ expired] SecureDrop Release Signing Key
Warning: apt-key output should not be parsed (stdout is not a terminal)
587-pub   rsa4096 2016-10-20 [SC] [expired: 2021-06-30]
639:      2224 5C81 E3BA EB41 38B3  6061 310F 5612 00F4 AD77
696-uid           [ expired] SecureDrop Release Signing Key
amnesia@amnesia:~/Persistent/securedrop$ git branch --points-at HEAD
* release/2.1.0
amnesia@amnesia:~/Persistent/securedrop$ for s in app mon; do ssh $s sudo apt-key list | grep -b2 -a1 "2359"; done
Warning: apt-key output should not be parsed (stdout is not a terminal)
326-pub   rsa4096 2021-05-10 [SC] [expires: 2022-07-04]
378:      2359 E653 8C06 13E6 5295  5E6C 188E DD3B 7B22 E6A3
435-uid           [ unknown] SecureDrop Release Signing Key <securedrop-release-key-2021@freedom.press>
Warning: apt-key output should not be parsed (stdout is not a terminal)
326-pub   rsa4096 2021-05-10 [SC] [expires: 2022-07-04]
378:      2359 E653 8C06 13E6 5295  5E6C 188E DD3B 7B22 E6A3
435-uid           [ unknown] SecureDrop Release Signing Key <securedrop-release-key-2021@freedom.press>
amnesia@amnesia:~/Persistent/securedrop$ git branch --points-at HEAD
* release/2.1.0
amnesia@amnesia:~/Persistent/securedrop$ ssh mon sudo grep -R "fwupd" /var/ossec/logs/alerts | grep -v "grep" | wc -l
0
amnesia@amnesia:~/Persistent/securedrop$ git branch --points-at HEAD
* release/2.1.0
amnesia@amnesia:~/Persistent/securedrop$ scp install_files/ansible-base/sd-backup-2021-10-08* app:/tmp/sd-backup-2021-10-08.tar.gz
sd-backup-2021-10-08--01-23-08.tar.gz         100%  107KB  72.1KB/s   00:01
amnesia@amnesia:~/Persistent/securedrop$ ./securedrop-admin --force restore --no-transfer sd-backup-2021-10-08.tar.gz
[...]
TASK [restore : Extract Tor configuration from backup] *************************
fatal: [app]: FAILED! => {
    "changed": false
}

MSG:

Source '/home/amnesia/Persistent/securedrop/install_files/ansible-base/sd-backup-2021-10-08.tar.gz' does not exist
[...]
amnesia@amnesia:~/Persistent/securedrop$ scp install_files/ansible-base/sd-backup-2021-10-08* app:/tmp/
sd-backup-2021-10-08--01-23-08.tar.gz         100%  107KB  93.4KB/s   00:01
amnesia@amnesia:~/Persistent/securedrop$ ./securedrop-admin --force restore --no-transfer sd-backup-2021-10-08--01-23-08.tar.gz
[...]
PLAY RECAP *********************************************************************
app                        : ok=17   changed=12   unreachable=0    failed=0    skipped=13   rescued=0    ignored=0

Even with --no-transfer, restore_file needs to exist in install_files/ansible-base—which might be worth documenting further as the intended behavior of #5909.

conorsch commented 2 years ago

Environment

Basic Server Testing

Command Line User Generation

Administration

(Not tested)

Application Acceptance Testing

Source Interface

Landing page base cases
First submission base cases
Returning source base cases

Journalist Interface

Login base cases
Index base cases

N.B. If you previously used "Safest" mode in Tor Browser as a Source, you'll have to re-enable JS to verify some of the functionality below.

Individual source page

2.1.0 release-specific changes

Web Applications

zenmonkeykstop commented 2 years ago

@cfm: re the manual transfer test, the playbook does require that the tarball be available locally as well to verify that it is valid - good catch on the docs side, this option hasn't been documented yet and should be for the release.

conorsch commented 2 years ago

Updated the OP with mention of 2.1.0~rc2. @cfm if you've got cycles today, simply re-testing the problematic sections of your previous testing report on rc1 would be ideal. For simplicity's sake, I'd say take the VM upgrade scenario again, and I'll take clean install VMs again and post results.

conorsch commented 2 years ago

So far so good on 2.1.0~rc2. One issue of note is that on the clean install scenario, I observed an apt-update failure:

sd-qa-2 1 0-rc2-apt-failure-1

It appears the order of operations is as follows:

  1. All apt packages updated in prepare-servers, as expected
  2. apt-test key is added via local modifications of the install-fpf-repo vars, as part of QA
  3. apt-test repo URL is added via local modifications of the install-fpf-repo vars, as part of QA
  4. the securedrop-keyring package is installed from apt-test, which clobbers the test key with the prod key upon installation
  5. the common role fails to update apt lists again, since the apt-test repo is configured but the key is removed

This issue only affects QA testing, it isn't a problem for prod. But I'm documenting it here in case other testers encounter it.

cfm commented 2 years ago

All issues raised about 2.1.0\~rc1 in https://github.com/freedomofpress/securedrop/issues/6103#issuecomment-938341940 are resolved in 2.1.0\~rc2. Still outstanding against 2.1.0\~rc2:

amnesia@amnesia:~/Persistent/securedrop$ ./securedrop-admin verify ```sh-session =========================== short test summary info ============================ FAILED app/test_apparmor.py::test_apparmor_pkg[paramiko:/app-apparmor-utils] FAILED app/test_apparmor.py::test_apparmor_pkg[paramiko:/app-apparmor] - para... FAILED app/test_apparmor.py::test_apparmor_apache_capabilities[paramiko:/app-dac_override] FAILED app/test_ossec_agent.py::test_hosts_files[paramiko:/app] - AssertionEr... FAILED common/test_fpf_apt_repo.py::test_fpf_apt_repo_present[paramiko:/app] FAILED common/test_fpf_apt_repo.py::test_fpf_apt_repo_present[paramiko:/mon] FAILED common/test_automatic_updates.py::test_unattended_upgrades_functional[paramiko:/app] FAILED mon/test_ossec_server.py::test_ossec_connectivity[paramiko:/mon] - Ass... FAILED mon/test_ossec_server.py::test_hosts_files[paramiko:/mon] - AssertionE... FAILED mon/test_postfix.py::test_postfix_generic_maps[paramiko:/mon] - Assert... = 10 failed, 425 passed, 7 skipped, 3 xfailed, 1 xpassed, 10 warnings in 2016.78s (0:33:36) = [...] ``` ```sh-session =================================== FAILURES =================================== _______________ test_apparmor_pkg[paramiko://app-apparmor-utils] _______________ [gw1] linux -- Python 3.7.3 /home/amnesia/Persistent/securedrop/admin/.venv3/bin/python3 self = command = b'uname -s', args = (), kwargs = {} def run(self, command, *args, **kwargs): command = self.get_command(command, *args) command = self.encode(command) try: > rc, stdout, stderr = self._exec_command(command) ../../admin/.venv3/lib/python3.7/site-packages/testinfra/backend/paramiko.py:115: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = command = b'uname -s' def _exec_command(self, command): > chan = self.client.get_transport().open_session() ../../admin/.venv3/lib/python3.7/site-packages/testinfra/backend/paramiko.py:102: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = obj = cls = def __get__(self, obj, cls): if obj is None: return self > value = obj.__dict__[self.func.__name__] = self.func(obj) ../../admin/.venv3/lib/python3.7/site-packages/testinfra/utils/__init__.py:29: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = @cached_property def client(self): client = paramiko.SSHClient() client.set_missing_host_key_policy(paramiko.WarningPolicy()) cfg = { "hostname": self.host.name, "port": int(self.host.port) if self.host.port else 22, "username": self.host.user, "timeout": self.timeout, } if self.ssh_config: with open(self.ssh_config) as f: ssh_config = paramiko.SSHConfig() ssh_config.parse(f) self._load_ssh_config(client, cfg, ssh_config) else: # fallback reading ~/.ssh/config default_ssh_config = os.path.join( os.path.expanduser('~'), '.ssh', 'config') try: with open(default_ssh_config) as f: ssh_config = paramiko.SSHConfig() ssh_config.parse(f) except IOError: pass else: self._load_ssh_config(client, cfg, ssh_config) if self.ssh_identity_file: cfg["key_filename"] = self.ssh_identity_file > client.connect(**cfg) ../../admin/.venv3/lib/python3.7/site-packages/testinfra/backend/paramiko.py:98: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = hostname = 'sypudnu7biyq3cxacrgohjwpveeabi7inrtxop2543674siadyxhrqqd.onion' port = 22, username = 'vagrant', password = None, pkey = None key_filename = None, timeout = 10, allow_agent = True, look_for_keys = True compress = False, sock = gss_auth = False, gss_kex = False, gss_deleg_creds = True, gss_host = None banner_timeout = None, auth_timeout = None, gss_trust_dns = True passphrase = None, disabled_algorithms = None def connect( self, hostname, port=SSH_PORT, username=None, password=None, pkey=None, key_filename=None, timeout=None, allow_agent=True, look_for_keys=True, compress=False, sock=None, gss_auth=False, gss_kex=False, gss_deleg_creds=True, gss_host=None, banner_timeout=None, auth_timeout=None, gss_trust_dns=True, passphrase=None, disabled_algorithms=None, ): """ Connect to an SSH server and authenticate to it. The server's host key is checked against the system host keys (see `load_system_host_keys`) and any local host keys (`load_host_keys`). If the server's hostname is not found in either set of host keys, the missing host key policy is used (see `set_missing_host_key_policy`). The default policy is to reject the key and raise an `.SSHException`. Authentication is attempted in the following order of priority: - The ``pkey`` or ``key_filename`` passed in (if any) - ``key_filename`` may contain OpenSSH public certificate paths as well as regular private-key paths; when files ending in ``-cert.pub`` are found, they are assumed to match a private key, and both components will be loaded. (The private key itself does *not* need to be listed in ``key_filename`` for this to occur - *just* the certificate.) - Any key we can find through an SSH agent - Any "id_rsa", "id_dsa" or "id_ecdsa" key discoverable in ``~/.ssh/`` - When OpenSSH-style public certificates exist that match an existing such private key (so e.g. one has ``id_rsa`` and ``id_rsa-cert.pub``) the certificate will be loaded alongside the private key and used for authentication. - Plain username/password auth, if a password was given If a private key requires a password to unlock it, and a password is passed in, that password will be used to attempt to unlock the key. :param str hostname: the server to connect to :param int port: the server port to connect to :param str username: the username to authenticate as (defaults to the current local username) :param str password: Used for password authentication; is also used for private key decryption if ``passphrase`` is not given. :param str passphrase: Used for decrypting private keys. :param .PKey pkey: an optional private key to use for authentication :param str key_filename: the filename, or list of filenames, of optional private key(s) and/or certs to try for authentication :param float timeout: an optional timeout (in seconds) for the TCP connect :param bool allow_agent: set to False to disable connecting to the SSH agent :param bool look_for_keys: set to False to disable searching for discoverable private key files in ``~/.ssh/`` :param bool compress: set to True to turn on compression :param socket sock: an open socket or socket-like object (such as a `.Channel`) to use for communication to the target host :param bool gss_auth: ``True`` if you want to use GSS-API authentication :param bool gss_kex: Perform GSS-API Key Exchange and user authentication :param bool gss_deleg_creds: Delegate GSS-API client credentials or not :param str gss_host: The targets name in the kerberos database. default: hostname :param bool gss_trust_dns: Indicates whether or not the DNS is trusted to securely canonicalize the name of the host being connected to (default ``True``). :param float banner_timeout: an optional timeout (in seconds) to wait for the SSH banner to be presented. :param float auth_timeout: an optional timeout (in seconds) to wait for an authentication response. :param dict disabled_algorithms: an optional dict passed directly to `.Transport` and its keyword argument of the same name. :raises: `.BadHostKeyException` -- if the server's host key could not be verified :raises: `.AuthenticationException` -- if authentication failed :raises: `.SSHException` -- if there was any other error connecting or establishing an SSH session :raises socket.error: if a socket error occurred while connecting .. versionchanged:: 1.15 Added the ``banner_timeout``, ``gss_auth``, ``gss_kex``, ``gss_deleg_creds`` and ``gss_host`` arguments. .. versionchanged:: 2.3 Added the ``gss_trust_dns`` argument. .. versionchanged:: 2.4 Added the ``passphrase`` argument. .. versionchanged:: 2.6 Added the ``disabled_algorithms`` argument. """ if not sock: errors = {} # Try multiple possible address families (e.g. IPv4 vs IPv6) to_try = list(self._families_and_addresses(hostname, port)) for af, addr in to_try: try: sock = socket.socket(af, socket.SOCK_STREAM) if timeout is not None: try: sock.settimeout(timeout) except: pass retry_on_signal(lambda: sock.connect(addr)) # Break out of the loop on success break except socket.error as e: # Raise anything that isn't a straight up connection error # (such as a resolution error) if e.errno not in (ECONNREFUSED, EHOSTUNREACH): raise # Capture anything else so we know how the run looks once # iteration is complete. Retain info about which attempt # this was. errors[addr] = e # Make sure we explode usefully if no address family attempts # succeeded. We've no way of knowing which error is the "right" # one, so we construct a hybrid exception containing all the real # ones, of a subclass that client code should still be watching for # (socket.error) if len(errors) == len(to_try): raise NoValidConnectionsError(errors) t = self._transport = Transport( sock, gss_kex=gss_kex, gss_deleg_creds=gss_deleg_creds, disabled_algorithms=disabled_algorithms, ) t.use_compression(compress=compress) t.set_gss_host( # t.hostname may be None, but GSS-API requires a target name. # Therefore use hostname as fallback. gss_host=gss_host or hostname, trust_dns=gss_trust_dns, gssapi_requested=gss_auth or gss_kex, ) if self._log_channel is not None: t.set_log_channel(self._log_channel) if banner_timeout is not None: t.banner_timeout = banner_timeout if auth_timeout is not None: t.auth_timeout = auth_timeout if port == SSH_PORT: server_hostkey_name = hostname else: server_hostkey_name = "[{}]:{}".format(hostname, port) our_server_keys = None our_server_keys = self._system_host_keys.get(server_hostkey_name) if our_server_keys is None: our_server_keys = self._host_keys.get(server_hostkey_name) if our_server_keys is not None: keytype = our_server_keys.keys()[0] sec_opts = t.get_security_options() other_types = [x for x in sec_opts.key_types if x != keytype] sec_opts.key_types = [keytype] + other_types t.start_client(timeout=timeout) # If GSS-API Key Exchange is performed we are not required to check the # host key, because the host is authenticated via GSS-API / SSPI as # well as our client. if not self._transport.gss_kex_used: > server_key = t.get_remote_server_key() ../../admin/.venv3/lib/python3.7/site-packages/paramiko/client.py:412: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_remote_server_key(self): """ Return the host key of the server (in client mode). .. note:: Previously this call returned a tuple of ``(key type, key string)``. You can get the same effect by calling `.PKey.get_name` for the key type, and ``str(key)`` for the key string. :raises: `.SSHException` -- if no session is currently active. :return: public key (`.PKey`) of the remote server """ if (not self.active) or (not self.initial_kex_done): > raise SSHException("No existing session") E paramiko.ssh_exception.SSHException: No existing session ../../admin/.venv3/lib/python3.7/site-packages/paramiko/transport.py:834: SSHException During handling of the above exception, another exception occurred: host = , pkg = 'apparmor-utils' @pytest.mark.parametrize('pkg', ['apparmor', 'apparmor-utils']) def test_apparmor_pkg(host, pkg): """ Apparmor package dependencies """ > assert host.package(pkg).is_installed app/test_apparmor.py:13: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../../admin/.venv3/lib/python3.7/site-packages/testinfra/host.py:111: in __getattr__ obj = module_class.get_module(self) ../../admin/.venv3/lib/python3.7/site-packages/testinfra/modules/base.py:19: in get_module klass = cls.get_module_class(_host) ../../admin/.venv3/lib/python3.7/site-packages/testinfra/modules/package.py:65: in get_module_class if host.system_info.type == 'windows': ../../admin/.venv3/lib/python3.7/site-packages/testinfra/modules/systeminfo.py:144: in type return self.sysinfo["type"] ../../admin/.venv3/lib/python3.7/site-packages/testinfra/utils/__init__.py:29: in __get__ value = obj.__dict__[self.func.__name__] = self.func(obj) ../../admin/.venv3/lib/python3.7/site-packages/testinfra/modules/systeminfo.py:31: in sysinfo uname = self.run_expect([0, 1], 'uname -s') ../../admin/.venv3/lib/python3.7/site-packages/testinfra/host.py:75: in run return self.backend.run(command, *args, **kwargs) ../../admin/.venv3/lib/python3.7/site-packages/testinfra/backend/paramiko.py:117: in run if not self.client.get_transport().is_active(): ../../admin/.venv3/lib/python3.7/site-packages/testinfra/utils/__init__.py:29: in __get__ value = obj.__dict__[self.func.__name__] = self.func(obj) ../../admin/.venv3/lib/python3.7/site-packages/testinfra/backend/paramiko.py:98: in client client.connect(**cfg) ../../admin/.venv3/lib/python3.7/site-packages/paramiko/client.py:412: in connect server_key = t.get_remote_server_key() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_remote_server_key(self): """ Return the host key of the server (in client mode). .. note:: Previously this call returned a tuple of ``(key type, key string)``. You can get the same effect by calling `.PKey.get_name` for the key type, and ``str(key)`` for the key string. :raises: `.SSHException` -- if no session is currently active. :return: public key (`.PKey`) of the remote server """ if (not self.active) or (not self.initial_kex_done): > raise SSHException("No existing session") E paramiko.ssh_exception.SSHException: No existing session ../../admin/.venv3/lib/python3.7/site-packages/paramiko/transport.py:834: SSHException __________________ test_apparmor_pkg[paramiko://app-apparmor] __________________ [gw0] linux -- Python 3.7.3 /home/amnesia/Persistent/securedrop/admin/.venv3/bin/python3 host = , pkg = 'apparmor' @pytest.mark.parametrize('pkg', ['apparmor', 'apparmor-utils']) def test_apparmor_pkg(host, pkg): """ Apparmor package dependencies """ > assert host.package(pkg).is_installed app/test_apparmor.py:13: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../../admin/.venv3/lib/python3.7/site-packages/testinfra/host.py:111: in __getattr__ obj = module_class.get_module(self) ../../admin/.venv3/lib/python3.7/site-packages/testinfra/modules/base.py:19: in get_module klass = cls.get_module_class(_host) ../../admin/.venv3/lib/python3.7/site-packages/testinfra/modules/package.py:65: in get_module_class if host.system_info.type == 'windows': ../../admin/.venv3/lib/python3.7/site-packages/testinfra/modules/systeminfo.py:144: in type return self.sysinfo["type"] ../../admin/.venv3/lib/python3.7/site-packages/testinfra/utils/__init__.py:29: in __get__ value = obj.__dict__[self.func.__name__] = self.func(obj) ../../admin/.venv3/lib/python3.7/site-packages/testinfra/modules/systeminfo.py:31: in sysinfo uname = self.run_expect([0, 1], 'uname -s') ../../admin/.venv3/lib/python3.7/site-packages/testinfra/host.py:75: in run return self.backend.run(command, *args, **kwargs) ../../admin/.venv3/lib/python3.7/site-packages/testinfra/backend/paramiko.py:115: in run rc, stdout, stderr = self._exec_command(command) ../../admin/.venv3/lib/python3.7/site-packages/testinfra/backend/paramiko.py:102: in _exec_command chan = self.client.get_transport().open_session() ../../admin/.venv3/lib/python3.7/site-packages/testinfra/utils/__init__.py:29: in __get__ value = obj.__dict__[self.func.__name__] = self.func(obj) ../../admin/.venv3/lib/python3.7/site-packages/testinfra/backend/paramiko.py:98: in client client.connect(**cfg) ../../admin/.venv3/lib/python3.7/site-packages/paramiko/client.py:412: in connect server_key = t.get_remote_server_key() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_remote_server_key(self): """ Return the host key of the server (in client mode). .. note:: Previously this call returned a tuple of ``(key type, key string)``. You can get the same effect by calling `.PKey.get_name` for the key type, and ``str(key)`` for the key string. :raises: `.SSHException` -- if no session is currently active. :return: public key (`.PKey`) of the remote server """ if (not self.active) or (not self.initial_kex_done): > raise SSHException("No existing session") E paramiko.ssh_exception.SSHException: No existing session ../../admin/.venv3/lib/python3.7/site-packages/paramiko/transport.py:834: SSHException ________ test_apparmor_apache_capabilities[paramiko://app-dac_override] ________ [gw1] linux -- Python 3.7.3 /home/amnesia/Persistent/securedrop/admin/.venv3/bin/python3 host = , cap = 'dac_override' @pytest.mark.parametrize('cap', apache2_capabilities) def test_apparmor_apache_capabilities(host, cap): """ check for exact list of expected app-armor capabilities for apache2 """ c = host.run( > r"perl -nE '/^\s+capability\s+(\w+),$/ && say $1' /etc/apparmor.d/usr.sbin.apache2" ) app/test_apparmor.py:34: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../../admin/.venv3/lib/python3.7/site-packages/testinfra/host.py:75: in run return self.backend.run(command, *args, **kwargs) ../../admin/.venv3/lib/python3.7/site-packages/testinfra/backend/paramiko.py:115: in run rc, stdout, stderr = self._exec_command(command) ../../admin/.venv3/lib/python3.7/site-packages/testinfra/backend/paramiko.py:102: in _exec_command chan = self.client.get_transport().open_session() ../../admin/.venv3/lib/python3.7/site-packages/testinfra/utils/__init__.py:29: in __get__ value = obj.__dict__[self.func.__name__] = self.func(obj) ../../admin/.venv3/lib/python3.7/site-packages/testinfra/backend/paramiko.py:98: in client client.connect(**cfg) ../../admin/.venv3/lib/python3.7/site-packages/paramiko/client.py:412: in connect server_key = t.get_remote_server_key() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_remote_server_key(self): """ Return the host key of the server (in client mode). .. note:: Previously this call returned a tuple of ``(key type, key string)``. You can get the same effect by calling `.PKey.get_name` for the key type, and ``str(key)`` for the key string. :raises: `.SSHException` -- if no session is currently active. :return: public key (`.PKey`) of the remote server """ if (not self.active) or (not self.initial_kex_done): > raise SSHException("No existing session") E paramiko.ssh_exception.SSHException: No existing session ../../admin/.venv3/lib/python3.7/site-packages/paramiko/transport.py:834: SSHException ------------------------------ Captured log call ------------------------------- ERROR paramiko.transport:transport.py:1819 Exception: Error reading SSH protocol banner ERROR paramiko.transport:transport.py:1817 Traceback (most recent call last): ERROR paramiko.transport:transport.py:1817 File "/home/amnesia/Persistent/securedrop/admin/.venv3/lib/python3.7/site-packages/paramiko/transport.py", line 2211, in _check_banner ERROR paramiko.transport:transport.py:1817 buf = self.packetizer.readline(timeout) ERROR paramiko.transport:transport.py:1817 File "/home/amnesia/Persistent/securedrop/admin/.venv3/lib/python3.7/site-packages/paramiko/packet.py", line 380, in readline ERROR paramiko.transport:transport.py:1817 buf += self._read_timeout(timeout) ERROR paramiko.transport:transport.py:1817 File "/home/amnesia/Persistent/securedrop/admin/.venv3/lib/python3.7/site-packages/paramiko/packet.py", line 622, in _read_timeout ERROR paramiko.transport:transport.py:1817 raise socket.timeout() ERROR paramiko.transport:transport.py:1817 socket.timeout ERROR paramiko.transport:transport.py:1817 ERROR paramiko.transport:transport.py:1817 During handling of the above exception, another exception occurred: ERROR paramiko.transport:transport.py:1817 ERROR paramiko.transport:transport.py:1817 Traceback (most recent call last): ERROR paramiko.transport:transport.py:1817 File "/home/amnesia/Persistent/securedrop/admin/.venv3/lib/python3.7/site-packages/paramiko/transport.py", line 2039, in run ERROR paramiko.transport:transport.py:1817 self._check_banner() ERROR paramiko.transport:transport.py:1817 File "/home/amnesia/Persistent/securedrop/admin/.venv3/lib/python3.7/site-packages/paramiko/transport.py", line 2216, in _check_banner ERROR paramiko.transport:transport.py:1817 "Error reading SSH protocol banner" + str(e) ERROR paramiko.transport:transport.py:1817 paramiko.ssh_exception.SSHException: Error reading SSH protocol banner ERROR paramiko.transport:transport.py:1817 ERROR paramiko.transport:transport.py:1819 Exception: Error reading SSH protocol banner ERROR paramiko.transport:transport.py:1817 Traceback (most recent call last): ERROR paramiko.transport:transport.py:1817 File "/home/amnesia/Persistent/securedrop/admin/.venv3/lib/python3.7/site-packages/paramiko/transport.py", line 2211, in _check_banner ERROR paramiko.transport:transport.py:1817 buf = self.packetizer.readline(timeout) ERROR paramiko.transport:transport.py:1817 File "/home/amnesia/Persistent/securedrop/admin/.venv3/lib/python3.7/site-packages/paramiko/packet.py", line 380, in readline ERROR paramiko.transport:transport.py:1817 buf += self._read_timeout(timeout) ERROR paramiko.transport:transport.py:1817 File "/home/amnesia/Persistent/securedrop/admin/.venv3/lib/python3.7/site-packages/paramiko/packet.py", line 622, in _read_timeout ERROR paramiko.transport:transport.py:1817 raise socket.timeout() ERROR paramiko.transport:transport.py:1817 socket.timeout ERROR paramiko.transport:transport.py:1817 ERROR paramiko.transport:transport.py:1817 During handling of the above exception, another exception occurred: ERROR paramiko.transport:transport.py:1817 ERROR paramiko.transport:transport.py:1817 Traceback (most recent call last): ERROR paramiko.transport:transport.py:1817 File "/home/amnesia/Persistent/securedrop/admin/.venv3/lib/python3.7/site-packages/paramiko/transport.py", line 2039, in run ERROR paramiko.transport:transport.py:1817 self._check_banner() ERROR paramiko.transport:transport.py:1817 File "/home/amnesia/Persistent/securedrop/admin/.venv3/lib/python3.7/site-packages/paramiko/transport.py", line 2216, in _check_banner ERROR paramiko.transport:transport.py:1817 "Error reading SSH protocol banner" + str(e) ERROR paramiko.transport:transport.py:1817 paramiko.ssh_exception.SSHException: Error reading SSH protocol banner ERROR paramiko.transport:transport.py:1817 _______________________ test_hosts_files[paramiko://app] _______________________ [gw1] linux -- Python 3.7.3 /home/amnesia/Persistent/securedrop/admin/.venv3/bin/python3 host = def test_hosts_files(host): """ Ensure host files mapping are in place """ f = host.file('/etc/hosts') mon_ip = os.environ.get('MON_IP', sdvars.mon_ip) mon_host = sdvars.monitor_hostname assert f.contains(r'^127.0.0.1\s*localhost') > assert f.contains(r'^{}\s*{}\s*securedrop-monitor-server-alias$'.format( mon_ip, mon_host)) E AssertionError: assert False E + where False = >('^192.168.121.58\\s*mon\\s*securedrop-monitor-server-alias$') E + where > = .contains E + and '^192.168.121.58\\s*mon\\s*securedrop-monitor-server-alias$' = ('192.168.121.58', 'mon') E + where = '^{}\\s*{}\\s*securedrop-monitor-server-alias$'.format app/test_ossec_agent.py:19: AssertionError __________________ test_fpf_apt_repo_present[paramiko://app] ___________________ [gw1] linux -- Python 3.7.3 /home/amnesia/Persistent/securedrop/admin/.venv3/bin/python3 host = def test_fpf_apt_repo_present(host): """ Ensure the FPF apt repo, apt.freedom.press, is configured. This repository is necessary for the SecureDrop Debian packages, including: * securedrop-app-code * securedrop-keyring * securedrop-grsec Depending on the host, additional FPF-maintained packages will be installed, e.g. for OSSEC. Install state for those packages is tested separately. """ # If the var fpf_apt_repo_url test var is apt-test, validate that the # apt repository is configured on the host if test_vars.fpf_apt_repo_url == "https://apt-test.freedom.press": f = host.file('/etc/apt/sources.list.d/apt_test_freedom_press.list') else: f = host.file('/etc/apt/sources.list.d/apt_freedom_press.list') repo_regex = r'^deb \[arch=amd64\] {} {} main$'.format( re.escape(test_vars.fpf_apt_repo_url), re.escape(host.system_info.codename)) > assert f.contains(repo_regex) E AssertionError: assert False E + where False = >('^deb \\[arch=amd64\\] https://apt\\.freedom\\.press focal main$') E + where > = .contains common/test_fpf_apt_repo.py:35: AssertionError __________________ test_fpf_apt_repo_present[paramiko://mon] ___________________ [gw1] linux -- Python 3.7.3 /home/amnesia/Persistent/securedrop/admin/.venv3/bin/python3 host = def test_fpf_apt_repo_present(host): """ Ensure the FPF apt repo, apt.freedom.press, is configured. This repository is necessary for the SecureDrop Debian packages, including: * securedrop-app-code * securedrop-keyring * securedrop-grsec Depending on the host, additional FPF-maintained packages will be installed, e.g. for OSSEC. Install state for those packages is tested separately. """ # If the var fpf_apt_repo_url test var is apt-test, validate that the # apt repository is configured on the host if test_vars.fpf_apt_repo_url == "https://apt-test.freedom.press": f = host.file('/etc/apt/sources.list.d/apt_test_freedom_press.list') else: f = host.file('/etc/apt/sources.list.d/apt_freedom_press.list') repo_regex = r'^deb \[arch=amd64\] {} {} main$'.format( re.escape(test_vars.fpf_apt_repo_url), re.escape(host.system_info.codename)) > assert f.contains(repo_regex) E AssertionError: assert False E + where False = >('^deb \\[arch=amd64\\] https://apt\\.freedom\\.press focal main$') E + where > = .contains common/test_fpf_apt_repo.py:35: AssertionError _____________ test_unattended_upgrades_functional[paramiko://app] ______________ [gw0] linux -- Python 3.7.3 /home/amnesia/Persistent/securedrop/admin/.venv3/bin/python3 host = def test_unattended_upgrades_functional(host): """ Ensure unatteded-upgrades completes successfully and ensures all packages are up-to-date. """ c = host.run('sudo unattended-upgrades --dry-run --debug') assert c.rc == 0 expected_origins = ( "Allowed origins are: origin=Ubuntu,archive=focal, origin=Ubuntu,archive=focal-security" ", origin=Ubuntu,archive=focal-updates, origin=SecureDrop,codename=focal" ) expected_result = ( "No packages found that can be upgraded unattended and no pending auto-removals" ) assert expected_origins in c.stdout > assert expected_result in c.stdout E assert 'No packages found that can be upgraded unattended and no pending auto-removals' in "Starting unattended upgrades script\nAllowed origins are: origin=Ubuntu,archive=focal, origin=Ubuntu,archive=focal-se...ades installed\nInstCount=0 DelCount=0 BrokenCount=0\nThe list of kept packages can't be calculated in dry-run mode.\n" E + where "Starting unattended upgrades script\nAllowed origins are: origin=Ubuntu,archive=focal, origin=Ubuntu,archive=focal-se...ades installed\nInstCount=0 DelCount=0 BrokenCount=0\nThe list of kept packages can't be calculated in dry-run mode.\n" = CommandResult(command=b'sudo unattended-upgrades --dry-run --debug', exit_status=0, stdout=b"Starting unattended upgra.../usr/bin/dpkg --force-confdef --force-confold --force-confdef --force-confold --status-fd 10 --configure --pending \n').stdout common/test_automatic_updates.py:130: AssertionError ___________________ test_ossec_connectivity[paramiko://mon] ____________________ [gw0] linux -- Python 3.7.3 /home/amnesia/Persistent/securedrop/admin/.venv3/bin/python3 host = def test_ossec_connectivity(host): """ Ensure ossec-server machine has active connection to the ossec-agent. The ossec service will report all available agents, and we can inspect that list to make sure it's the host we expect. """ desired_output = "{}-{} is available.".format( securedrop_test_vars.app_hostname, os.environ.get('APP_IP', securedrop_test_vars.app_ip)) with host.sudo(): c = host.check_output("/var/ossec/bin/list_agents -a") > assert c == desired_output E AssertionError: assert 'app-prod-192...is available.' == 'app-192.168....is available.' E - app-192.168.121.148 is available. E + app-prod-192.168.121.148 is available. E ? +++++ mon/test_ossec_server.py:22: AssertionError _______________________ test_hosts_files[paramiko://mon] _______________________ [gw1] linux -- Python 3.7.3 /home/amnesia/Persistent/securedrop/admin/.venv3/bin/python3 host = def test_hosts_files(host): """ Ensure host files mapping are in place """ f = host.file('/etc/hosts') app_ip = os.environ.get('APP_IP', securedrop_test_vars.app_ip) app_host = securedrop_test_vars.app_hostname assert f.contains('^127.0.0.1.*localhost') > assert f.contains(r'^{}\s*{}$'.format(app_ip, app_host)) E AssertionError: assert False E + where False = >('^192.168.121.148\\s*app$') E + where > = .contains E + and '^192.168.121.148\\s*app$' = ('192.168.121.148', 'app') E + where = '^{}\\s*{}$'.format mon/test_ossec_server.py:89: AssertionError __________________ test_postfix_generic_maps[paramiko://mon] ___________________ [gw0] linux -- Python 3.7.3 /home/amnesia/Persistent/securedrop/admin/.venv3/bin/python3 host = def test_postfix_generic_maps(host): """ Check configuration of Postfix generic map when sasl_domain is set and ossec_from_address is not specified. """ assert host.file("/etc/postfix/generic").exists > assert host.file("/etc/postfix/generic").contains( "^ossec@{} {}@{}".format( securedrop_test_vars.monitor_hostname, securedrop_test_vars.sasl_username, securedrop_test_vars.sasl_domain, ) ) E AssertionError: assert False E + where False = >('^ossec@mon tuzmjvur@riseup.net') E + where > = .contains E + where = ('/etc/postfix/generic') E + where = .file E + and '^ossec@mon tuzmjvur@riseup.net' = ('mon', 'tuzmjvur', 'riseup.net') E + where = '^ossec@{} {}@{}'.format E + and 'mon' = {}.monitor_hostname E + and 'tuzmjvur' = {}.sasl_username E + and 'riseup.net' = {}.sasl_domain mon/test_postfix.py:36: AssertionError =========================== short test summary info ============================ FAILED app/test_apparmor.py::test_apparmor_pkg[paramiko:/app-apparmor-utils] FAILED app/test_apparmor.py::test_apparmor_pkg[paramiko:/app-apparmor] - para... FAILED app/test_apparmor.py::test_apparmor_apache_capabilities[paramiko:/app-dac_override] FAILED app/test_ossec_agent.py::test_hosts_files[paramiko:/app] - AssertionEr... FAILED common/test_fpf_apt_repo.py::test_fpf_apt_repo_present[paramiko:/app] FAILED common/test_fpf_apt_repo.py::test_fpf_apt_repo_present[paramiko:/mon] FAILED common/test_automatic_updates.py::test_unattended_upgrades_functional[paramiko:/app] FAILED mon/test_ossec_server.py::test_ossec_connectivity[paramiko:/mon] - Ass... FAILED mon/test_ossec_server.py::test_hosts_files[paramiko:/mon] - AssertionE... FAILED mon/test_postfix.py::test_postfix_generic_maps[paramiko:/mon] - Assert... = 10 failed, 425 passed, 7 skipped, 3 xfailed, 1 xpassed, 10 warnings in 2016.78s (0:33:36) = -------- ```

Test plan and results

Environment

  1. [x] Take a backup of your existing SecureDrop installation using the securedrop-admin backup command. You will need enough free space on your Admin Workstation USB to complete this backup.

  2. [x] ~Either~ run the ansible QA playbook from qa-update-playbook (pending #6123):
    https://github.com/freedomofpress/securedrop/blob/ed008a0e49738ab8695fe438255b9ee06362831f/install_files/ansible-base/securedrop-qa.yml#L7-L18

Retesting only those cases that failed or raised questions in https://github.com/freedomofpress/securedrop/issues/6103#issuecomment-938341940...

2.1.0 release-specific changes

amnesia@amnesia:~/Persistent/securedrop$ git branch --points-at HEAD
* release/2.1.0
amnesia@amnesia:~/Persistent/securedrop$ ssh app "grep 'SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1 -TLSv1.2' /etc/apache2/sites-enabled/source.conf"
SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1 -TLSv1.2
amnesia@amnesia:~/Persistent/securedrop$ testssl https://e336cukmz45e4ittiaa35gxjojz6467355tkssnpjbclv3omk2fmb6yd.onion/
amnesia@amnesia:~/Persistent/securedrop$ testssl 
[...]
 Testing protocols via sockets except NPN+ALPN 

 SSLv2      not offered (OK)
 SSLv3      not offered (OK)
 TLS 1      not offered
 TLS 1.1    not offered
 TLS 1.2    not offered
 TLS 1.3    offered (OK): final
 NPN/SPDY   not offered
 ALPN/HTTP2 http/1.1 (offered)
[...]
amnesia@amnesia:~/Persistent/securedrop$ git branch --points-at HEAD
* release/2.1.0
amnesia@amnesia:~/Persistent/securedrop$ for s in app mon; do ssh $s sudo apt-key list | grep -b2 -a1 "2224"; done
Warning: apt-key output should not be parsed (stdout is not a terminal)
Warning: apt-key output should not be parsed (stdout is not a terminal)
amnesia@amnesia:~/Persistent/securedrop$ git branch --points-at HEAD
* release/2.1.0
amnesia@amnesia:~/Persistent/securedrop$ for s in app mon; do ssh $s sudo apt-key list | grep -b2 -a1 "2359"; done
Warning: apt-key output should not be parsed (stdout is not a terminal)
326-pub   rsa4096 2021-05-10 [SC] [expires: 2022-07-04]
378:      2359 E653 8C06 13E6 5295  5E6C 188E DD3B 7B22 E6A3
435-uid           [ unknown] SecureDrop Release Signing Key <securedrop-release-key-2021@freedom.press>
Warning: apt-key output should not be parsed (stdout is not a terminal)
326-pub   rsa4096 2021-05-10 [SC] [expires: 2022-07-04]
378:      2359 E653 8C06 13E6 5295  5E6C 188E DD3B 7B22 E6A3
435-uid           [ unknown] SecureDrop Release Signing Key <securedrop-release-key-2021@freedom.press>
conorsch commented 2 years ago

No further concerns to point out. 100% of all testinfra tests pass on my prod VMs, which is mostly due to the key clobbering mentioned in https://github.com/freedomofpress/securedrop/issues/6103#issuecomment-943852881, but again, not a release-blocker. 2.1.0 is looking good to me! Will focus on LM tasks over the weekend to see if we can't drum up a bit more coverage. Thereafter, it's by the numbers.

zenmonkeykstop commented 2 years ago

So far so good on 2.1.0~rc2. One issue of note is that on the clean install scenario, I observed an apt-update failure: ... It appears the order of operations is as follows:

1. All apt packages updated in `prepare-servers`, as expected

2. apt-test key is added via local modifications of the `install-fpf-repo` vars, as part of QA

3. apt-test repo URL is added via local modifications of the `install-fpf-repo` vars, as part of QA

4. the `securedrop-keyring` package is installed from apt-test, which clobbers the test key with the prod key upon installation

5. the `common` role fails to update apt lists again, since the apt-test repo is configured but the key is removed

This issue only affects QA testing, it isn't a problem for prod. But I'm documenting it here in case other testers encounter it.

I'm confused as to why we weren't seeing this all the time in QA, it looks like the only possibly-relevant change was the upgrade added in prepare-servers. It could probably be avoided by either:

I don't think it merits another RC if there's a workaround, but we should fix this for the next release, as it means clean install test scenarios aren't exactly representative of reality.

cfm commented 2 years ago

I've updated https://github.com/freedomofpress/securedrop/issues/6103#issuecomment-943859841 to log some surprising testinfra failures in the VM upgrade scenario, which I'll investigate further on Monday.

cfm commented 2 years ago

https://github.com/freedomofpress/securedrop/issues/6103#issuecomment-944786736:

I've updated #6103 (comment) to log some surprising testinfra failures in the VM upgrade scenario, which I'll investigate further on Monday.

Initial investigations follow.

FAILED app/test_apparmor.py::test_apparmor_pkg[paramiko:/app-apparmor-utils]
FAILED app/test_apparmor.py::test_apparmor_pkg[paramiko:/app-apparmor] - para...
FAILED app/test_apparmor.py::test_apparmor_apache_capabilities[paramiko:/app-dac_override]

paramiko.ssh_exception.SSHException: No existing session errors look like transient SSH failures.

FAILED app/test_ossec_agent.py::test_hosts_files[paramiko:/app] - AssertionEr...

Seems to be looking for default hostname mon rather than configured hostname mon-prod:

amnesia@amnesia:~/Persistent/securedrop$ ssh app cat /etc/hosts
127.0.0.1    localhost
127.0.1.1    app-prod    app-prod

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
192.168.121.58  mon-prod securedrop-monitor-server-alias
FAILED common/test_fpf_apt_repo.py::test_fpf_apt_repo_present[paramiko:/app]
FAILED common/test_fpf_apt_repo.py::test_fpf_apt_repo_present[paramiko:/mon]

Expected per https://github.com/freedomofpress/securedrop/issues/6103#issuecomment-939044457.

FAILED common/test_automatic_updates.py::test_unattended_upgrades_functional[paramiko:/app]

Looks like a change in unattended-upgrades --dry-run behavior:

E       assert 'No packages found that can be upgraded unattended and no pending auto-removals' in "Starting unattended upgrades script\nAllowed origins are: origin=Ubuntu,archive=focal, origin=Ubuntu,archive=focal-se...ades installed\nInstCount=0 DelCount=0 BrokenCount=0\nThe list of kept packages can't be calculated in dry-run mode.\n"
E        +  where "Starting unattended upgrades script\nAllowed origins are: origin=Ubuntu,archive=focal, origin=Ubuntu,archive=focal-se...ades installed\nInstCount=0 DelCount=0 BrokenCount=0\nThe list of kept packages can't be calculated in dry-run mode.\n" = CommandResult(command=b'sudo unattended-upgrades --dry-run --debug', exit_status=0, stdout=b"Starting unattended upgra.../usr/bin/dpkg --force-confdef --force-confold --force-confdef --force-confold --status-fd 10 --configure --pending \n').stdout
FAILED mon/test_ossec_server.py::test_ossec_connectivity[paramiko:/mon] - Ass...
FAILED mon/test_ossec_server.py::test_hosts_files[paramiko:/mon] - AssertionE...
FAILED mon/test_postfix.py::test_postfix_generic_maps[paramiko:/mon] - Assert...

Looking for default hostnames app and mon rather than configured hostnames {app,mon}-prod (see https://github.com/freedomofpress/securedrop/issues/6127#issuecomment-938342242).

zenmonkeykstop commented 2 years ago

Thanks @cfm:

It would be good to get a clean run if possible on the unattended-upgrades one (immediately after a non-dry-run one should be cool), but otherwise I think we're ok here.

cfm commented 2 years ago

@zenmonkeykstop in https://github.com/freedomofpress/securedrop/issues/6103#issuecomment-946018659:

  • The unattended-upgrades error may just be bad luck - the test seems to assume that the system is up-to-date, which will not be true if new updates have been made available since installation or the overnight run.

It would be good to get a clean run if possible on the unattended-upgrades one (immediately after a non-dry-run one should be cool), but otherwise I think we're ok here.

Thanks for this suggestion. Confirmed that common/test_automatic_updates.py::test_unattended_upgrades_functional has subsequently passed with:

amnesia@amnesia:~/Persistent/securedrop$ ssh app "sudo unattended-upgrades -d && sudo reboot"
amnesia@amnesia:~/Persistent/securedrop$ ssh mon "sudo unattended-upgrades -d && sudo reboot"
amnesia@amnesia:~/Persistent/securedrop$ ./securedrop-admin verify

So all of https://github.com/freedomofpress/securedrop/issues/6103#issuecomment-945983929 is safe to ignore for QA and release purposes.

zenmonkeykstop commented 2 years ago

Basic Server Testing

Command Line User Generation

Administration

Application Acceptance Testing

Source Interface SKIPPED

Journalist Interface SKIPPED

Login base cases

2.1.0 release-specific changes

Web Applications

eloquence commented 2 years ago

Updated an Admin Workstation on Tails 4.22 from SecureDrop 2.0.2 to SecureDrop 2.1.0 successfully using the graphical updater.