SecureDrop Workstation 0.2.2beta QA

[eloquence]

This is an issue to track QA findings against beta release 0.2.2 (Debian packaging release 0.2.1) of the SecureDrop Workstation, dated 2020-02-03, using the provisional test plan as a starting point.

[eloquence]

Server: v2+v3 prod server running SecureDrop 1.2.1, using the v3 address in config.json

• [x] when SecureDrop desktop icon is double-clicked, preflight updater is displayed
• [x] After preflight updater runs, when user clicks Continue, login dialog is displayed

Login

• [x] show/hide password functionality works
• [x] incorrect password cannot log in
• [x] invalid 2FA token cannot log in
• [x] 2FA immediate reuse cannot log in (tested by first submitting wrong passphrase, then correcting passphrase and altering token - TODO: test steps should be clarified)
• [x] valid credentials and 2FA can log in (initial login after install failed w/ timeouts)

  After login

• [x] the login dialog closes
• [x] source data is downloaded and source list is populated
• [x] user is prompted for GPG key access
• [x] submissions and replies are decrypted
• [x] the source list is displayed but no sources are selected by default
• [x] the conversation view is not populated

  After selecting source

• [x] conversation view is populated with source conversation
• [x] a source message containing HTML is displayed as unformatted text (tested with message containing <>& '", TODO: test plan should be clarified)
• [x] source submissions have an active Download button (Hover state still broken until https://github.com/freedomofpress/securedrop-client/pull/805 makes it into a release)
• [x] source submission compressed file size is displayed accurately (File size matched size reported in Journalist Interface)

  Deleting source

• [x] a menu is displayed with a delete source account option
• when delete source account is selected:
  • [ ] the source is deleted from the source list and the conversation view is blanked :x: (that eventually happened, but first I got a "Failed to delete source from server" message and nothing happened; NB: I had previously restarted the server while client was running, but SI was loading fine in Tor)
  • [x] the source is deleted from the server and not restored on next sync
  • [ ] source submissions and messages are removed from the client's data directory :x: (I am still finding the deleted file in the client's data directory)

[rmol]

Found numerous problems around deletion and conversation view updating.

SecureDrop Workstation test scenarios

Qubes scenarios

Verify mime handling in sd-app

• [ ] Behavior in client (e.g. mailto, http:// link w/ modified client that disables escaping)

• [x] Review default mime handler apps in sd-app:

  user@sd-app:~$ for i in $(awk '{print $1}' /etc/mime.types | grep -v '#'); do xdg-mime query default $i; done | sort | uniq
  display-im6.q16.desktop
  gcr-viewer.desktop
  open-in-dvm.desktop
  org.gnome.Nautilus.desktop
  thunderbird.desktop
  vim.desktop

  • Images that would be opened locally with display-im6.q16.desktop:

    image/pcx
    image/x-icon
    image/x-ms-bmp
    image/x-rgb
    image/x-xwindowdump

  • Certificate files that would be opened locally with gcr-viewer.desktop:

    application/pkcs10: gcr-viewer.desktop
    application/pkcs7-mime: gcr-viewer.desktop
    application/pkix-cert: gcr-viewer.desktop
    application/pkix-crl: gcr-viewer.desktop
    application/x-x509-ca-cert: gcr-viewer.desktop

  • application/gzip would be opened by org.gnome.Nautilus.desktop

  • message/rfc822 would be opened by thunderbird.desktop, but thunderbird segfaults

  • vim.desktop would be used to open these, but vim is not installed:

    application/x-shellscript
    text/english
    text/x-c++hdr
    text/x-chdr
    text/x-c++src
    text/x-csrc
    text/x-java
    text/x-makefile
    text/x-moc
    text/x-pascal
    text/x-tcl
    text/x-tex

Verify mime handling in sd-viewer

• [ ] Review default mime handler apps in sd-viewer
• [x] Send a .desktop file that execute code, see what happens
  • Sent qubes-run-terminal.desktop. A viewer dispVM opened, a blank light gray window covered the screen then disappeared, and the dispVM halted. A terminal was not started, as far as I could tell. Replacing vm-file-editor in the viewer template with a script containing a sleep, I could see no related process running in the dispVM.
• [x] Check macro execution default policy in libreoffice
  • High. Only signed macros from trusted sources are allowed to run. Unsigned macros are disabled.

Packages

• [ ] Ensure reproducibility of workstation packages: review hashes/diffoscope of Workstation Python packages (client, proxy, log, export), given wheels on pypi mirror.

RPC Policies

• [x] Final audit of Qubes RPC policies
  • [x] qubes.ClipboardPaste
  • [x] qubes.FeaturesRequest
  • [x] qubes.Filecopy
  • [x] qubes.Gpg
  • [x] qubes.GpgImportKey
  • [x] qubes.OpenInVM
  • [x] qubes.OpenURL
  • [x] qubes.PdfConvert
  • [x] qubes.StartApp
  • [x] qubes.USB
  • [x] qubes.USBAttach
  • [x] qubes.VMRootShell
  • [x] qubes.VMShell
  • [x] securedrop.Log
  • [x] securedrop.Proxy

Hardware/virtualization

• [x] Can we check if Hyperthreading is disabled in dom0
  • lscpu reports Thread(s) per core == 1
  • cat /sys/devices/system/cpu/smt/active returns zero
  • dmidecode -t processor reports Core Count and Thread Count are equal

Archive handling

• [x] What happens when a zip archive is submitted to the source interface?
  • It's encrypted and stored on the server.
  • If you download and open it in the client, it's opened with File Roller/Archive Manager in a disposable VM.

Logging

• [x] Logs are sent to sd-log VM
• [x] No sensitive information is stored in sd-log

Client scenarios

Scenario: Online mode

Prerequisites:

• server is available and contains source test data
• access to sd-gpg keyring has not been previously granted
• client data directory is empty
• the sd-devices VM is not running (shut down manually if necessary)
• a supported printer is available, but not attached.
• all VMs are up-to-date

Login

• [x] when SecureDrop desktop icon is double-clicked, preflight updater is displayed
  • This happened automatically when I logged in, using XFCE.
• [x] After preflight updater runs, when user clicks Continue, login dialog is displayed
• In login dialog:
  • [x] show/hide password functionality works
  • [x] incorrect password cannot log in
  • [x] invalid 2FA token cannot log in
  • [x] 2FA immediate reuse cannot log in
  • [x] valid credentials and 2FA can log in

Sources

• after valid login:
  • [x] the login dialog closes
  • [x] source data is downloaded and source list is populated
  • [x] user is prompted for GPG key access
  • [x] submissions and replies are decrypted
  • [x] the source list is displayed but no sources are selected by default
  • [x] the conversation view is not populated
• when a source is selected in source list:
  • [x] conversation view is populated with source conversation
  • [x] a source message containing HTML is displayed as unformatted text
  • [x] source submissions have an active Download button
  • [x] source submission compressed file size is displayed accurately
• when the upper right 3-dot button is clicked:
  • [x] a menu is displayed with a delete source account option
  • when delete source account is selected:
  • [x] the source is deleted from the source list and the converation view is blanked
  • [x] the source is deleted from the server and not restored on next sync
  • [ ] source submissions and messages are removed from the client's data directory
    • :x: They are not, because of https://github.com/freedomofpress/securedrop-client/issues/856
• when a source is starred in source list, and the client is closed and reopened in Online mode:
  • [x] the source is still starred in the source list

Replies

• when a source is selected in the source list:
  • [x] the reply panel is available for use and there is no message asking the user to sign in
  • [x] a reply can be added to the conversations
  • [x] a reply containing HTML is displayed as unformatted text
  • [x] two replies added immediately after each other are ordered correctly

Submissions

Preview

• when Download is clicked on a submission:
  • [x] the submission is downloaded and decrypted
  • [x] the Download button is replaced with Print and Export options
  • [x] the submission filename is displayed.
• For a DOC submission:
  • [x] when the submission filename is clicked, a disposable VM (dispVM) is started.
  • [x] after the dispVM starts, the submission is displayed in LibreOffice
  • [x] when LibreOffice is closed, the dispVM shuts down
• For an XLSX submission:
  • [x] when the submission filename is clicked, a disposable VM (dispVM) is started.
  • [x] after the dispVM starts, the submission is displayed in LibreOffice
  • [x] when LibreOffice is closed, the dispVM shuts down
• For a PDF submission:
  • [x] when the submission filename is clicked, a dispVM is started.
  • [x] after the dispVM starts, the submission is displayed in evince
  • [x] when evince is closed, the dispVM shuts down
• For a JPEG submission:
  • [x] when the submission filename is clicked, a dispVM is started.
  • [x] after the dispVM starts, the submission is displayed in Image Viewer
  • [x] when the image viewer is closed, the dispVM shuts down

Export

• NOTE: I had to increase the minimum memory allocated to sd-devices to avoid cryptsetup operations being targeted by the OOM-killer. 1.5G seems reliable. Alternatively, @conorsch suggested bouncing qmemman with sudo systemctl restart qubes-qmemman if it had failed, which can be determined with the script in this comment.

• When Export is first clicked on a submission:

  • [x] the "Preparing to export..." message is displayed
  • [x] the sd-devices VM is started
  • [x] the user is prompted to insert an Export USB
  • [x] On clicking Cancel, the prompt closes and the file is not exported

• When Export is clicked on the submission again:

  • [x] the "Preparing to export..." message is displayed
  • [x] the user is prompted to insert an Export USB
  • When the user inserts an invalid Export USB, attaches it to the sd-devices VM and clicks OK:
  • [x] a message is displayed indicating that the Export USB is invalid and the user is prompted to insert a valid device

• When Export is clicked on the submission again:

  • [x] the "Preparing to export..." message is displayed
  • [x] the user is prompted to insert an Export USB
  • When the user inserts a valid Export USB, attaches it to the sd-devices VM, and clicks OK:
  • [x] the user is prompted for the Export USB's password
  • When the user enters an invalid Export USB password and clicks Submit:
  • [x] a failure message is displayed and the user is prompted to enter the password again
  • When the user enters an valid Export USB password and clicks Submit:
  • [x] the file is saved to the Export USB

• When the user detaches the Export USB and mounts it on another VM or computer:

  • [x] the decrypted submission is available in on the Export USB, in a directory sd-export-<timestamp>/export_data

Print

• When the user clicks Print on a downloaded submission:
  • [x] a "Preparing to print..." message is displayed
  • [x] the sd-devicesVM is started
  • [x] the user is prompted to connect a supported printer
• When the user connects a printer, attaches it to the sd-devices VM, and clicks Continue:
  • [x] a "Printing..." message is displayed
  • [x] the X Printer Panel dialog is displayed with the printer selected
• When the user clicks Print in the X Printer Panel:
  • [x] the submission is printed successflly.

Closing the client

• When the user clicks the main window close button:
  • [x] the client exits.

Scenario: Offline mode without existing data

Prerequisites:

• server is available and contains source test data
• client data directory is empty
• the sd-devices VM is not running (shut down manually if necessary)
• a supported printer is available, but not attached.

Offline to Online

• [ ] When SecureDrop desktop icon is double-clicked, preflight updater is displayed
• [ ] After preflight updater runs, when user clicks Continue, login dialog is displayed
• [x] When user clicks Work Offline, login dialog closes and main window opens
• after startup:
  • [x] there is no sync attempt with the server
  • [x] the source list is empty
• When the user clicks the top-left user icon and chooses Sign in:
  • [x] the login dialog is displayed over the main window
• When the user enters valid login details and clicks Log in:
  • [x] the login dialog closes
  • [x] The user icon is updated to reflect the user's details
  • [x] the client is synced with the server and the source list is updated
• When the user selects a source with submissions from the source list:
  • [x] the conversation view is populated with the source conversation
  • [x] the reply panel is active
  • [x] a reply can be sent to the source
  • [x] a submission can be downloaded
  • [x] a downloaded submission can be exported
• When the user clicks the main window close button:
  • [x] the client exits.

Scenario: Offline mode with existing data

Prerequisites:

• server is available and contains source test data
• test data includes at least one previously downloaded submission
• test data includes at least one undownloaded submission
• client data directory has been synced with server in a previous login
• the sd-devices VM is not running (shut down manually if necessary)
• a supported printer is available, but not attached.

Offline to Online

• [ ] When SecureDrop desktop icon is double-clicked, preflight updater is displayed
• [ ] After preflight updater runs, when user clicks Continue, login dialog is displayed
• [x] When user clicks Work Offline, login dialog closes and main window opens
• after startup:
  • [x] there is no sync attempt with the server
  • [x] the source list is populated with contents of last server sync
• When the user selects a source with submissions from the source list:
  • [x] the conversation view is populated with the source conversation
  • [x] the reply panel is inactive, with a "Sign in" message
  • [x] a previously downloaded submission can be exported
  • [x] a previously downloaded submission can be printed
  • [x] When the user clicks Download on an undownloaded submission, a message is displayed instructing the user to sign in to perform the download
• When the user clicks the top-left user icon and chooses Sign in:
  • [x] the login dialog is displayed over the main window
• When the user enters valid login details and clicks Log in:
  • [x] the login dialog closes
  • [x] The user icon is updated to reflect the user's details
  • [x] source data is synced with the server
• When the user selects a source with submissions from the source list:
  • [x] the conversation view is populated with the source conversation
  • [x] the reply panel is active
  • [x] When the user replies to a source, the reply is added to the source conversation
  • [x] When the user clicks Download on an undownloaded submission, the submission is downloaded and decrypted
  • [x] When the user clicks Export on a submission, the export process can be completed
  • [x] When the user clicks Print on a submission, the print process can be completed
• When the user clicks the main window close button:
  • [x] the client exits.

Scenario: Client and Journalist Interface both in use

Note: this scenario requires access to the Journalist Interface (JI) via Tor Browser. If the scenario is being tested on Qubes, the JI address can be found in sd-whonix in /usr/local/etc/torrc.d/50_user.conf. The sd-proxy VM includes Tor Browser, and can be used to access the JI without config changes.

Prerequisites:

• server is available and contains source test data
• client data directory is empty

Login

• [ ] when SecureDrop desktop icon is double-clicked, preflight updater is displayed
• [ ] After preflight updater runs, when user clicks Continue, login dialog is displayed
• after valid login to client:
  • [x] the login dialog closes
  • [x] source data is downloaded and source list is populated
  • [x] user is prompted for GPG key access
  • [x] submissions and replies are decrypted
  • [x] the source list is displayed but no sources are selected by default
  • [x] the conversation view is not populated
• when the JI address is visited in Tor Browser:
  • [x] JI login page is displayed
• after valid login to JI using same account as for client:
  • [x] sources page is displayed, containing the same sources as the client (order may differ)

Sources, replies, submissions

• when a source is starred in the client:

  • [x] the source is also starred in the JI after a page reload

• when a starred source is unstarred in the JI:

  • [ ] the source is also unstarred in the client after next sync.
  • :x: Nope:

    securedrop_client.logic:462(on_sync_failure) DEBUG: The SecureDrop server cannot be reached due to Error: (sqlite3.OperationalError) database is locked
    [SQL: UPDATE sources SET last_updated=? WHERE sources.id = ?]
    [parameters: ('2020-03-04 14:32:57.478957', 1)]
    (Background on this error at: http://sqlalche.me/e/e3q8)

    Syncs apparently stopped there. The client had to be restarted. Eventually the source was unstarred.

• when a reply is sent to a source via the client:

  • [x] the reply is visible in the JI and can be viewed by the source in the Source Interface

• when a reply is sent to a source via the JI:

  • [x] the reply is visible in the source conversation view after next sync

• when a reply is deleted by a source:

  • [ ] the reply is flagged as having being read in the client
  • :x: No, nothing changes. Replies don't have an is_read indicator.

• when an individual file submission is deleted in the JI:

  • [ ] the submission is no longer listed in the conversation view

  • :x: The export/print/filename widgets for the deleted submission are still visible on top of the first submission in the conversation view. This persists until the client is restarted.

  • :x: For a submission that had not been downloaded, nothing changes, and clicking download on it crashes the client with sqlalchemy.orm.exc.NoResultFound: No row was found for one()

  • [ ] the submission files are deleted from the client data directory

  • :x: No, again because of https://github.com/freedomofpress/securedrop-client/issues/856

• when an individual message is deleted in the JI:

  • [ ] the message is no longer listed in the conversation view
  • :x: It is still there after several syncs and clicking on other sources and coming back. The message has been deleted from the client db, and if I restart the client, it is no longer shown in the conversation view.
  • [ ] the message files are deleted from the client data directory
  • :x: I don't think this is a valid check. There are never message files in the client data directory after they're downloaded and decrypted. Only file submissions are kept in the data directory now.

• when a source is deleted in the JI:

  • [x] the source is no longer listed in the client after next sync
  • [ ] files associated with the source are no longer present in the client data directory
  • :x: No, again because of https://github.com/freedomofpress/securedrop-client/issues/856

• when a source is deleted in the client:

  • [x] the source is no longer listed in the JI after a page reload

[eloquence]

I suggest we capture the configured server environment as well in these reports to make sure we have adequate coverage on testing with both v2 and v3 configurations. Just added that to my report.

[eloquence]

the reply is flagged as having being read in the client :x: No, nothing changes. Replies don't have an is_read indicator.

This is in fact expected behavior, as this feature has not been implemented yet. However, I did not find an issue for it, so I added one: https://github.com/freedomofpress/securedrop-client/issues/889. I'll update the test plan on this point.

[eloquence]

when an individual file submission is deleted in the JI:

the submission is no longer listed in the conversation view :x: The export/print/filename widgets for the deleted submission are still visible on top of the first submission in the conversation view. This persists until the client is restarted.

Was not able to reproduce for a single submission (it was removed from the conversation view as expected). Will now try with multiple.

[eloquence]

^ I uploaded a new file as this source, after deleting the previous upload from the JI. That resulted in this state:

Note how the "Encrypted file on server" preview snippet appears, but there is no file widget. This is the corresponding JI state:

Submission 3 is the one that was deleted in the previous step. Will attempt a clean repro just on this behavior.

[eloquence]

^ This is now tracked in https://github.com/freedomofpress/securedrop-client/issues/891. As noted there, the file widget is rendered correctly after a client restart, or after switching into offline mode by signing out.

[eloquence]

• when an individual file submission is deleted in the JI:
  • [ ] the submission files are deleted from the client data directory
  • :x: No, again because of https://github.com/freedomofpress/securedrop-client/issues/856

I don't think this is due to https://github.com/freedomofpress/securedrop-client/issues/856 as I don't see any AppArmor denial in the logs. I've filed a separate issue for this in https://github.com/freedomofpress/securedrop-client/issues/892.

[eloquence]

Thanks again @rmol for the detailed report. There are still more issues to be uncovered from it, but I also think we're reaching the point where a new release will help us have more confidence that QA results reflect the current state of the codebase.

Big shout-out to @zenmonkeykstop for the very thorough test plan that's helping us to uncover these issues.

[eloquence]

Creating a new issue for QA reports about subsequent releases of RPM/Debian packages, cross-referencing this one.