redshiftzero commented 6 years ago

This is a tracking issue for the upcoming release of SecureDrop 0.10.0 - tasks may get added or modified.

Feature freeze: October 9, 2018 String freeze: October 16, 2018 Pre-release announcement: October 16, 2018 Release date: October 23, 2018

SecureDrop maintainers and testers: As you QA 0.10.0, please report back your testing results as comments on this ticket. File GitHub issues for any problems found, tag them "QA: Release", and associate them with the 0.10.0 milestone for tracking (or ask a maintainer to do so).

Test debian packages will be posted on https://apt-test.freedom.press signed with the test key. An Ansible playbook testing the upgrade path is here.

Prepare release candidate (0.10.0~rc1)

[x] Prepare 0.10.0-rc1 release changelog - @redshiftzero
[x] Branch off 0.10.0~rc1 into release/0.10.0 - @redshiftzero
[skipped] Build debs (including new Tor package) and put up 0.10.0~rc1 on test apt server

We merged in two important fixes prior to building debs for rc1 so stopped here

Prepare release candidate (0.10.0~rc2)

[x] Prepare 0.10.0-rc2 release changelog - @redshiftzero
[x] Prepare test plan for 0.10.0~rc2 - @zenmonkeykstop and @redshiftzero
[x] Push 0.10.0-rc2 tag - @redshiftzero
[x] Build debs (including new Tor package) and put up 0.10.0~rc2 on test apt server - @emkll

Prepare release candidate (0.10.0~rc3)

[x] Prepare 0.10.0-rc3 release changelog - @redshiftzero
[x] Prepare test plan for 0.10.0~rc3 - @emkll
[x] Push 0.10.0-rc3 tag - @redshiftzero
[x] Build debs and put up 0.10.0~rc3 on test apt server - @emkll

Prepare release candidate (0.10.0~rc4)

[x] Prepare 0.10.0-rc4 release changelog - @redshiftzero
[x] Prepare test plan for 0.10.0~rc4 @zenmonkeykstop @emkll
[x] Push 0.10.0-rc4 tag - @redshiftzero
[x] Build debs and put up 0.10.0~rc4 on test apt server - @emkll

[we will add checklists for rc5 release candidates and later as we proceed through the release process]

Test plan to be posted during business hours Pacific time on Wednesday, October 10th.

QA Matrix for 0.10.0

Test Plan for 0.10.0

After each test, please update the QA matrix and post details for Basic Server Testing, Application Acceptance Testing and 0.10.0-specific testing below in comments to this ticket.

Final release

[x] Ensure builder in release branch is updated and/or update builder image - @emkll or @msheiny
[x] Merge final translations - @kushaldas
[x] Push signed tag - @redshiftzero
[x] Build final Debian packages for 0.10.0 - @conorsch
[x] Upload Debian packages (including new linux-image, linux-firmware, and tor packages) to apt test - @conorsch
[x] Pre-Flight: Test install and upgrade of 0.10.0 works w/ prod repo debs, test updater logic in Tails - @zenmonkeykstop
[x] Write and send release announcement - @eloquence
[x] Publish blog post about 0.10.0 Debian package release and instructions for admins - @eloquence

Post release

[x] Merge changelog (i.e. rc commits) back to develop - @redshiftzero
[x] Bump version on develop in prep for 0.11.0 release - @redshiftzero

zenmonkeykstop commented 6 years ago

Note to self (and others wishing to test upgrade path on HW) - ansible playbook to set instance up to use apt-test. instead of apt. is here: https://gist.github.com/conorsch/e7556624df59b2a0f8b81f7c0c4f9b7d

zenmonkeykstop commented 6 years ago

0.10.0 QA Checklist

For both upgrades and fresh installs, here is a list of functionality that requires testing. You can use this for copy/pasting into your QA report. Feel free to edit this message to update the plan as appropriate.

If you have submitted a QA report already for a 0.10.0 release candidate with successful basic server testing and application acceptance testing sections, then you can skip these sections in subsequent reports, unless otherwise indicated by the Release Manager. This is to ensure that you focus your QA effort on the 0.10.0-specific changes as well as changes since the previous release candidate.

Environment

Install target: prod VMs
Tails version: 3.9.1
Test Scenario: Clean install
SSH over Tor: enabled
Release candidate: 0.10.0~rc2
General Notes: OSSEC emails not checked for VMs, will test on HW

Basic Server Testing

[x] I can access both the source and journalist interfaces
[x] I can SSH into both machines over Tor
[x] AppArmor is loaded on app
[x] AppArmor is loaded on mon
[x] Both servers are running grsec kernels
[x] iptables rules loaded
[ ] OSSEC emails begin to flow after install NOT TESTED
[ ] OSSEC emails are decrypted to correct key and I am able to decrypt them NOT TESTED

Command Line User Generation

[x] Can successfully add admin user and login

Administration

[ ] I have backed up and successfully restored the app server following the documentation here: https://docs.securedrop.org/en/latest/backup_and_restore.html FAILED :exclamation: #3878 created to track this bug
[ ] If doing upgrade testing, make a backup on 0.9.x and restore this backup on 0.10.0. Confirm that no further action is necessary and that (#3737) worked correctly. N/A

Application Acceptance Testing

Source Interface

Landing page base cases

[x] JS warning bar does not appear when using Security Slider high
[x] JS warning bar does appear when using Security Slider Low

First submission base cases

[x] On generate page, refreshing codename produces a new 7-word codename
[x] On submit page, empty submissions produce flashed message
[x] On submit page, short message submitted successfully
[x] On submit page, file greater than 500 MB produces "The connection was reset" in Tor Browser quickly before the entire file is uploaded
[x] On submit page, file less than 500 MB submitted successfully

Returning source base cases

[x] Nonexistent codename cannot log in
[x] Empty codename cannot log in
[x] Legitimate codename can log in
[x] Returning user can view journalist replies - need to log into journalist interface to test

Journalist Interface

Login base cases

[x] Can log in with 2FA tokens
[x] incorrect password cannot log in
[x] invalid 2fa token cannot log in
[x] 2fa immediate reuse cannot log in

Index base cases

[x] Filter by codename works
[x] Starring and unstarring works
[x] Click select all selects all submissions
[x] Selecting all and clicking "Download all" works There is no Download all

Individual source page

[x] You can submit a reply and a flashed message and new row appears
[x] You cannot submit an empty reply
[x] Clicking "Delete collection" and the source and docs are deleted it's Delete Source And Submissions
[ ] You can click on a document and successfully decrypt using application private key NOT TESTED`

Basic Tails Testing

Updater GUI

After updating to this release candidate and running securedrop-admin tailsconfig

[ ] The Updater GUI appears on boot FAILED ~/Persistent/securedrop not mounted on boot in Tails VM
[ ] Updating occurs without issue FAILED

0.10.0-specific changes

[x] OSSEC agent and server is 3.0.0 (#3701)
- [ ] Upgrade test: ensure that OSSEC alerts continue to flow after upgrade NOT TESTED
- [ ] Fresh install: ensure that OSSEC alerts flow without issue NOT TESTED
[ ] Tor is running 0.3.4.8 (#3813) FAILED tor update not in this release
[ ] (Upgrade only) Latest kernel version is selected for boot on a previously downgraded instance (#3857)

Preflight

These tests should be performed the day of release prior to live debian packages on apt.freedom.press

Basic testing

[ ] Install or upgrade occurs without error
[ ] Source interface is available and version string indicates it is 0.10.0
[ ] A message can be successfully submitted

Tails

[ ] The updater GUI appears on boot
[ ] The update successfully occurs to 0.10.0
[ ] After reboot, updater GUI no longer appears

zenmonkeykstop commented 6 years ago

0.10.0 QA Checklist

For both upgrades and fresh installs, here is a list of functionality that requires testing. You can use this for copy/pasting into your QA report. Feel free to edit this message to update the plan as appropriate.

If you have submitted a QA report already for a 0.10.0 release candidate with successful basic server testing and application acceptance testing sections, then you can skip these sections in subsequent reports, unless otherwise indicated by the Release Manager. This is to ensure that you focus your QA effort on the 0.10.0-specific changes as well as changes since the previous release candidate.

Environment

Install target: 2014 Mac Mini
Tails version: 3.9.1
Test Scenario: clean install
SSH over Tor: enabled
Release candidate: 0.10.0-rc2
General notes:

Basic Server Testing

[x] I can access both the source and journalist interfaces
[x] I can SSH into both machines over Tor
[x] AppArmor is loaded on app
[x] AppArmor is loaded on mon
[x] Both servers are running grsec kernels
[x] iptables rules loaded
[x] OSSEC emails begin to flow after install
[x] OSSEC emails are decrypted to correct key and I am able to decrypt them

Command Line User Generation

[x] Can successfully add admin user and login

Administration

[x] I have backed up and successfully restored the app server following the documentation here: https://docs.securedrop.org/en/latest/backup_and_restore.html
[ ] If doing upgrade testing, make a backup on 0.9.x and restore this backup on 0.10.0. Confirm that no further action is necessary and that (#3737) worked correctly.

Application Acceptance Testing

Source Interface

Landing page base cases

[x] JS warning bar does not appear when using Security Slider high
[x] JS warning bar does appear when using Security Slider Low

First submission base cases

[x] On generate page, refreshing codename produces a new 7-word codename
[x] On submit page, empty submissions produce flashed message
[x] On submit page, short message submitted successfully
[x] On submit page, file greater than 500 MB produces "The connection was reset" in Tor Browser quickly before the entire file is uploaded
[x] On submit page, file less than 500 MB submitted successfully

Returning source base cases

[x] Nonexistent codename cannot log in
[x] Empty codename cannot log in
[x] Legitimate codename can log in
[x] Returning user can view journalist replies - need to log into journalist interface to test

Journalist Interface

Login base cases

[x] Can log in with 2FA tokens
[x] incorrect password cannot log in
[x] invalid 2fa token cannot log in
[x] 2fa immediate reuse cannot log in

Index base cases

[x] Filter by codename works
[x] Starring and unstarring works
[x] Click select all selects all submissions
[x] Selecting all and clicking "Download" works

Individual source page

[x] You can submit a reply and a flashed message and new row appears
[x] You cannot submit an empty reply
[x] Clicking "Delete Source And Submissions" and the source and docs are deleted
[x] You can click on a document and successfully decrypt using application private key

Basic Tails Testing

Updater GUI

After updating to this release candidate and running securedrop-admin tailsconfig

[x] The Updater GUI appears on boot
[x] Updating occurs without issue after ensuring there were no staged changes

0.10.0-specific changes

[x] OSSEC agent and server is 3.0.0 (#3701)
- [ ] Upgrade test: ensure that OSSEC alerts continue to flow after upgrade
- [ ] Fresh install: ensure that OSSEC alerts flow without issue - Alerts flow but message format has changed, filed #3882 to cover this
[ ] (Upgrade only) Latest kernel version is selected for boot on a previously downgraded instance (#3857)

Preflight

These tests should be performed the day of release prior to live debian packages on apt.freedom.press

Basic testing

[ ] Install or upgrade occurs without error
[ ] Source interface is available and version string indicates it is 0.10.0
[ ] A message can be successfully submitted

Tails

[ ] The updater GUI appears on boot
[ ] The update successfully occurs to 0.10.0
[ ] After reboot, updater GUI no longer appears

emkll commented 6 years ago

0.10.0 QA Checklist (Completed)

Environment

Install target: Intel NUC5PYH
Tails version: 3.9
Test Scenario: clean install
SSH over Tor: disabled
Release candidate: 0.10.0-rc2

Basic Server Testing

[x] I can access both the source and journalist interfaces
[x] I can SSH into both machines over Tor
[x] AppArmor is loaded on app
[x] AppArmor is loaded on mon
[x] Both servers are running grsec kernels
[x] iptables rules loaded
[x] OSSEC emails begin to flow after install
[x] OSSEC emails are decrypted to correct key and I am able to decrypt them

Command Line User Generation

[x] Can successfully add admin user and login

Administration

[ ] I have backed up and successfully restored the app server following the documentation here: https://docs.securedrop.org/en/latest/backup_and_restore.html
[ ] If doing upgrade testing, make a backup on 0.9.x and restore this backup on 0.10.0. Confirm that no further action is necessary and that (#3737) worked correctly.

Application Acceptance Testing

Source Interface

Landing page base cases

[x] JS warning bar does not appear when using Security Slider high
[x] JS warning bar does appear when using Security Slider Low

First submission base cases

[x] On generate page, refreshing codename produces a new 7-word codename
[x] On submit page, empty submissions produce flashed message
[x] On submit page, short message submitted successfully
[x] On submit page, file greater than 500 MB produces "The connection was reset" in Tor Browser quickly before the entire file is uploaded
[x] On submit page, file less than 500 MB submitted successfully

Returning source base cases

[x] Nonexistent codename cannot log in
[x] Empty codename cannot log in
[x] Legitimate codename can log in
[x] Returning user can view journalist replies - need to log into journalist interface to test

Journalist Interface

Login base cases

[x] Can log in with 2FA tokens
[x] incorrect password cannot log in
[x] invalid 2fa token cannot log in
[x] 2fa immediate reuse cannot log in

Index base cases

[x] Filter by codename works
[x] Starring and unstarring works
[x] Click select all selects all submissions
[x] Selecting all and clicking "Download" works

Individual source page

[x] You can submit a reply and a flashed message and new row appears
[x] You cannot submit an empty reply
[x] Clicking "Delete Source And Submissions" and the source and docs are deleted
[x] You can click on a document and successfully decrypt using application private key

Basic Tails Testing

Updater GUI

After updating to this release candidate and running securedrop-admin tailsconfig

[x] The Updater GUI appears on boot
[ ] Updating occurs without issue after ensuring there were no staged changes

0.10.0-specific changes

[x] OSSEC agent and server is 3.0.0 (#3701)
- [x] Upgrade test: ensure that OSSEC alerts continue to flow after upgrade
- [x] Fresh install: ensure that OSSEC alerts flow without issue
[x] (Upgrade only) Latest kernel version is selected for boot on a previously downgraded instance (#3857)

robzim commented 6 years ago

Guys, I’m thinking this is an indication that SecureDrop 0.10 will ship soon. Rob

From: mickael e. [mailto:notifications@github.com] Sent: Wednesday, October 17, 2018 11:49 AM To: freedomofpress/securedrop securedrop@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: [EXTERNAL] Re: [freedomofpress/securedrop] Release SecureDrop 0.10.0 (#3849)

0.10.0 QA Checklist Environment

Install target: Intel NUC5PYH
Tails version: 3.9
Test Scenario: clean install
SSH over Tor: disabled
Release candidate: 0.10.0-rc2

Basic Server Testing

[X] I can access both the source and journalist interfaces
[X] I can SSH into both machines over Tor
[X] AppArmor is loaded on app
[X] AppArmor is loaded on mon
[X] Both servers are running grsec kernels
[X] iptables rules loaded
[X] OSSEC emails begin to flow after install
[X] OSSEC emails are decrypted to correct key and I am able to decrypt them

Command Line User Generation

[X] Can successfully add admin user and login

Administration

[ ] I have backed up and successfully restored the app server following the documentation here: https://docs.securedrop.org/en/latest/backup_and_restore.html
[ ] If doing upgrade testing, make a backup on 0.9.x and restore this backup on 0.10.0. Confirm that no further action is necessary and that (#3737https://github.com/freedomofpress/securedrop/pull/3737) worked correctly.

Application Acceptance Testing Source Interface Landing page base cases

[X] JS warning bar does not appear when using Security Slider high
[X] JS warning bar does appear when using Security Slider Low

First submission base cases

[X] On generate page, refreshing codename produces a new 7-word codename
[X] On submit page, empty submissions produce flashed message
[X] On submit page, short message submitted successfully
[X] On submit page, file greater than 500 MB produces "The connection was reset" in Tor Browser quickly before the entire file is uploaded
[X] On submit page, file less than 500 MB submitted successfully

Returning source base cases

[X] Nonexistent codename cannot log in
[X] Empty codename cannot log in
[X] Legitimate codename can log in
[X] Returning user can view journalist replies - need to log into journalist interface to test

Journalist Interface Login base cases

[X] Can log in with 2FA tokens
[X] incorrect password cannot log in
[X] invalid 2fa token cannot log in
[X] 2fa immediate reuse cannot log in

Index base cases

[X] Filter by codename works
[X] Starring and unstarring works
[X] Click select all selects all submissions
[X] Selecting all and clicking "Download" works

Individual source page

[X] You can submit a reply and a flashed message and new row appears
[X] You cannot submit an empty reply
[X] Clicking "Delete Source And Submissions" and the source and docs are deleted
[X] You can click on a document and successfully decrypt using application private key

Basic Tails Testing Updater GUI

After updating to this release candidate and running securedrop-admin tailsconfig

[ ] The Updater GUI appears on boot
[ ] Updating occurs without issue after ensuring there were no staged changes

0.10.0-specific changes

[X] OSSEC agent and server is 3.0.0 (#3701https://github.com/freedomofpress/securedrop/issues/3701)
- [ ] Upgrade test: ensure that OSSEC alerts continue to flow after upgrade
- [X] Fresh install: ensure that OSSEC alerts flow without issue
[ ] (Upgrade only) Latest kernel version is selected for boot on a previously downgraded instance (#3857https://github.com/freedomofpress/securedrop/pull/3857)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/freedomofpress/securedrop/issues/3849#issuecomment-430683101, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AKBwx3HZ1J8oIARHc8QaZuo26FLhiC1Dks5ul1F_gaJpZM4XK_dU.

zenmonkeykstop commented 6 years ago

@robzim :crossed_fingers:

zenmonkeykstop commented 6 years ago

0.10.0 QA Checklist

For both upgrades and fresh installs, here is a list of functionality that requires testing. You can use this for copy/pasting into your QA report. Feel free to edit this message to update the plan as appropriate.

If you have submitted a QA report already for a 0.10.0 release candidate with successful basic server testing and application acceptance testing sections, then you can skip these sections in subsequent reports, unless otherwise indicated by the Release Manager. This is to ensure that you focus your QA effort on the 0.10.0-specific changes as well as changes since the previous release candidate.

Environment

Install target: 2014 Mac Minis
Tails version: 3.9.1
Test Scenario: 0.9.1->0.10.0 upgrade
SSH over Tor: enabled
Release candidate: 0.10.0~rc2
General notes: app and mon downgraded to 3.14.79 before upgrade

Basic Server Testing

[x] I can access both the source and journalist interfaces
[x] I can SSH into both machines over Tor
[x] AppArmor is loaded on app
[x] AppArmor is loaded on mon
[x] Both servers are running grsec kernels
[x] iptables rules loaded
[x] OSSEC emails begin to flow after install
[x] OSSEC emails are decrypted to correct key and I am able to decrypt them

Command Line User Generation

[x] Can successfully add admin user and login

Administration

[x] I have backed up and successfully restored the app server following the documentation here: https://docs.securedrop.org/en/latest/backup_and_restore.html
[x] If doing upgrade testing, make a backup on 0.9.x and restore this backup on 0.10.0. Confirm that no further action is necessary and that (#3737) worked correctly.

Application Acceptance Testing

Source Interface

Landing page base cases

[x] JS warning bar does not appear when using Security Slider high
[x] JS warning bar does appear when using Security Slider Low

First submission base cases

[x] On generate page, refreshing codename produces a new 7-word codename
[x] On submit page, empty submissions produce flashed message
[x] On submit page, short message submitted successfully
[x] On submit page, file greater than 500 MB produces "The connection was reset" in Tor Browser quickly before the entire file is uploaded
[x] On submit page, file less than 500 MB submitted successfully

Returning source base cases

[x] Nonexistent codename cannot log in
[x] Empty codename cannot log in
[x] Legitimate codename can log in
[x] Returning user can view journalist replies - need to log into journalist interface to test

Journalist Interface

Login base cases

[x] Can log in with 2FA tokens
[x] incorrect password cannot log in
[x] invalid 2fa token cannot log in
[x] 2fa immediate reuse cannot log in

Index base cases

[x] Filter by codename works
[x] Starring and unstarring works
[x] Click select all selects all submissions
[x] Selecting all and clicking "Download" works

Individual source page

[x] You can submit a reply and a flashed message and new row appears
[x] You cannot submit an empty reply
[x] Clicking "Delete Source And Submissions" and the source and docs are deleted
[x] You can click on a document and successfully decrypt using application private key

Basic Tails Testing

Updater GUI

After updating to this release candidate and running securedrop-admin tailsconfig

[x] The Updater GUI appears on boot
[x] Updating occurs without issue

0.10.0-specific changes

[x] OSSEC agent and server is 3.0.0 (#3701)
- [x] Upgrade test: ensure that OSSEC alerts continue to flow after upgrade
- [ ] Fresh install: ensure that OSSEC alerts flow without issue
[x] (Upgrade only) Latest kernel version is selected for boot on a previously downgraded instance (#3857)

Preflight

These tests should be performed the day of release prior to live debian packages on apt.freedom.press

Basic testing

[ ] Install or upgrade occurs without error
[ ] Source interface is available and version string indicates it is 0.10.0
[ ] A message can be successfully submitted

Tails

[ ] The updater GUI appears on boot
[ ] The update successfully occurs to 0.10.0
[ ] After reboot, updater GUI no longer appears

zenmonkeykstop commented 6 years ago

0.10.0 QA Checklist

For both upgrades and fresh installs, here is a list of functionality that requires testing. You can use this for copy/pasting into your QA report. Feel free to edit this message to update the plan as appropriate.

If you have submitted a QA report already for a 0.10.0 release candidate with successful basic server testing and application acceptance testing sections, then you can skip these sections in subsequent reports, unless otherwise indicated by the Release Manager. This is to ensure that you focus your QA effort on the 0.10.0-specific changes as well as changes since the previous release candidate.

Environment

Install target: Intel NUCs
Tails version: 3.9.1
Test Scenario: upgrade from 0.9.1
SSH over Tor: enabled
Release candidate: 0.10.0~rc3
General notes:

Basic Server Testing

[x] I can access both the source and journalist interfaces
[x] I can SSH into both machines over Tor
[x] AppArmor is loaded on app
[x] AppArmor is loaded on mon
[x] Both servers are running grsec kernels
[x] iptables rules loaded
[x] OSSEC emails begin to flow after install
[x] OSSEC emails are decrypted to correct key and I am able to decrypt them

Command Line User Generation

[x] Can successfully add admin user and login

Administration

[x] I have backed up and successfully restored the app server following the documentation here: https://docs.securedrop.org/en/latest/backup_and_restore.html
[x] If doing upgrade testing, make a backup on 0.9.x and restore this backup on 0.10.0. Confirm that no further action is necessary and that (#3737) worked correctly.

Application Acceptance Testing

Source Interface

Landing page base cases

[x] JS warning bar does not appear when using Security Slider high
[x] JS warning bar does appear when using Security Slider Low

First submission base cases

[x] On generate page, refreshing codename produces a new 7-word codename
[x] On submit page, empty submissions produce flashed message
[x] On submit page, short message submitted successfully
[x] On submit page, file greater than 500 MB produces "The connection was reset" in Tor Browser quickly before the entire file is uploaded
[x] On submit page, file less than 500 MB submitted successfully

Returning source base cases

[x] Nonexistent codename cannot log in
[x] Empty codename cannot log in
[x] Legitimate codename can log in
[x] Returning user can view journalist replies - need to log into journalist interface to test

Journalist Interface

Login base cases

[x] Can log in with 2FA tokens
[x] incorrect password cannot log in
[x] invalid 2fa token cannot log in
[x] 2fa immediate reuse cannot log in

Index base cases

[x] Filter by codename works
[x] Starring and unstarring works
[x] Click select all selects all submissions
[x] Selecting all and clicking "Download" works

Individual source page

[x] You can submit a reply and a flashed message and new row appears
[x] You cannot submit an empty reply
[x] Clicking "Delete Source And Submissions" and the source and docs are deleted
[x] You can click on a document and successfully decrypt using application private key

Basic Tails Testing

Updater GUI

After updating to this release candidate and running securedrop-admin tailsconfig

[x] The Updater GUI appears on boot
[x] Updating occurs without issue

0.10.0-specific changes

[x] OSSEC agent and server is 3.0.0 (#3701)
- [x] Upgrade test: ensure that OSSEC alerts continue to flow after upgrade
- [ ] Fresh install: ensure that OSSEC alerts flow without issue
[x] (Upgrade only) Latest kernel version is selected for boot on a previously downgraded instance (#3857)

0.10.0-rc3-specific changes

[x] OSSEC emails do not contain alert type (see https://github.com/freedomofpress/securedrop/issues/3882)

Preflight

[ ] Ensure the builder image is up-to-date on release day

These tests should be performed the day of release prior to live debian packages on apt.freedom.press

Basic testing

[ ] Install or upgrade occurs without error
[ ] Source interface is available and version string indicates it is 0.10.0
[ ] A message can be successfully submitted

Tails

[ ] The updater GUI appears on boot
[ ] The update successfully occurs to 0.10.0
[ ] After reboot, updater GUI no longer appears

conorsch commented 6 years ago

0.10.0 QA Checklist

Performed a limited upgrade test on two (2) 1U form-factor servers, specifically to confirm upgrade to latest kernel from rolled back kernel (3.14.79-grsec). Testing was a success. Did not perform the general application testing steps, since those are well covered already by the reports above.

Environment

Install target: Dell PowerEdge R620 (for app), HP ProLiant DL385 G7 (for mon)
Tails version: 3.9.1
Test Scenario: upgrade from 0.9.1
SSH over Tor: enabled
Release candidate: 0.10.0~rc3
General notes:

Basic Server Testing

[x] I can access both the source and journalist interfaces
[x] I can SSH into both machines over Tor
[x] AppArmor is loaded on app
[x] AppArmor is loaded on mon
[x] Both servers are running grsec kernels
[x] iptables rules loaded
[x] OSSEC emails begin to flow after install
[x] OSSEC emails are decrypted to correct key and I am able to decrypt them

Relevant terminal output

For validating kernel -> downgrade -> upgrade process. Manually activated virtualenv, then ran through a few steps for informative output. Confirming the latest kernels are running on 0.9.1:

(.venv) amnesia@amnesia:~/Persistent/securedrop/install_files/ansible-base$ ansible all -a 'uname -r'
app | SUCCESS | rc=0 >>
4.4.144-grsec

mon | SUCCESS | rc=0 >>
4.4.144-grsec

Display hardware type for each server, to aid debugging if problems arise:

(.venv) amnesia@amnesia:~/Persistent/securedrop/install_files/ansible-base$ ansible all -m setup -a filter=ansible_product_name -b
app | SUCCESS => {
    "ansible_facts": {
        "ansible_product_name": "PowerEdge R620"
    }, 
    "changed": false
}
mon | SUCCESS => {
    "ansible_facts": {
        "ansible_product_name": "ProLiant DL385 G7"
    }, 
    "changed": false
}

Followed the docs and manually rolled back both servers to 3.14.79-grsec. Worked:

(.venv) amnesia@amnesia:~/Persistent/securedrop/install_files/ansible-base$ sleep 300 && ansible all -a 'uname -r'
app | SUCCESS | rc=0 >>
3.14.79-grsec

mon | SUCCESS | rc=0 >>
3.14.79-grsec

Ran QA playbook, cron-apt command, then rebooted via CLI to test the kernel update procedure.

(.venv) amnesia@amnesia:~/Persistent/securedrop/install_files/ansible-base$ ansible all -a 'uname -r'
app | SUCCESS | rc=0 >>
4.4.144-grsec

mon | SUCCESS | rc=0 >>
4.4.144-grsec

Success!

emkll commented 6 years ago

0.10.0-rc4 (Completed)

Environment

Install target: NUC 5PYH
Tails version: 3.9
Test Scenario: Upgrade from 0.9.1
SSH over Tor: Yes
Release candidate: rc4
General notes: Downgraded kernels to 3.14 prior to updating using /etc/default/grub

Basic Server Testing

[x] I can access both the source and journalist interfaces
[x] I can SSH into both machines over Tor
[x] AppArmor is loaded on app
[x] AppArmor is loaded on mon
[x] Both servers are running grsec kernels
[x] iptables rules loaded
[x] OSSEC emails begin to flow after install
[x] OSSEC emails are decrypted to correct key and I am able to decrypt them

Command Line User Generation

[x] Can successfully add admin user and login

Administration

[x] I have backed up and successfully restored the app server following the documentation here: https://docs.securedrop.org/en/latest/backup_and_restore.html
[x] If doing upgrade testing, make a backup on 0.9.x and restore this backup on 0.10.0. Confirm that no further action is necessary and that (#3737) worked correctly.

Application Acceptance Testing

Source Interface

Landing page base cases

[x] JS warning bar does not appear when using Security Slider high
[x] JS warning bar does appear when using Security Slider Low

First submission base cases

[x] On generate page, refreshing codename produces a new 7-word codename
[x] On submit page, empty submissions produce flashed message
[x] On submit page, short message submitted successfully
[x] On submit page, file greater than 500 MB produces "The connection was reset" in Tor Browser quickly before the entire file is uploaded
[x] On submit page, file less than 500 MB submitted successfully

Returning source base cases

[x] Nonexistent codename cannot log in
[x] Empty codename cannot log in
[x] Legitimate codename can log in
[x] Returning user can view journalist replies - need to log into journalist interface to test

Journalist Interface

Login base cases

[x] Can log in with 2FA tokens
[x] incorrect password cannot log in
[x] invalid 2fa token cannot log in
[x] 2fa immediate reuse cannot log in

Index base cases

[x] Filter by codename works
[x] Starring and unstarring works
[x] Click select all selects all submissions
[x] Selecting all and clicking "Download" works

Individual source page

[x] You can submit a reply and a flashed message and new row appears
[x] You cannot submit an empty reply
[x] Clicking "Delete Source And Submissions" and the source and docs are deleted
[x] You can click on a document and successfully decrypt using application private key

Basic Tails Testing

Updater GUI

After updating to this release candidate and running securedrop-admin tailsconfig

[x] The Updater GUI appears on boot
[ ] Updating occurs without issue

0.10.0-specific changes

[x] OSSEC agent and server is 3.0.0 (#3701)
- [x] Upgrade test: ensure that OSSEC alerts continue to flow after upgrade
- [x] Fresh install: ensure that OSSEC alerts flow without issue
[x] (Upgrade only) Latest kernel version is selected for boot on a previously downgraded instance (#3857)

0.10.0-rc3-specific changes

[x] OSSEC emails do not contain alert type (see https://github.com/freedomofpress/securedrop/issues/3882)

0.10.0-rc4-specific changes

[x] Ensure a source cannot delete a journalist reply for which they are not authorized (see https://github.com/freedomofpress/securedrop/pull/3893)

zenmonkeykstop commented 6 years ago

0.10.0 QA Checklist

For both upgrades and fresh installs, here is a list of functionality that requires testing. You can use this for copy/pasting into your QA report. Feel free to edit this message to update the plan as appropriate.

If you have submitted a QA report already for a 0.10.0 release candidate with successful basic server testing and application acceptance testing sections, then you can skip these sections in subsequent reports, unless otherwise indicated by the Release Manager. This is to ensure that you focus your QA effort on the 0.10.0-specific changes as well as changes since the previous release candidate.

Environment

Install target: 2014 Mac Mini
Tails version: 3.9.1
Test Scenario: upgrade from 0.9.1
SSH over Tor: enabled
Release candidate: 0.10.0-rc4
General notes:

Basic Server Testing

[x] I can access both the source and journalist interfaces
[x] I can SSH into both machines over Tor
[x] AppArmor is loaded on app
[x] AppArmor is loaded on mon
[x] Both servers are running grsec kernels
[x] iptables rules loaded
[x] OSSEC emails begin to flow after install
[x] OSSEC emails are decrypted to correct key and I am able to decrypt them

Command Line User Generation

[x] Can successfully add admin user and login

Administration

[x] I have backed up and successfully restored the app server following the documentation here: https://docs.securedrop.org/en/latest/backup_and_restore.html
[x] If doing upgrade testing, make a backup on 0.9.x and restore this backup on 0.10.0. Confirm that no further action is necessary and that (#3737) worked correctly.

Application Acceptance Testing

Source Interface

Landing page base cases

[x] JS warning bar does not appear when using Security Slider high
[x] JS warning bar does appear when using Security Slider Low

First submission base cases

[x] On generate page, refreshing codename produces a new 7-word codename
[x] On submit page, empty submissions produce flashed message
[x] On submit page, short message submitted successfully
[x] On submit page, file greater than 500 MB produces "The connection was reset" in Tor Browser quickly before the entire file is uploaded
[x] On submit page, file less than 500 MB submitted successfully

Returning source base cases

[x] Nonexistent codename cannot log in
[x] Empty codename cannot log in
[x] Legitimate codename can log in
[x] Returning user can view journalist replies - need to log into journalist interface to test

Journalist Interface

Login base cases

[x] Can log in with 2FA tokens
[x] incorrect password cannot log in
[x] invalid 2fa token cannot log in
[x] 2fa immediate reuse cannot log in

Index base cases

[x] Filter by codename works
[x] Starring and unstarring works
[x] Click select all selects all submissions
[x] Selecting all and clicking "Download" works

Individual source page

[x] You can submit a reply and a flashed message and new row appears
[x] You cannot submit an empty reply
[x] Clicking "Delete Source And Submissions" and the source and docs are deleted
[x] You can click on a document and successfully decrypt using application private key

Basic Tails Testing

Updater GUI

After updating to this release candidate and running securedrop-admin tailsconfig

[x] The Updater GUI appears on boot
[x] Updating occurs without issue

0.10.0-specific changes

[x] OSSEC agent and server is 3.0.0 (#3701)
- [x] Upgrade test: ensure that OSSEC alerts continue to flow after upgrade
- [ ] Fresh install: ensure that OSSEC alerts flow without issue
[x] (Upgrade only) Latest kernel version is selected for boot on a previously downgraded instance (#3857)

0.10.0-rc3-specific changes

[x] OSSEC emails do not contain alert type (see https://github.com/freedomofpress/securedrop/issues/3882)

0.10.0-rc4-specific changes

[x] Ensure a source cannot delete a journalist reply for which they are not authorized (see https://github.com/freedomofpress/securedrop/pull/3893)

Preflight
[ ] Ensure the builder image is up-to-date on release day

These tests should be performed the day of release prior to live debian packages on apt.freedom.press

Basic testing

[ ] Install or upgrade occurs without error
[ ] Source interface is available and version string indicates it is 0.10.0
[ ] A message can be successfully submitted

Tails

[ ] The updater GUI appears on boot
[ ] The update successfully occurs to 0.10.0
[ ] After reboot, updater GUI no longer appears

freedomofpress / securedrop

Release SecureDrop 0.10.0 #3849

Prepare release candidate (0.10.0~rc1)

Prepare release candidate (0.10.0~rc2)

Prepare release candidate (0.10.0~rc3)

Prepare release candidate (0.10.0~rc4)

QA Matrix for 0.10.0

Test Plan for 0.10.0

Final release

Post release

0.10.0 QA Checklist

Environment

Basic Server Testing

Command Line User Generation

Administration

Application Acceptance Testing

Source Interface

Landing page base cases

First submission base cases

Returning source base cases

Journalist Interface

Login base cases

Index base cases

Individual source page

Basic Tails Testing

Updater GUI

0.10.0-specific changes

Preflight

Basic testing

Tails

0.10.0 QA Checklist

Environment

Basic Server Testing

Command Line User Generation

Administration

Application Acceptance Testing

Source Interface

Landing page base cases

First submission base cases

Returning source base cases

Journalist Interface

Login base cases

Index base cases

Individual source page

Basic Tails Testing

Updater GUI

0.10.0-specific changes

Preflight

Basic testing

Tails

0.10.0 QA Checklist (Completed)

Environment

Basic Server Testing

Command Line User Generation

Administration

Application Acceptance Testing

Source Interface

Landing page base cases

First submission base cases

Returning source base cases

Journalist Interface

Login base cases

Index base cases

Individual source page

Basic Tails Testing

Updater GUI

0.10.0-specific changes

0.10.0 QA Checklist

Environment

Basic Server Testing

Command Line User Generation

Administration

Application Acceptance Testing

Source Interface

Landing page base cases

First submission base cases

Returning source base cases

Journalist Interface

Login base cases

Index base cases