search

whole-tale / wt-design-docs

MIT License
5 stars 9 forks source link

v1.2rc1 testing (Chrome/MacOS) #171

Open craig-willis opened 2 years ago

craig-willis commented 2 years ago

Test plan walkthrough on Staging

Test Plan

https://github.com/whole-tale/wt-design-docs/issues/new?template=TEST_PLAN.md

Note: For all tests, repeat for supported browser/OS combinations.

Preconditions:

End-to-End Test

Notes:

Unauthenticated access

Signing in

Navigation

Tale Dashboard

Preconditions:

Managing Data

Preconditions

Run Tale

Preconditions:

Versions

Recorded Runs

Settings

Tale Creation

Analyze in Whole Tale

These test cases cover potential situations that can occur when importing datasets from Dataverse.

Tale metadata tests

The purpose of these tests are to confirm that the metadata files (manifest.json, environment.json, LICENSE) we generate are correct.

import urllib.parse urllib.parse.uses_relative.append('arcp') urllib.parse.uses_netloc.append('arcp')

g = Graph().parse(source="manifest.json", format="json-ld") print(g.serialize(format='turtle', indent=2).decode())


* [ ] environment.json
1. Open the archive from the manifest.json test
1. Navigate to `metadata/environment.json`
1. Open the girder web API
1. Locate and GET the Image that the Tale used
1. Confirm that the `config` section from girder matches the `config` section in `environment.json`

* [ ] LICENSE
1. Open the archive from the manifest.json test
1. Open the `LICENSE` file
1. Navigate to Run > metadata
1. Confirm that the opened `LICENSE` file matches the selected license on the metadata page
1. Change the Tale's license
1. Export the Tale
1. Confirm that the `LICENSE` changed appropriately

## Tale Sharing Tests

These tests are centered around the feature for sharing Tales with other Whole Tale
users. To run these tests, register at least one other account, referred to as User B.

* [x] Case 1: Sharing with 'Can View' permissions
  1. Create a new Tale
  1. Add data to the workspace and add at least one dataset
  1. Navigate to the 'Share' tab
  1. Share the Tale with User B
  1. Confirm that the default permission is 'Can View'
  1. Switch the account to User B
  1. Navigate to the 'Shared with Me' tab on the Browse page
  1. Confirm that you see the Tale from step 1
  1. Confirm that you can't edit the Workspace, External Data, or metadata
  1. Unshare tale with User B
  1. Note modal displays that tale has been unshared.

* [x] Case 2: Sharing with 'Can Edit' permissions
  1. Create a new Tale
  1. Add data to the workspace and add at least one dataset
  1. Navigate to the 'Share' tab
  1. Share the Tale with User B
  1. Set the permissions to 'Can Edit'
  1. Switch the account to User B
  1. Navigate to the 'Shared with Me' tab on the Browse page
  1. Confirm that you see the Tale from step 1
  1. Remove one of the files from the Workspace
  1. Add a dataset to External Data
  1. Change a field in the metadata (title, description, etc)
  1. Switch to User A
  1. Confirm that the changes made to the Tale persisted

* [x] Case 3: Edit sync modal
  1. Login as User A and User B in separate browser sessions
  1. As User B, select "Shared with me"
  1. As User A, share a tale with User B; as User B confirm the tale appears
  1. As User B, select "View" on the shared tale
  1. As User A and B edit the tale metadata
  1. As User A change title, author, category, description; save changes
  1. As User B confirm Tale edited modal, select Yes, confirm updates to metadata

## Publishing tests

* [x] Case 1: No Accounts Configured
  1. Disconnect all of your external accounts
  1. Open a Tale that you own
  1. Click the ellipsis dropdown on the Run page
  1. Click `Publish tale...`
  1. Confirm that your are notified to link accounts on the settings page
  1. Click `Go to Settings`
  1. Confirm you are re-directed to settings

* [x] Case 2: Publishing to DataONE
  1. Connect to DataONE (dev server)
  1. Open a Tale that you own
  1. Add some data to it
  1. Click `Publish tale...`
  1. Confirm that the publishing modal opens
  1. Confirm that the third party that you are connected to appears in the dropdown
  1. Select the third party
  1. Click `Publish`
  1. Once complete, navigate to the metadata page
  1. Confirm that the published location is present
  1. Visit the published location
  1. Confirm that you see the following files plus yours
      1. `metadata.xml`
      1. `LICENSE`
      1. `README.md`
      1. `manifest.json`
      1. `environment.json`
  1. After publishing to DataONE, use girder to query the Tale
  1. Ensure that the 'pid' field in the 'publishInfo' object has an identifier in a doi format

* [x] Case 3: Re-publish Tale to DataONE
1. Launch a Tale that has been published before
1. Take note of the `Published Location` in Run > metadata
1. Re-publish the Tale to the third party
1. Confirm that the `Published Location` has changed to the correct package landing page

## Zenodo integration tests

* [x] Register Zenodo data
1. On a new or existing tale, select "Files" > "External Data"
1. Select "+" > "Web (DOI or URL)"
1. Search for Concept DOI `doi:10.5281/zenodo.16384`
1. Confirm "Belmont Forum..." dataset is found on Zenodo with suffix `ver_1`
1. Search for URL  `https://zenodo.org/record/1172960#.XidIvlNKh24`
1. Confirm "Belmont Forum..." dataset is found on Zenodo with suffix `ver_2`
1. Search for DOI `doi:10.5281/zenodo.1172960`
1. Confirm "Belmont Forum..." dataset is found on Zenodo with suffix `ver_2`
1. Search for Concept DOI `doi:10.5281/zenodo.608984`
1. Confirm "Belmont Forum..." dataset is found on Zenodo with suffix `ver_2`
1. Select Register
1. Select "+" > "WT Data Catalog"
1. Add "Belmot Forum..." dataset to Tale
1. Run Tale and confirm contents on file system match Zenodo

The register tests the following cases.
   * Datasets with multiple files
   * Datasets with multiple versions
   * Concept DOI

* [x] Zenodo dataset via AiWT
1. Open https://dashboard.stage.wholetale.org/mine?uri=https%3A%2F%2Fdoi.org%2F10.5281%2Fzenodo.820575&name=Automotive%20Sensor%20Data
1. Confirm Source Data URL and Title match above URL and "READ ONLY" is selected
1. Select JupyterLab environment
1. Select 'Create New Tale'
1. Confirm dataset is mounted read-only, citation and related ids are as expected,
1. Run IDE and use terminal to `ls ../data/Auto*` to confirm zipfile is not extracted

The register tests the following cases.
   * Dataset with single file (zipfile)

* [x] Zenodo Binder via AiWT
1. Open https://dashboard.stage.wholetale.org/mine?uri=https%3A%2F%2Fdoi.org%2F10.5281%2Fzenodo.3242073&environment=JupyterLab&name=Simple%20requirements.txt%20based%20example%20for%20repo2docker&asTale=true
1. COnfirm Source Data and Title match URL and "READ WRITE" is selected
1. Select 'Create Tale'
1. Confirm image builds and notebook can run

* [x] Zenodo Tale via AiWT
1. Open https://girder.stage.wholetale.org/api/v1/integration/zenodo?doi=10.5281%2Fzenodo.490449&resource_server=sandbox.zenodo.org
1. Confirm Water Tale is created.
1. Confirm citation on Metadata view
1. Confirm DOI matches above
1. Use the Swagger UI to confirm the `publishInfo` field on the tale object contains pid, repository, repository_id, uri
1. Run the tale and and notebook, confirm it works as expected (produces glorious map of Texas)

* [x] Publish/import to/from Zenodo Sandbox
1. Follow instructions above to configure your API key for Zenodo sandbox
1. Create your own Tale and publish to sandbox.zenodo.org
1. Confirm publishInfo contains Sandbox entry
1. Delete tale
1. Import tale from Zenodo
1. Confirm publishInfo
1. Modify the tale
1. Re-publish to sandbox.zenodo.org
1. Delete tale
1. Import tale from Zenodo
1. Confirm publishInfo
1. Download and extract the exported Tale
1. Execute the `run-local.sh` command to run the Tale locally

## OpenICPSR integration tests

* [ ] OpenICPSR via AinWT
1. Go to Settings, configure ICPSR provider.  Select "www.openicpsr.org". Provide you password for icpsr login. (Assumption: your globus/girder account uses the same email as icpsr entity.)
1. Go to: https://dashboard.stage.wholetale.org/mine?name=AEAREP-3198-Stata&environment=STATA%2016%20%28Desktop%29&uri=https%3A%2F%2Fwww.openicpsr.org%2Fopenicpsr%2Fproject%2F132081%2Fversion%2FV1%2Fview&asTale=true
1. Confirm import

* [ ] Register OpenICPSR dataset?
1. This isn't an officially supported use case, but register https://www.openicpsr.org/openicpsr/project/132081/version/V1/view and confirm it works.

## Git Integration Tests

* [x] Compose Tale from Git Repository
Covered by other tests
1. Select the Create New Tale button on the 'Browse' page
1. Select the `Create Tale from Git Repository` option
1. Paste a link to a Git repository in the correct field
1. Add a name and pick a compute environment
1. Create the Tale
1. Confirm notification progress and success
1. Confirm that the git repository was cloned to the `workspace/` root
1. Run the tale
1. Confirm interaction via git tools (may require environment customization)

* [x] Connect to Git
1. Create a new empty tale
1. Select "Connect to Git Repository" menu item from Tale menu
1. Confirm notification progress and success
1. Once complete, navigate to the Tale workspace and confirm that the repository has been cloned

## Other tests

* [x] Citations
1. Add files from multiple external datasets (e.g.: doi:10.5281/zenodo.3381658, doi:10.5065/D6862DM8, doi:10.5063/F10K26VR)
1. Confirm `Datasets Used` citations and related identifiers are updated

### Event Notification Tests
* [x] Tale sharing/unsharing
1. Login as user1 and user2 in separate browser sessions
1. As user2, select "Shared with me"
1. As user1, share a tale with user2; as user2 confirm the tale appears
1. As user2, select "View" on the shared tale
1. As user1 and 2 edit the tale metadata
1. As user1 save changes; as user2 confirm Tale edited modal
1. As user1, unshare the tale; as user2 confirm Tale unshared modal
1. As user1, re-share with user2
1. As user2, select "Shared with Me"
1. As user1, delete the shared tale; as user2 confirm that the tale disappears

* [x] Import tale
1. Import a tale from Github, Zenodo, and as Binder
1. Confirm metadata updates accordingly after import completes

## Vocabulary Tests

* [ ] Vocabularies
1. Go to https://vocabularies.stage.wholetale.org/wt/1.0/index.html
1. Confirm that it matches https://whole-tale.github.io/serialization-format/wt/1.0/

## Forward Authentication Test
* [x] Forward authentication
1. Start a tale
1. Confirm that the tale is accessible via iframe and popout
1. Right click popout icon and copy URL (this is necessary to get JupyterLab token)
1. Open an incognito window and paste the copied URL
1. Confirm that you are prompted for authentication and can access the tale once authenciated
1. Login as a user who does not have permission to access the tale
1. Confirm that they receive an access denied errror

## Changing the Environment
* [x] Changing the environment
1. Create a new JupyterLab tale
1. Run the Tale, confirm JupyterLab IDE displays
1. Edit metadata, change environment to "RStudio"
1. Confirm icon changes in title bar and Environment section
1. Stop/run tale or rebuild/restart and confirm RStudio IDE displays

## Idle Instances
* [ ] Idle instance culling
1. Create and start a MATLAB tale and a JupyterLab tale
1. Confirm both instances start and IDEs are accessible
1. Close all IDE tabs/windows
1. Wait for 2 hours and 24 hours respectively
1. Confirm both instances are stopped automatically

## Upload folder
* [ ] Upload folder to home
1. Create local folder containing files, folders, etc.
1. Upload folder to home. Confirm contents match local folder.
1. Download folder. Confirm contents match.
1. Repeat in workspace

## Advanced settings
* [x] Advanced settings
1. Create a new tale
1. Edit Metadata > Advanced, enter:

{ "environment": [ "MY_ENV=value" ], "memLimit":"12gb" }

  1. Start tale
  2. cat /sys/fs/cgroup/memory.max, confirm 12gb
  3. echo $MY_ENV, confirm value

Create Tale from DOI

Public image registry

craig-willis commented 2 years ago

Issues:

Can't repeat:

Observations: