Test results for WholeTale v0.7.0rc3 (Firefox)

[ThomasThelen]

This is the Firefox counterpart to #84

Authentication

• [x] Basic login flow

  1. Goto https://dashboard.wholetale.org
  2. Select "Access Whole Tale" button. You should be prompted to login via Globus Auth.
  3. Select your organization and login
  4. You should be taken to the Browse page
  5. Your Gravatar and user name should display

• [x] Basic logout flow

  1. Select the "Logout" button. You should be taken to the login page.
  2. Select "Access Whole Tale" button.
  3. Note that you are not prompted to login.
  4. Note PR #331 for issues with logout
  5. Note issue #384 for logout for Globus
  6. Restart your incognito session
  7. Select "Access Whole Tale" button. You should be prompted to login via Globus Auth.

• [x] Return-route for non-logged in users (#300)

  1. Logout from Whole Tale
  2. Attempt to access https://dashboard.dev.wholetale.org/run
  3. You should be redirected to https://dashboard.dev.wholetale.org/login?rd=%252Frun
  4. Select "Access Whole Tale".
  5. You should be redirected to the Run page.

Navigation

• [x] General navigation
  1. Select "Browse" button. Browse page should display
  2. Select "Run" button. Run page should display
  3. Select "Manage" button. Manage page should display
  4. Select "Compose" button. Compose page should display
  5. Select "i" button should open User Guide in new tab
  6. Select "Logout" button should logout from WT
  7. Select "Report a problem" button should open https://github.com/whole-tale/whole-tale/issues in new tab

Browse

Preconditions:

• Assumes production Tales present (e.g., LIGO, materials, etc).

• No running instances

• [x] General

  1. Select "Browse" button. Confirm browse page displays
  2. Confirm "i" button opens https://wholetale.readthedocs.io/en/stable/users_guide/browse.html
  3. Confirm "fullscreen" button opens Browse panel in fullscreen
  4. Confirm no tales in "Launched Tales" panel

• [x] Search

  1. Enter "Ligo" as search term. Confirm only LIGO tale displays
  2. Clear search term. Confirm all tales display

• [x] Filter

  1. Confirm filter dropdown is disabled
  2. Select "Filter" icon. Confirm that filter dropdown is enabled
  3. Select "All". Confirm that all tales are displayed
  4. Select "Mine". Confirm that only your tales are displayed
  5. Select "Published". Confirm that only published tales are displayed (no tales)
  6. Select "Recent". (What is supposed to happen here?)
  7. Select Run then Browse. Note that selection is not retained.

• [x] List and card views

  1. Select "Switch to list view". Confirm list view displays
  2. Select "Switch to card view". Confirm card view displays
  3. Select Run then Browse. Note that selection is not retained.

• [ ] View tale

  1. Select "View" button on LIGO Tale
  2. Confirm View page displays
  3. Confirm that you cannot delete or edit Tale properties
  4. Select Back button. Confirm that you are taken back to Browse page
  5. Select "View" button on Tale you own
  6. Confirm Delete button is present
  7. Confirm that you can edit Tale properties
  8. Select "Delete". Confirm that you are prompted to delete.
  9. Select "No" -- tale is not deleted
  10. Select "Yes" -- tale is deleted and you are retured to Browse view

• [x] Launch instance

  1. Select "Launch" button on "Ligo" tale
  2. Confirm that tale appears in "Launched Tales" panel
  3. Confirm spinner displays until Tale is ready
  4. Confirm "Tale Launched" notification displays on Tale card
  5. Confirm that the toast notification appears, it should say BUILDING
  6. Confirm that then the instance is started and can be selected in the Launched Tales panel
  7. Select "X" to delete the instance
  8. Confirmation dialog displays.
  9. Select "No" -- instance is not deleted
  10. Select "Yes" -- instance is deleted
  11. Select "Launch" button on Tale you own
  12. After the instance is launched, try to delete the Tale (not instance)
  13. Confirm cannot Delete Tale message displays
  14. After instance has started, select the instance in the Launched Tales panel
  15. Confirm that you are taken to the Run > Interact page for the instance
  16. Select "Launch" on the already-launched instance
  17. Confirm "The Tale...has been Launched!" message displays"
  18. Select "Go to Tale". Confirm that you are taken to the Run > Interact page.

Manage

Preconditions

• Empty home directory

• No registered data

• [x] Register General

  1. Select Manage > Data > "+" to open register panel
  2. Confirm list of providers matches list of supported providers (Globus, DataONE, Dataverse)

• [x] Register DataONE data

  1. Select Manage > Data tab
  2. Select "+". Confirm "Search for Data Sources" modal displays
  3. Search for DOI 10.5065/D6862DM8
  4. Confirm "Humans and Hydrology" dataset is found on DataONE.
  5. Select Register
  6. Confirm dataset appears in Manage Data list
  7. Confirm additional URL formats:
  8. https://search.dataone.org/view/doi:10.5063/F1Z60M87
  9. https://search.dataone.org/view/https://pasta.lternet.edu/package/metadata/eml/knb-lter-nwt/115/1
  10. https://search.dataone.org/view/doi:10.18739/A2CN6XZ7H
  11. https://raw.githubusercontent.com/whole-tale/dashboard/master/.travis.yml

• [ ] Dataverse

  1. Register dataset doi:10.7910/DVN/SZ9YXZ
  2. Confirm "Replication Data for: Direct Democracy and Women's Political Engagement" from Dataverse

• [x] Globus/MDF

  1. Confirm DOI resolution for http://dx.doi.org/doi:10.18126/M2301J results in https://publish.globus.org/jspui/handle/ITEM/113
  2. Register dataset https://publish.globus.org/jspui/handle/ITEM/113

Compose

• [x] Compose RStudio Tale

  1. Select Compose. Confirm Compose page displayed with Launch New Tale button disabled.
  2. Enter title "Test Tale"
  3. Select compute environment RStudio
  4. Select "Launch New Tale"
  5. Launching new Tale spinner displays
  6. "Building tale image" toaster notification appears
  7. After instance is started, you are redirected to Run page for instance

• [x] Compose Jupyter Tale

• [x] Compose OpenRefine Tale

• [x] Too many instances

  1. Make sure you have two launched Tales
  2. Attempt to create a Tale
  3. Confirm that you receive an error that you have the max limit

Run

Preconditions:

• No running Tale instances

• [x] General

  1. Go to the Run page
  2. Confirm "Choose from Launched Tales panel" displays
  3. Confirm no Tales in Launched Tales panel
  4. Launch the Ligo Tale and wait for instance to start
  5. Select the "Run" Menu.
  6. Confirm LIGO Tale instance is displayed in Run and Launched Tale panels
  7. Confirm Run panel displays Tale icon, title, and author
  8. Confirm Run page displays Interact, Files, Metadata tabs
  9. Select "..." menu.
  10. Read the docs displays https://wholetale.readthedocs.io/en/stable/users_guide/run.html
  11. Confirm fullscreen displays Run panel in full screen
  12. Confirm that the options for exporting the Tale are present

• [x] Interact tab

  1. Select the "Interact" tab
  2. Confirm Jupyter environment is displayed in iframe
  3. Select "popout" icon. Confirm Jupyter is opened in new Tab

• [x] Files tab

  1. Select "Files" tab
  2. You should see a vertical navigation menu with three tabs: "Home", "External Data", and "Tale Workspace".

• [x] Metadata tab - Read Only Tale

  1. Launch the LIGO Tale
  2. Navigate to Run
  3. Select "Metadata" tab
  4. Confirm that metadata displays for current tale
  5. Confirm that the form elements are read only

• [x] Metadata tab - Owned Tale

  1. Launch a Tale that you own
  2. Add multiple datasets that have DOIs to the Tale
  3. Navigate to Run > metadata
  4. Valid environments should be listed in the Environment dropdown menu
  5. Valid licenses should be listed in the License dropdown
  6. Confirm that you see Created by <your_name> under Authors
  7. Confirm that your Tale has the CC4 license assigned to it
  8. Confirm that the Tale's Published Location reads This Tale has not been published
  9. Confirm that there are citations for the external datasets
  10. Change the name of the Tale, Category, Description, Illustration, Public, License, and add multiple authors
  11. Select Save. Confirm changes are saved.
  12. Refresh the page. Your changes should be preserved between page refreshes, confirming that they are being properly persisted.
  13. Confirm that the new Tale authors are displayed on the Tale card

• [x] Home

  1. Select Files > Home. Confirm that your home directory displays
  2. Create a folder and upload a file to the folder
  3. Confirm file operations:
  4. Copy, rename, remove, download

• [x] External data

  1. Precondition: Register one or more Datasets (e.g. doi:10.18739/A29G5GD0V) on Manage > Data (see Manage > Register test cases above)
     • NOTE: You may need to wait for registration to complete
  2. Confirm "External Data" list is initially empty
  3. Confirm (+) button ("Select Data modal") behavior
  4. Confirm that "WholeTale Catalog" shows all registered datasets
  5. Confirm that "My Data" filters to only show your registered datasets
  6. "Add" data to associate it with a Tale
  7. "Remove" data to dissociate it with a Tale
  8. "Select" saves your changes back to the server
  9. "Cancel" aborts the operation without saving and reverts your changes
  10. Confirm "External Data" list matches what was saved

• [x] Workspaces

  1. Confirm create folder
  2. Confirm rename folder/file
  3. Confirm upload file
  4. Confirm copy file/folder from/to Home
  5. Confirm move file/folder from/to Home
  6. Confirm remove file/folder
  7. Select "+" button has options "New folder", "Upload file", "Select Data"
  8. Folder operations
  9. Confirm context menu Move To, Rename, Copy to Home, Download, Remove
  10. Move To...
      1. Issue -- should this be enabled
  11. Confirm rename folder
  12. Copy to Home confirm folder copied to home
  13. Confirm download -- zip file
  14. Confirm remove
  15. File operations
  16. Upload a file
  17. Confirm context menu Move To, Rename, Copy to Home, Download, Remove
  18. Move To... Home?
      1. Issue -- doesn't work
  19. Confirm rename folder
  20. Copy to Home confirm folder copied to home
  21. Confirm download
  22. Confirm remove
  23. Select "Select Data..."
  24. Note vertical navigation with Home and Tale Workspaces
  25. Copy to Workspace from Home
  26. Move to Workspace from Home
  27. Copy to Workspace from another workspace
  28. Move to workspace from another workspace

• [x] Exporting - ZIP

  1. Launch a Tale that you own
  2. Navigate to Run
  3. Click the ... dropdown
  4. Select Export as Zip
  5. Confirm that you are asked to start a download for the archive
  6. Open the zip archive and confirm that the top level has
  7. LICENSE
  8. README.md
  9. metadata/
  10. Navigate into the metadata folder
  11. Confirm that there is a manifest.json and environment.json file

• [x] Exporting - BagIt

  1. Launch a Tale that you own
  2. Navigate to Run
  3. Click the three-dot-dropdown
  4. Select 'Export as BagIt'
  5. Confirm that you are asked to start a download for the archive
  6. Open the archive
  7. Confirm that the top level has
  8. run-local.sh
  9. tagmanifest-sha256.txt
  10. tagmanifest-md5.txt
  11. manifest-sha256.txt
  12. manifest-md5.txt
  13. bag-info.txt
  14. README.md
  15. bagit.txt
  16. fetch.txt
  17. data/
  18. metadata/
  19. Navigate into the metadata folder
  20. Confirm that there is a manifest.json and environment.json file
  21. Navigate to /data
  22. Confirm that there is a LICENSE file

[ ] Read-only Tale

1. Login as User A
   1. Create a new Tale A
   2. Add file and folder to Workspace
   3. Set to public
2. Login as User B
   1. Launch Tale A
   2. Go to Browse, confirm delete "x" not available on Tale
   3. Go to old Tale view, confirm delete button not present
   4. Go to Run > Files > Tale Workspace
   5. Confirm that files/folders are listed
   6. Confirm "+" button disabled
   7. Confirm dropdown on folder and file only allow "Copy to Home" and "Download"
   8. Go to Run > Interact
   9. Confirm that you cannot delete a file/folder via environment UI or terminal
   10. Confirm that you cannot create a file/folder via environment UI or terminal
   11. Go to Run > Metadata
   12. Confirm that you cannot modify the tale metadata

Analyze in Whole Tale

Task 1: Importing a Dataset from Dataverse

These test cases cover potential situations that can occur when importing datasets from Dataverse.

• [x] Case 1: Ideal Scenario:

Testing Steps:

1. Remove all running Tale instances
2. Navigate to https://girder.stage.wholetale.org/api/v1/integration/dataverse?fileId=3323458&siteUrl=https%3A%2F%2Fdataverse.harvard.edu
3. Confirm that the Tale title reads as Replication Data for: "Agricultural Fires and Health at Birth"
4. Confirm that the only item in the Selected data section matches the uri with Data Source appended
5. Confirm that no environment is selected
6. Confirm that the Launch New Tale button is disabled
7. Select an environment
8. Click Launch New Tale
9. Confirm that the progress bar appears & properly updates
10. Confirm that once complete, you are redirected to the run page
11. Confirm that the Tale name matches the Tale Name in the compose page
12. Confirm that the data exists in the Tale

Task 2: Importing a Dataset from DataONE

These test cases cover different scenarios that may happen when importing datasets from DataONE.

• [x] Case 1: Ideal Scenario:

This test covers the ideal case where dataset importing has all of the needed information to properly run. This includes the dataset location, title, and selected environment.

Testing Steps:

1. Remove all running Tale instances
2. Navigate to https://girder.stage.wholetale.org/api/v1/integration/dataone?uri=https%3A%2F%2Fsearch.dataone.org%2Fview%2Fdoi%3A10.18739%2FA2VQ2S94D&title=Fire%20influences%20on%20forest%20recovery%20and%20associated%20climate%20feedbacks%20in%20Siberian%20Larch%20Forests%2C%20Russia&environment=RStudio
3. Confirm that the Tale title reads as Fire influences on forest recovery and associated climate feedbacks in Siberian Larch Forests, Russia
4. Confirm that the only item in the Selected data section matches the uri with Data Source appended
5. Confirm that Rstudio is selected in the Environments widget
6. Confirm that the Launch New Tale button is enabled
7. Click Launch New Tale
8. Confirm that the progress bar appears & properly updates
9. Confirm that once complete, you are redirected to the run page
10. Confirm that the Tale name matches the Tale Name in the compose page
11. Confirm that the data exists in the Tale

Task 3: Shared Behavior

These cases show errors that are common between Dataverse and DataONE. Although they are tested using DataONE URIs, the errors should be the same when replacing it with a Dataverse URI.

• [x] Case 1: Missing Data Location Query Param:

The test covers the case where the dataset location is missing as a query parameter. This query parameter is used to let the UI know that we're importing a dataset/Tale. If it is missing, the default Compose page should be observed.

Testing Steps:

1. Remove all running Tale instances
2. Navigate to https://dashboard.stage.wholetale.org/compose?uri=
3. Confirm that the Tale title is blank
4. Confirm that the Selected data section is blank
5. Confirm that no environment is selected
6. Confirm that the Launch New Tale button is disabled

• [x] Case 2: Missing Dataset Title Query Param:

This test covers the case where the name query parameter is missing. The expected behavior is that the uri query parameter is used instead.

Testing Steps:

1. Remove all running Tale instances
2. Navigate to https://dashboard.stage.wholetale.org/compose?uri=https%3A%2F%2Fdataverse.harvard.edu%2Fapi%2Faccess%2Fdatafile%2F3323458
3. Confirm that the Tale title reads as https://dataverse.harvard.edu/api/access/datafile/3323458
4. Confirm that the only item in the Selected data section matches the uri with Data Source appended
5. Confirm that no Environment is selected
6. Confirm that the Launch New Tale button is disabled
7. Select an environment
8. Click Launch New Tale
9. Confirm that the progress bar appears & properly updates
10. Confirm that once complete, you are redirected to the run page
11. Confirm that the Tale name matches the Tale Name in the compose page
12. Confirm that the data exists in the Tale

Tale metadata tests

The purpose of these tests are to confirm that the metadata files (manifest.json, environment.json, LICENSE) we generate are correct.

• [x] manifest.json

  1. Launch a Tale that you own
  2. Add a dataset to the Tale
  3. Add at least one author
  4. Add at least one file to the workspace
  5. Export the Tale
  6. Open the manifest.json file under metadata/
  7. Confirm that the author is listed under schema:author
  8. Confirm that you are listed under createdBy
  9. Confirm that the file(s) in the workspace are properly listed in aggregates
  10. Confirm that the external dataset is referenced in aggregates
  11. Confirm that the dataset is also listed under Datasets

• [ ] environment.json

  1. Open the archive from the manifest.json test
  2. Navigate to metadata/environment.json
  3. Open the girder web API
  4. Locate and GET the Image that the Tale used
  5. Confirm that the config section from girder matches the config section in environment.json

• [x] LICENSE

  1. Open the archive from the manifest.json test
  2. Open the LICENSE file
  3. Navigate to Run > metadata
  4. Confirm that the opened LICENSE file matches the selected license on the metadata page
  5. Change the Tale's license
  6. Export the Tale
  7. Confirm that the LICENSE changed appropriately

Publishing tests

• [x] Case 1: Un-published Tale

  1. Launch a Tale that you own
  2. Add data to your Tale-this can be to the workspace or as external data
  3. Click the ellipsis dropdown on the Run page
  4. Click Publish tale...
  5. Confirm that you are presented with a popup asking you to login to DataONE
  6. Click Continue to DataONE Login
  7. Log into ORCID
  8. Confirm you are redirected back to Whole Tale
  9. Confirm the publishing modal automatically opens
  10. Confirm the the links in the info icons open in new tabs and resolve
  11. Confirm that you can select production and test DataONE servers
  12. Select the DataONE Development repository
  13. Click Publish
  14. Confirm that the progress bar begins and disappears once complete
  15. Confirm that an identifier is shown in the modal dialog.
  16. Click the identifier
  17. Confirm that you are brought to DataONE in a new tab
  18. Confirm that the package lists
      1. metadata.xml
      2. LICENSE
      3. README.md
      4. manifest.json
      5. environment.json
  19. Confirm that the Data Set Creators section lists your account
  20. Confirm that the Data Set Usage Rights matches the license on the Tale's Run > metadata page
  21. Confirm that the Published Location on the Run > metadata page is correct

• [x] Case 2: Re-published Tale

  1. Launch a Tale that has been published before
  2. Take note of the Published Location in Run > metadata
  3. Re-publish the Tale to the DataONE Development repository
  4. Confirm that the Published Location has changed to the correct package landing page

Regression tests

• [ ] Internal-state problems (#326)

  1. Open a non-incognito browser tab, log into the UI and navigate to the /manage view
  2. Create a folder (e.g. test123) and click down into it
  3. Open an incognito browser tab, login to the UI again, and navigate to the /manage view
  4. In the incognito tab, delete the test123 folder and close the incognito tab
  5. Refresh your non-incognito browser tab
  6. You should not be stuck on a blue screen
  7. You should not need to manually refresh the page
  8. You should be taken to the /manage view of your Home folder
  9. NOTE: Your last refresh may take a couple of seconds longer than usual

• [ ] Invalid instance ID (#333)

  1. Manually change the path suffix in the address bar to /run/ThisIsObviouslyNotARealID
  2. You should be brought to /browse page instead

• [ ] Files don't overflow Manage page (#410)

  1. Navigate to the manage page
  2. Register enough/upload enough data to the point where you need to scroll through files on the right hand side
  3. Make sure that the files don't overflow the container, also check in Firefox

• [ ] Handle read only Tales (#406)

  1. Login to the WholeTale Dashboard
  2. Launch a Tale that does not allow you read access
  3. Navigate to Run > Files
  4. You should see that Tale Workspaces has been greyed out, disallowing reads
  5. Launch a Tale that allows you to read, but not write
  6. Navigate to Run > Files > Tale Workspaces 1.You should see the Tale's Workspace folders/files are listed here
  7. You should see that (+) has been greyed out, disallowing writes

[ThomasThelen]

Test Results:

1. Fullscreen browse on Firefox cuts off the bottom (Browse > General)
2. Redirect failure on Tale delete; noted in Chrome tests (Browse > View tale)
3. Registered Datasets don't auto populate Data (needs page refresh. Seen with 10.5065/D6862DM8) (Manage > Register DataONE data)
4. Error registering Dataverse (doi:10.7910/DVN/SZ9YXZ) (Manage > Dataverse)
5. License, Environment, & description can be modified (but not saved) on a read-only Tale
6. Copy to Home missing from Workspace

Misc Issues:

Test Update Doc (Fixes here)

1. Update LIGO Launch test with new notification system
2. Update Compose tests with new notification system
3. Fix [ ] Read-only Tale checkbox
4. Fix line break in Tale metadata tests

No UI State Change on Failed Launches

After a Tale fails to launch, the Tale stays in launching state in Launched Tales (Unrelated to these tests)

Failure to re-launch Tales

Launch LIGO Delete Launched LIGO Re-Launch Notice there is no notification

Deleting a Running Tale Shows Instance

Launch a Tale Navigate to its Run page Delete the Tale Note that the instance is still embedded in the browser

Removed Copied File Remains in UI

Launch a Tale Navigate to Files > Home Upload a File Copy File Delete Copied File Note that deleted file is still shown Refresh page Note that it's gone

File Upload Errors give [object: Object]

Upload n files until one fails Note that the error message along the lines of Not uploaded, error: [object Object]

Two Progress bars with Analyze in Whole Tale

When creating a Tale via https://girder.stage.wholetale.org/api/v1/integration/dataverse?fileId=3323458&siteUrl=https%3A%2F%2Fdataverse.harvard.edu, you see the progress bar on the Compose page and the one from the notification system. They are mostly in sync