whole-tale / wt-design-docs

MIT License
5 stars 9 forks source link

0.9 RC3 Testing [Firefox] #131

Closed ThomasThelen closed 4 years ago

ThomasThelen commented 4 years ago

Test Plan

https://github.com/whole-tale/wt-design-docs/issues/new?template=test_plan.md

Note: For all tests, repeat for supported browser/OS combinations.

Preconditions:

Authentication

Navigation

Browse

Preconditions:

Manage

Preconditions

Run

Preconditions:

Settings

Tale Creation

Analyze in Whole Tale

Task 1: Importing a Dataset from Dataverse

These test cases cover potential situations that can occur when importing datasets from Dataverse.

Testing Steps:

  1. Remove all running Tale instances
  2. Navigate to https://girder.stage.wholetale.org/api/v1/integration/dataverse?datasetPid=doi%3A10.7910%2FDVN%2F3MJ7IR&siteUrl=https%3A%2F%2Fdataverse.harvard.edu
  3. Confirm that the Tale title reads as Replication Data for: "Agricultural Fires and Health at Birth"
  4. Confirm that the only item in the Input data section matches the uri with Data Source appended
  5. Confirm that no environment is selected
  6. Confirm that the Create New Tale and Launch button is disabled
  7. Select an environment
  8. Confirm that READ/WRITE is selected
  9. Click Create New Tale and Launch
  10. Confirm that you are redirected to the run page
  11. Confirm that the Tale name matches the Tale Name in the Create Tale Modal
  12. Confirm that the data exists under Tale Workspace
  13. Confirm that the Tale's category is science

Task 2: Importing a Dataset from DataONE

These test cases cover different scenarios that may happen when importing datasets from DataONE.

This test covers the ideal case where dataset importing has all of the needed information to properly run. This includes the dataset location, title, and selected environment.

Testing Steps:

  1. Remove all running Tale instances
  2. Navigate to https://girder.stage.wholetale.org/api/v1/integration/dataone?uri=https%3A%2F%2Fsearch.dataone.org%2Fview%2Fdoi%3A10.18739%2FA2VQ2S94D&title=Fire%20influences%20on%20forest%20recovery%20and%20associated%20climate%20feedbacks%20in%20Siberian%20Larch%20Forests%2C%20Russia&environment=RStudio
  3. Confirm that the Tale title reads as Fire influences on forest recovery and associated climate feedbacks in Siberian Larch Forests, Russia
  4. Confirm that the only item in the Selected data section matches the uri with Data Source appended
  5. Confirm that Rstudio is selected in the Environments widget
  6. Confirm that the Create New Tale and Launch button is enabled
  7. Confirm READ ONLY is selected
  8. Click Create New Tale and Launch
  9. Confirm that you are redirected to the run page
  10. Confirm that the Tale name matches the Tale Name in the Create Tale Modal
  11. Confirm that the data exists in the Tale under External Data
  12. Confirm that the Tale's category is science

Links:

  1. https://dashboard.stage.wholetale.org/browse?api=https%3A%2F%2Fdev.nceas.ucsb.edu%2Fknb%2Fd1%2Fmn%2Fv2&name=Dataone%20Dataset&uri=https://dev.nceas.ucsb.edu/view/doi:10.5072/FK2K075M25
  2. https://dashboard.stage.wholetale.org/browse?api=https://test.arcticdata.io/metacat/d1/mn/v2&name=Dataone%20Dataset&uri=https://test.arcticdata.io/view/doi:10.5065/D6N014NC

Testing Steps:

  1. Remove all running Tale instances
  2. Navigate to each of the two links above
  3. Select an environment
  4. Click Create New Tale and Launch
  5. Confirm that the correct data exists in the Tale under External Data

Task 3: Shared Behavior

These cases show errors that are common between Dataverse and DataONE. Although they are tested using DataONE URIs, the errors should be the same when replacing it with a Dataverse URI.

This test covers the case where the name query parameter is missing. The expected behavior is that the uri query parameter is used instead.

Testing Steps:

  1. Remove all running Tale instances
  2. Navigate to https://dashboard.stage.wholetale.org/browse?uri=https%3A%2F%2Fdataverse.harvard.edu%2Fapi%2Faccess%2Fdatafile%2F3323458
  3. Confirm that the Tale title reads as https://dataverse.harvard.edu/api/access/datafile/3323458
  4. Confirm that the only item in the Selected data section matches the uri with Data Source appended
  5. Confirm that no Environment is selected
  6. Confirm that the Create New Tale and Launch button is disabled
  7. Select an environment
  8. Click Create New Tale and Launch
  9. Confirm that the notification bar appears & properly updates
  10. Confirm that you are redirected to the run page
  11. Confirm that the Tale name matches the Tale Name in the Create Tale Modal
  12. Confirm that the data exists in the Tale under External Data

This test covers the case where a user imports data from an external source and selects READ/WRITE in the Create Tale Modal.

Testing Steps (Variant 1):

  1. Remove all running Tale instances
  2. Navigate to https://dashboard.stage.wholetale.org/browse?uri=https%3A%2F%2Fdataverse.harvard.edu%2Fapi%2Faccess%2Fdatafile%2F3323458
  3. Confirm that the Tale title matches the URI above
  4. Confirm that the only item in the Selected data section matches the uri with Data Source appended
  5. Confirm that the Create New Tale and Launch button is disabled
  6. Select an environment
  7. Select READ/WRITE
  8. Click Create New Tale and Launch
  9. Confirm that the notification bar appears & properly updates
  10. Confirm that you are redirected to the run page
  11. Confirm that the Tale name matches the Tale Name in the Create Tale Modal
  12. Confirm that the data exists in the Tale Workspace

Testing Steps (Variant 2):

  1. Remove all running Tale instances
  2. Navigate to https://dashboard.stage.wholetale.org/browse?api=https%3A%2F%2Fdev.nceas.ucsb.edu%2Fknb%2Fd1%2Fmn%2Fv2&environment=JupyterLab&name=proveit&uri=doi%3A10.5072%2FFK27P92Z55
  3. Confirm that the Tale title matches the dataset
  4. Confirm that the only item in the Selected data section matches the uri with Data Source appended
  5. Confirm that the Create New Tale and Launch button is disabled
  6. Select READ/WRITE
  7. Click Create New Tale and Launch
  8. Confirm that the notification bar appears & properly updates
  9. Confirm that you are redirected to the run page
  10. Confirm that the Tale name matches the Tale Name in the Create Tale Modal
  11. Confirm that the data exists in the Tale Workspace

Tale metadata tests

The purpose of these tests are to confirm that the metadata files (manifest.json, environment.json, LICENSE) we generate are correct.

Publishing tests

Zenodo integration tests

The register tests the following cases.

The register tests the following cases.

  1. Open https://dashboard.stage.wholetale.org/browse?uri=https%3A%2F%2Fdoi.org%2F10.5281%2Fzenodo.3242073&environment=JupyterLab&name=Simple%20requirements.txt%20based%20example%20for%20repo2docker&asTale=true
  2. COnfirm Source Data and Title match URL and "READ WRITE" is selected
  3. Select 'Create and Launch Tale'
  4. Confirm image builds and notebook can run
  1. Open https://girder.stage.wholetale.org/api/v1/integration/zenodo?doi=10.5072%2Fzenodo.490450&resource_server=sandbox.zenodo.org
  2. Confirm Water Tale is created.
  3. Confirm citation on Metadata view
  4. Confirm DOI matches above
  5. Use the Swagger UI to confirm the publishInfo field on the tale object contains pid, repository, repository_id, uri
  6. Run the tale and and notebook, confirm it works as expected (produces glorious map of Texas)

Regression tests

ThomasThelen commented 4 years ago

Authentication: Passed Navigation: Passed Browse: Passed Tale Creation: Failed Settings: Passed Publishing tests: Passed Regression tests: Passed Run: Passed

ThomasThelen commented 4 years ago

Tale Creation

When creating a Jupyter with Spark! Tale I recieved the following trace. I created two new Tales with Jupyter Spark! they worked fine.

APIError: 500 Server Error: Internal Server Error ("mkdir /var/lib/docker/overlay2/6d9546c33aea02ae5321c0c1cb5a2af57400973dd02608b67f9afabd5499db12-init: no space left on device")
  File "/usr/local/lib/python3.5/dist-packages/celery/app/trace.py", line 382, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/girder_worker/task.py", line 148, in __call__
    results = super(Task, self).__call__(*_t_args, **_t_kwargs)
  File "/usr/local/lib/python3.5/dist-packages/celery/app/trace.py", line 641, in __protected_call__
    return self.run(*args, **kwargs)
  File "/gwvolman/gwvolman/tasks.py", line 437, in build_tale_image
    ret = _build_image(cli, tale_id, image, tag, temp_dir, REPO2DOCKER_VERSION)
  File "/gwvolman/gwvolman/utils.py", line 317, in _build_image
    'bind': '/host/tmp', 'mode': 'ro'
  File "/usr/local/lib/python3.5/dist-packages/docker/models/containers.py", line 803, in run
    detach=detach, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/docker/models/containers.py", line 861, in create
    resp = self.client.api.create_container(**create_kwargs)
  File "/usr/local/lib/python3.5/dist-packages/docker/api/container.py", line 430, in create_container
    return self.create_container_from_config(config, name)
  File "/usr/local/lib/python3.5/dist-packages/docker/api/container.py", line 441, in create_container_from_config
    return self._result(res, True)
  File "/usr/local/lib/python3.5/dist-packages/docker/api/client.py", line 267, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.5/dist-packages/docker/api/client.py", line 263, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.5/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
Xarthisius commented 4 years ago

APIError: 500 Server Error: Internal Server Error ("mkdir /var/lib/docker/overlay2/6d9546c33aea02ae5321c0c1cb5a2af57400973dd02608b67f9afabd5499db12-init: no space left on device")

Cleaned the node up. Should be good to go.

ThomasThelen commented 4 years ago

Aside from the noted regressions in the other issues, I don't see any newly injected bugs. Closing.