search

LandSciTech / ccviR

Implement NatureServe climate change vulnerability index in R
https://landscitech.github.io/ccviR/
Other
2 stars 1 forks source link

Test set for release #151

Open adricaswell opened 4 months ago

adricaswell commented 4 months ago

This issue outlines 5 tests to be used as a minimum test set that can be manually run in the ccviR app prior to release. These 5 tests have been designed to check that a range of app functions work properly. While testing, complete the test checklist when the described item responds as expected.

When testing the app for release, be sure to simulate usage by new users. Install the library from GitHub in a fresh R session with remotes::install_github("LandSciTech/ccviR").

Sample spatial data sourced from the internet to be used for testing can be found this zipped folder. The README contained in the folder outlines what data is included and its intended use.

While testing, be sure to check for errors, warnings, and unexpected results both within the app and in the R console.

Test 1

 **Check that the app can handle the following:** - [ ] Using the app with demo data - [ ] Assessing a plant
**To test this:** Run the app in demo mode `run_ccvi_app("demo")` and follow the "Using the ccviR app" tutorial

Test 2

 **Check that the app can handle the following:** - [ ] Using the app with data sourced by user - [ ] Assessing an animal - [ ] Calculating CCVI (before spatial analysis has been run) - [ ] Saving progress (before spatial analysis is run)
**To test this:** 1. Run the ccviR app `run_ccvi_app("path/to/user/data")` 2. Fill out *Species Information* selecting "Mammal" as the taxonomic group 3. Input spatial data (from internet) in *Spatial Data Analysis* but do not run spatial analysis 4. Fill out *Vulnerability Questions* and *Spatial Vulnerability Questions* 5. Attempt to calculate the CCVI 6. Save progress

Test 3

 **Check that the app can handle the following:** - [ ] Loading from a previous assessment (using the wrong data) - [ ] Loading from a previous assessment (where spatial analysis was not run) - [ ] Calculating CCVI - [ ] Generating a report - [ ] Saving progress (after spatial analysis is run) - [ ] Assessing a special kind of species
**To test this:** 1. Run the ccviR app `run_ccvi_app("path/to/user/data")` 2. Select *Load from previous assessment* and load csv with wrong or no data 3. Select *Load from previous assessment* again and load assessment from Test 2 4. Check that previous assessment loaded as expected (*Species Information*, *Vulnerability Questions*, and *Spatial Vulnerability Questions* are filled out) 5. Change *Species Information* - check *Obligate of caves* box 6. Input spatial data (from internet) in *Spatial Data Analysis* and run spatial analysis 7. Check *Exposure Reuslts* 8. Check *Spatial Vulnerability Questions* and change the response of C2ai 9. Calculate CCVI 10. Generate report and check it over 11. Save progress

Test 4

 **Check that the app can handle the following:** - [ ] Loading from a previous assessment (where spatial analysis was run) - [ ] Generating a report (immediately after loading from a previous assessment) - [ ] Changing inputs after spatial analysis is run and CCVI is calculated - [ ] Changing climate data and multiple climate scenarios
**To test this:** 1. Run the ccviR app `run_ccvi_app("path/to/user/data")` 2. Select *Load from previous assessment* again and load assessment from Test 3 3. Navigate to *Index Results* and generate report (without re-calculating CCVI), check over report 4. Check over all tabs to make sure data loaded in from previous assessment correctly (make sure C2ai response remained changed) 5. In *Spatial Data Analysis*, change climate data (from one scenario to two scenarios, or vice versa) 6. Do not run spatial analysis, advance to *Exposure Results*, make sure maps and tables match 7. Return to *Spatial Data Analysis* and run spatial analysis 8. Return to *Index Results*, re-calculate CCVI and generate report

Test 5

 **Check that the app can handle:** - [ ] Inputting the wrong shapefile datatype - [ ] Inputting the wrong range change raster - [ ] User correction after inputting wrong data - [ ] App exits properly
**To test this:** 1. Run the ccviR app `run_ccvi_app("path/to/user/data")` 2. Fill out *Species Information* 3. Navigate to *Spatial Data Analysis* - Input point data for range, input climate data and assessment area correctly, run spatial analysis, check that error is handled - Input line data for assessment area, input climate data and range correctly, run spatial analysis, check that error is handled - Input climate data, assessment polygon, and range polygon correctly, input wrong range change raster, run spatial analysis, check that error is handled - Remove range change raster, input all data correctly, run spatial analysis 4. Check *Exposure Results* 5. Fill out *Vulnerability Questions* 6. Check/fill out *Spatial Vulnerability Questions* 7. Calculate CCVI 8. Exit app
adricaswell commented 2 months ago

4/26/2024 Test Branch: 127-exposure-results-table PR #158

Test 4 revealed that the spatial vulnerability question response changed by user in test 3 (overriding the one calculated during spatial data analysis) was not retained when re-loading from saved csv. Instead, the response reverted to the response calculated during spatial data analysis, erasing the user selection. This issue seems to only occur in 127-exposure-results-table, the master branch handles this scenario as expected.

see24 commented 2 months ago

Can you give more details of your test for tests 3 and 4? What data was used etc? Modifying the spatial results, saving to csv and then reloading seems to work fine in this branch just using the built in data. Or see if you can reproduce this problem again out side of the other parts of the test. I don't know how anything changed in this branch would create this issue.

adricaswell commented 2 months ago

Was able to reproduce this issue using the following steps:

  1. Fill out Species Information (cave obligate checked)
  2. Input spatial data (range, assessment area, and climate data) using data linked in this issue's description
  3. Run spatial analysis
  4. Navigate to Spatial Vulnerability Questions and change 2ai (make a comment describing the change you made)
  5. Navigate to Index Results and calculate CCVI
  6. Save progress
  7. Exit app
  8. Run the ccviR app
  9. Load from previous assessment using progress just saved
  10. Navigate to Spatial Vulnerability Questions and check response (response should match the change outlined in the comment)

When I followed this workflow, I got this:

image

This issue only seems to occur when cave obligate is checked