search

CentralValleyModeling / wrims

5 stars 4 forks source link

Smoke test of the WRIMS v2.2 installer #132

Open dezidizon opened 2 months ago

dezidizon commented 2 months ago

Smoke test of the WRIMS v2.2 installer

rma-tom commented 1 day ago

@hxiedwr @dwr-ycheng @dwr-zroy -- I'm including you in this discussion because you were on the other testing thread.

In a recent status call, I mentioned that Josh and I have been informally doing "smoke tests" to catch obvious problems with installer builds before sending them to DWR for more extensive tests. Over time the smoke tests have become more consistent and we've been following checklists like this one:

  1. Install wrims GUI from GitHub artifact or from zipfile on RMA RAID
  2. Launch program
  3. As necessary, set Eclipse workspace to a location where there are CalSim or CalLite models for testing -- may require restarts
  4. Open a wresl file from a CalSim or CalLite model and verify that the editor displays syntax highlighting and that an outline of the wresl contents displays in the eclipse outline panel work
  5. Open DSS perspective and verify that DSS files (DSS6 and DSS7) open and selected time series display in tables and graphs
  6. Open schematic perspective and verify that schematics display with values from DSS files opened in DSS perspective
  7. In IDE perspective, open a CalLite or CalSim model and run in debugger -- check that values from reference DSS can be used in watchlist and that the model runs to completion

For the run testing step, I've been using a CalLite model that was provided to us in March, labeled "CalLite 4.1 TF." For the smoke tests I just run the model for one year so that it completes in under a minute.

Additionally, if we're working on a particular problem we check that the fixes that have been checked in make the changes that we expect.

Do you see any gaps in this? Is that CalLite model an appropriate choice for this purpose? It runs the CBC solver, so we're not screening for problems any others.

hxiedwr commented 1 day ago

We usually don't change workspace but load the study into WRIMS GUI.

I think using CalLite model to test on your side is good. It is just used to prove the concept. Once the new package is forwarded to us, we will test the package with cs3, which takes much longer time.

I prefer RMA not to spend a lot of time on testing.