leginonbot commented 8 months ago

Author Name: Amber Herold (Amber Herold) Original Redmine Issue: 2071, https://emg.nysbc.org/redmine/issues/2071 Original Date: 2012-10-01 Original Assignee: Amber Herold

We need to identify all test data needed to test all features of CTF Estimation found in appion.

We need to know what the output from these data should be.
We need to identify all the tests that need to be run.

CTF Estimation Test Matrix

Ace2 Tests Note: After running command, results are found in ...imagename.mrc.ctf.txt

|.Astigmatism|.Over/Under Focus|.Command|.Image Name|.Defocus|.Ang Astigmatism|.Amplitude Contrast|.Confidence| | none| over| ace2.exe -i 12oct01e_00059en.mrc -b 2 -c 2.0 -k 200.0 -a 1.3656 -e 10.0,0.001 -r 0.0| 12oct01e_00059en.mrc|5.568862e-07 6.559742e-07 |-70.235983|0.091264|0.815827| | none| under| ace2.exe -i 12oct01e_00055en.mrc -b 2 -c 2.0 -k 200.0 -a 1.3656 -e 10.0,0.001 -r 0.0| 12oct01e_00055en.mrc|1.280421e-06 1.318144e-06 |15.267624|0.192086|0.947226| | direction 1| over| ace2.exe -i 12oct01e_00041ea.mrc -b 2 -c 2.0 -k 200.0 -a 1.3656 -e 10.0,0.001 -r 0.0| 12oct01e_00041ea.mrc|1.337478e-06 1.482890e-06 |72.485569|0.050000|0.863265| | direction 1| under| ace2.exe -i 12oct01e_00032ea.mrc -b 2 -c 2.0 -k 200.0 -a 1.3656 -e 10.0,0.001 -r 0.0| 12oct01e_00032ea.mrc| 1.773057e-06 1.867818e-06 |-66.550039|0.182214|0.957607| | direction 2| over | ace2.exe -i 12oct01e_00050eb.mrc -b 2 -c 2.0 -k 200.0 -a 1.3656 -e 10.0,0.001 -r 0.0| 12oct01e_00050eb.mrc| 1.477478e-06 1.674565e-06 |-64.236799|0.050000|0.876410| | direction 2| under| ace2.exe -i 12oct01e_00054eb.mrc -b 2 -c 2.0 -k 200.0 -a 1.3656 -e 10.0,0.001 -r 0.0| 12oct01e_00054eb.mrc| 1.533405e-06 1.649353e-06 |29.912167| 0.194930|0.958123|

Ace1 Tests Note: After running command, results are found in

|.Astigmatism|.Over/Under Focus|.Command|.Image Name|.Defocus|.Ang Astigmatism|.Amplitude Contrast|.Confidence| | none| over| | ||||| | none| under| | |||||

CTFFind Tests Note: After running command, results are found in rundir/logfiles/imagename-ctfprog.log if run in appion. Output files end in -pow.mrc and are attached to this issue.

Xmipp CTF Tests Note: After running command, results are found in

|.Astigmatism|.Over/Under Focus|.Command|.Image Name|.Defocus|.Ang Astigmatism|.Amplitude Contrast|.Confidence| | none| over| | ||||| | none| under| | ||||| | direction 1| over| || |||| | direction 1| under| || |||| | direction 2| over | | ||||| | direction 2| under| | |||||

CTF Correction Test Matrix

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Bridget Carragher (@bcarr-czi) Original Date: 2012-10-01T17:30:27Z

The matrix is a good idea in theory but seems we will never get all these tests done in practice. So how about just picking some obvious ones and making sure those work. We certainly never did these exhaustive tests in the past. I would also assume for now during this "emergency" that all we should test are underfocus as that would be 99% of the images we take and I would just post a warning to users that we are unsure of the behavior for overfocus images.

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Anchi Cheng (@anchi2c) Original Date: 2012-10-01T19:24:57Z

This is for the future testing and the over/under only need to be done at the algorithm level and one image each.

I will aquire one image for each case for this today.

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2012-10-01T20:53:49Z

Currently, the pipeline is only designed to handle underfocus. Overfocus should fail, which may be a good test nonetheless.

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Amber Herold (Amber Herold) Original Date: 2012-10-08T16:10:02Z

r17167 adds testAce2.py to myami. It uses the standard python unittest class to run all the tests outlined in the Ace2 CTF Estimation test matrix and compare the results to what is expected.

### This class is used to test the Ace2 program. It will dowload the needed test images from the AMI website, 
### then run ace2.exe and compare the results to what is ecpected. It will remove results files from a prior run
### when running subsequent tests. It does not remove the files after a test is run so that the usr can review 
### the result files if needed. Any functions in this class that begin with "test_" is a seperate test that
### will be executed by unittest.

### To run this test:
### 1. Create a temporary test directory and change directories to it. ex. mkdir ace2test, cd ace2test
### 2. run this file by providing the path to it. ex. python ../workspace/myami/appion/test/testAce2.py
### 3. review the results printed to the terminal.

leginon-org / leginon-redmine-archive

Ctf testing matrix #2071

We need to identify all test data needed to test all features of CTF Estimation found in appion.

CTF Estimation Test Matrix

CTF Correction Test Matrix