search

SCIInstitute / ShapeWorks

ShapeWorks
http://sciinstitute.github.io/ShapeWorks/
Other
103 stars 32 forks source link

ShapeWorks 6.2 Testing #1509

Closed akenmorris closed 2 years ago

akenmorris commented 3 years ago

TODO: clean up this note to remove the "done" portions below. @sheryjoe there are several people missing from our repo who attend our weekly meetings.

Please edit and add a ✅ indicating success and ❌ indicating failure or 🕒 for a test in progress with your username when you complete a task for a given platform. Also, go ahead and add new tasks that might not already be on here.

When a test fails, please add a github issue:

Please use the most recent release candidate for all testing (be careful which shapeworks is in your $PATH).

The most recent is RC9:

https://github.com/SCIInstitute/ShapeWorks/releases/tag/v6.2.0-rc9

Example: Windows Mac Linux
Notebooks 🕒 (@archanasri - A)
Usecase: Ellipsoid ✅ (@akenmorris - A)
Usecase: All tiny-test ❌ (#1073) ✅ (@cchriste - A)
Systems chart Username System
(@HeavenlyBerserker - A) Ubuntu 20.04.2 LTS (Linux) [Binaries]
(@HeavenlyBerserker - B) Microsoft Windows 10 Education (Windows) [Release Branch]
(@iyerkrithika21 - A) Microsoft Windows 10 Home OS Build 19042.1052 (Windows) [Binaries]
(@archanasri - A) macOS Monterey 12.0.1 (OSX) [Binaries]
(@cchriste - A) maxOS Big Sur 11.3.1 (OSX)
(@cchriste - B) openSUSE 15.1 (Linux)
(@cchriste - C) Windows 10
(@akenmorris - A) Windows 10
(@akenmorris - B) openSUSE Leap 15.3
(@akenmorris - C) MacOS 10.5.7
(@akenmorris - D) Ubuntu 20.04 LTS
(@jadie1 - A) openSUSE 15.0 (Linux)
@patkins - A) OpenSUSE 15.2
@patkins - B) Windows 10
(@username - B) System (System Type)

Instructions for running and testing items

Testing modes:

  1. Binary downloads
    • Go to https://github.com/SCIInstitute/ShapeWorks, click releases, click the system you are testing, download and unzip the first asset.
    • Set your paths to the release binaries instead of your regular paths. Make sure python scripts are running them.
  2. Release/master branch
    • Fetch, checkout master, and pull; Make sure it builds. If you're on your usual repository directory, you might need to delete the dependencies directory and rerun source build_dependencies.sh.
  3. Clean installation
    • Find a different machine or create a VM and download the binaries (or clone branches and build if you're feeling industrious).
    • Run 1 or 2 items in each category.

General instructions

Notebooks

  1. Go to ShapeWorks/Examples/Python/notebooks/tutorials on terminal/CMD.
  2. Run the jupyter notebook command.
  3. Open whichever file you wish to test.
  4. Make sure it runs to completion and each input matches what the explanation says.

UseCases

  1. Go to ShapeWorks/Examples/Python/ on terminal/CMD or on your favorite IDE that runs python.
  2. Run the python RunUseCase.py --use_case <use_case> command, where is replaced by the command given. For instance python RunUseCase.py --use_case femur, or python RunUseCase.py --use_case femur --groom_images.
  3. Wait until it finishes. A Studio Analyze window should pop up. Make sure sampling, correspondence, and analyze->PCA->animate are reasonable.

Studio

  1. Go to ShapeWorks/Examples/Python/<item> on terminal/CMD, where "" is the item you're testing.
  2. Run the command ShapeWorksStudio <item>.xlsx.
  3. Click on "Run Groom" and make sure it's reasonable. Click on the "Optimize" tab, and click "Run Optimize". Go to the "Analyze" tab and make sure sampling, correspondence, and analyze->PCA->animate are reasonable.

Studio: MeshGrooming

  1. Download test_meshes.zip and unzip.
  2. Run command ShapeWorksStudio, or open ShapeWorksStudio somehow.
  3. "Start New Project" -> "+ to import shapes" -> Select all 4 downloaded shapes and open.
  4. Play with the mesh groom parameters and the views. Adjust parameters and make sure it still makes sense; e.g. check Groom->Smooth and adjust laplacian, then "Run Groom"; e.g. move around, click Center, increase viewers, run groom with other parameters, etc.

Ok, now the real thing!

*(not sure where to find results in Use Cases documentation for comparison)

Windows Mac Linux
Clean installation ✅ (@akenmorris - A)-rc6 ✅(@iyerkrithika21 - A)-rc7 ✅ (@akenmorris - C)-rc6 ✅ (@akenmorris - D)-rc6
Notebooks: getting-started-with-notebooks ✅ (@akenmorris - A)-rc6 ✅(@iyerkrithika21 - A)-rc7 ✅ (@akenmorris - C) ✅(@archanasri - A)-rc7 ✅ (@akenmorris - D)-rc6
Notebooks: getting-started-with-segmentations ✅ (@akenmorris - A)-rc6 ✅(@iyerkrithika21 - A)-rc7 #1573 ✅ (@akenmorris - C) ✅(@archanasri - A)-rc7 ✅ (@akenmorris - D)-rc6
Notebooks: getting-started-with-exploring-segmentations ✅ (@akenmorris - A)-rc6 ✅(@iyerkrithika21 - A)-rc7 #1573 ✅ (@akenmorris - C) ✅ (@archanasri - A)-rc7 ✅ (@akenmorris - D)-rc6
Notebooks: getting-started-with-meshes (@akenmorris - A)-rc6 ✅(@iyerkrithika21 - A)-rc7 #1573 ✅ (@akenmorris - C) ✅(@archanasri - A)-rc7 ✅ (@akenmorris - D)-rc6
Notebooks: getting-started-with-data-augmentation ✅ (@akenmorris - A)-rc6 #1573 ✅ (@akenmorris - C) ✅(@archanasri - A)-rc7 ✅ (@akenmorris - D)-rc6
Notebooks: getting-started-with-shape-cohort-generation ✅ (@akenmorris - A)-rc6 ✅(@iyerkrithika21 - A)-rc7 ✅ (@akenmorris - C) ✅(@archanasri - A)-rc7 ✅ (@akenmorris - D)-rc6
Usecase: ellipsoid ✅ (@cchriste - C) ✅(@iyerkrithika21 - A)-rc7 ✅ (@akenmorris - C)-rc6 ✅(@archanasri - A)-rc7 ✅ (@akenmorris - B)-rc6
Usecase: ellipsoid_cut ✅ (@akenmorris - A)-rc6 ✅(@iyerkrithika21 - A)-rc7 ✅ (@akenmorris - C)-rc6 ✅(@archanasri - A)-rc7 ✅ (@akenmorris - B)-rc6
Usecase: ellipsoid_evaluate ✅ (@akenmorris - A)-rc6 ✅(@iyerkrithika21 - A)-rc7 ✅ (@akenmorris - C)-rc6 ✅(@archanasri - A)-rc7 ✅ (@akenmorris - B)-rc6
Usecase: ellipsoid_pca ✅ (@cchriste - C) ✅(@iyerkrithika21 - A)-rc7 ✅ (@akenmorris - C)-rc6 ✅(@archanasri - A)-rc7 ✅ (@akenmorris - B)-rc6
Usecase: ellipsoid_fd ✅ (@cchriste - C) ✅(@iyerkrithika21 - A)-rc7 ✅ (@akenmorris - C)-rc6 ✅(@archanasri - A)-rc7 ✅ (@akenmorris - B)-rc6
Usecase: ellipsoid_mesh ✅ (@cchriste - C)✅(@iyerkrithika21 - A)-rc7 ✅ (@akenmorris - C)-rc6 ✅(@archanasri - A)-rc7 ✅ (@akenmorris - B)-rc6
Usecase: ellipsoid_multiple_domain ✅ (@cchriste - C) ❌(#1572) ✅ (@akenmorris - C)-rc6 ✅(@archanasri - A)-rc7 ✅ (@akenmorris - B)-rc6
Usecase: ellipsoid_multiple_domain_mesh ✅ (@akenmorris - A)-rc6 ✅(@iyerkrithika21 - A)-rc7 ✅ (@akenmorris - C)-rc6 ✅(@archanasri - A)-rc7 ✅ (@jadie1 - A-rc5)
Usecase: femur ✅ (@akenmorris - A)-rc6 ✅ (@akenmorris - C)-rc6 ✅(@archanasri - A)-rc7 ✅ (@jadie1 - A-rc5)
Usecase: femur --groom_images ✅ (@akenmorris - A)-rc6 ✅ (@akenmorris - C)-rc6 ✅(@archanasri - A)-rc7 ✅ (@jadie1 - A-rc5)
Usecase: femur_mesh ✅ (@akenmorris - A)-rc6 ✅ (@akenmorris - C)-rc6 ✅(@archanasri - A)-rc7 ✅ (@jadie1 - A-rc5)
Usecase: femur_cut ✅ (@akenmorris - A)-rc6 ✅ (@akenmorris - C)-rc6 ✅(@archanasri - A)-rc7 ✅ (@akenmorris - B)-rc6
Usecase: femur_cut --groom_images ✅ (@akenmorris - A)-rc6 ✅ (@akenmorris - C)-rc6 ✅(@archanasri - A)-rc7 ✅ (@jadie1 - A-rc5)
Usecase: femur_pvalues ✅ (@cchriste - C) ✅(@iyerkrithika21 - A)-rc7 ✅ (@akenmorris - C)-rc6 ✅(@archanasri - A)-rc7 ✅ (@akenmorris - B)-rc6
Usecase: left_atrium ✅ (@cchriste - C) ✅ (@akenmorris - C)-rc6 ✅(@archanasri - A)-rc7 ✅ (@akenmorris - B)-rc6
Usecase: left_atrium --groom_images ✅ (@akenmorris - A)-rc6 ✅ (@akenmorris - C)-rc6 ✅(@archanasri - A)-rc7 ✅ (@akenmorris - B)-rc6
Usecase: lumps ✅ (@cchriste - C) ✅(@iyerkrithika21 - A)-rc7 ✅ (@akenmorris - C)-rc6 ✅(@archanasri - A)-rc7 ✅ (@akenmorris - B)-rc6
Usecase: thin_cavity_bean ✅ (@akenmorris - A)-rc6 ✅ (@akenmorris - C)-rc6 ✅(@archanasri - A)-rc7 ✅ (@akenmorris - B)-rc6
Usecase: supershapes_1mode_contour ✅ (@akenmorris - A)-rc6 ✅(@iyerkrithika21 - A)-rc7 ✅ (@akenmorris - C)-rc6 ✅(@archanasri - A)-rc7 ✅ (@akenmorris - B)-rc6
Usecase: deep_ssm* ✅ (@akenmorris - A)-rc6 ✅ (@akenmorris - C)-rc6 ✅ (@akenmorris - D)-rc6
Studio: ellipsoid ✅ (@akenmorris - A)-rc6 ✅(@iyerkrithika21 - A)-rc7 ✅ (@akenmorris - C)-rc6 ✅ (@akenmorris - D)-rc6
Studio: FeatureMap ✅ (@akenmorris - A)-rc6 ✅(@iyerkrithika21 - A)-rc7 ✅ (@akenmorris - C)-rc6 ✅ (@akenmorris - D)-rc6
Studio: MeshGrooming ✅ (@akenmorris - A)-rc6 ✅(@iyerkrithika21 - A)-rc7 ✅ (@akenmorris - C)-rc6 ✅ (@akenmorris - D)-rc6
Studio: EllipsoidMultipleDomain ✅(@iyerkrithika21 - A)-rc7
...

* Edit deep_ssm.py so that num_samples = 3 on line 76 and "epochs": 3, on line 132, otherwise it will take upwards of two days.

Please keep comments on this issue to a minimum. Let's try to keep the status in the table and not in the comments.

akenmorris commented 3 years ago

I tried source install_shapeworks.sh with a fairly clean install of OSX 10.15 with no anaconda, and I get this:

Building wheels for collected packages: DatasetUtils
  Building wheel for DatasetUtils (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/amorris/miniconda3/envs/shapeworks/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/Applications/ShapeWorks/Python/DatasetUtilsPackage/setup.py'"'"'; __file__='"'"'/Applications/ShapeWorks/Python/DatasetUtilsPackage/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/nm/nbmb1cfx0xx0vqvmk9bfk0jm0000gn/T/pip-wheel-5ccxlc4j
       cwd: /Applications/ShapeWorks/Python/DatasetUtilsPackage/
  Complete output (8 lines):
  running bdist_wheel
  running build
  running build_py
  installing to build/bdist.macosx-10.9-x86_64/wheel
  running install
  running install_lib
  creating build/bdist.macosx-10.9-x86_64/wheel
  error: could not create 'build/bdist.macosx-10.9-x86_64/wheel': Permission denied
  ----------------------------------------
  ERROR: Failed building wheel for DatasetUtils
  Running setup.py clean for DatasetUtils

Not sure where the "10.9" is coming from, or if it's relevant.

On a pkg installed system, /Applications/ShapeWorks/Python/DatasetUtilsPackage/ won't be writable. This is more likely the problem.

patkins commented 3 years ago

I can't seem to edit above, but I have OpenSUSE 15.2 and Windows 10 that I can test on.

cchriste commented 3 years ago

I can't seem to edit above, but I have OpenSUSE 15.2 and Windows 10 that I can test on.

Thanks Penny. I'm not sure why you cannot edit, especially since you can be assigned! But please keep commenting and one of us can add the details you share above.

jadie1 commented 3 years ago

Should we be testing RC3?

akenmorris commented 3 years ago

Yes. I am working on automated verification though.

cchriste commented 3 years ago

Should we be testing RC3?

Hi Jadie, I think it would be really helpful to have a demonstration of how to manually verify a use case is correct by looking at the results in Studio. If you have expertise doing something like this, would it be possible to record a short video demonstrating and explaining how one might go about this? This aid can help everyone to better ensure not only that the use cases complete, but that their results seem correct (or incorrect).

jadie1 commented 3 years ago

@cchriste All of the use cases have updated results in the documentation. I would just recommend running the use cases then comparing what you see in studio with the snapshots and videos of expected output in the documentation.

cchriste commented 3 years ago

When should this Difference portion of the GUI be active? Should it be in a dropdown like the metrics? image

cchriste commented 3 years ago

Are these the correct results for ellipsoid_pca:

(sw6.2.0a) E:\sw6.2.0a\Examples\Python>python RunUseCase.py ellipsoid_pca
Using shapeworks module from C:\Users\cam\anaconda3\envs\sw6.2.0a\lib\site-packages\shapeworks\__init__.py
Using shapeworks from E:\sw6.2.0a\bin\shapeworks.EXE

Analysis - ShapeWorks PCA Python API

VDimension = 3-------------
m_numSamples = 15-------------
m_domainsPerShape = 1-------------
global_pts.size() = 15-------------

The sample size of the dataset is :  15

The dimensions of the dataset are :  384

Saving the PCA loadings and eigen vectors in the directory : Output/ellipsoid/pca/

Number of modes covering 99% varaince -  2

Loadings plot saved in directory - Output/ellipsoid/pca/

PCA metrics plot saved in directory - Output/ellipsoid/pca/

Shapeworks Pipeline Complete!

(spelling error of "variance" aside)

cchriste commented 3 years ago

ellipsoid_multiiple_domain misspelled in help (someone please correct this with another PR since it's really expensive to only do it for one pr)

iyerkrithika21 commented 3 years ago

Are these the correct results for ellipsoid_pca:

(sw6.2.0a) E:\sw6.2.0a\Examples\Python>python RunUseCase.py ellipsoid_pca
Using shapeworks module from C:\Users\cam\anaconda3\envs\sw6.2.0a\lib\site-packages\shapeworks\__init__.py
Using shapeworks from E:\sw6.2.0a\bin\shapeworks.EXE

Analysis - ShapeWorks PCA Python API

VDimension = 3-------------
m_numSamples = 15-------------
m_domainsPerShape = 1-------------
global_pts.size() = 15-------------

The sample size of the dataset is :  15

The dimensions of the dataset are :  384

Saving the PCA loadings and eigen vectors in the directory : Output/ellipsoid/pca/

Number of modes covering 99% varaince -  2

Loadings plot saved in directory - Output/ellipsoid/pca/

PCA metrics plot saved in directory - Output/ellipsoid/pca/

Shapeworks Pipeline Complete!

(spelling error of "variance" aside)

This is the correct output

cchriste commented 3 years ago

ellipsoid_multiple_domain looks good, the single sample median never changes (maybe the first sample is the median)

iyerkrithika21 commented 3 years ago

ellipsoid_multiiple_domain misspelled in help (someone please correct this with another PR since it's really expensive to only do it for one pr)

I will create a PR for the the typos in ellipsoid pca and multiple domains.

iyerkrithika21 commented 3 years ago

ellipsoid_multiple_domain looks good, the single sample median never changes (maybe the first sample is the median)

In the joint dataset, only one ellipsoid changes rotation and or size w.r.t to the first one. So one of the ellipsoids will not have any modes of variation. I am not sure if you are referring to this.

akenmorris commented 3 years ago

When should this Difference portion of the GUI be active? Should it be in a dropdown like the metrics? image

It will be active if there are groups present (e.g. group_<> columns). I think maybe a tooltip or message why it's greyed out could be helpful. I think it's in the right place.

akenmorris commented 3 years ago

RC6 on linux:

---------------------------------------------
Testing Results:
---------------------------------------------
The following use cases passed (40)
[ PASSED ] :     29 seconds : ellipsoid --tiny_test
[ PASSED ] :     22 seconds : ellipsoid_mesh --tiny_test
[ PASSED ] :     39 seconds : ellipsoid_fd --tiny_test
[ PASSED ] :     30 seconds : ellipsoid_cut --tiny_test
[ PASSED ] :     21 seconds : lumps --tiny_test
[ PASSED ] :     48 seconds : left_atrium --tiny_test
[ PASSED ] :     64 seconds : femur --tiny_test
[ PASSED ] :     25 seconds : ellipsoid --tiny_test
[ PASSED ] :     17 seconds : ellipsoid_mesh --tiny_test
[ PASSED ] :     34 seconds : ellipsoid_fd --tiny_test
[ PASSED ] :     28 seconds : ellipsoid_cut --tiny_test
[ PASSED ] :     18 seconds : lumps --tiny_test
[ PASSED ] :     40 seconds : left_atrium --tiny_test
[ PASSED ] :     31 seconds : femur --tiny_test
[ PASSED ] :     88 seconds : femur --groom_images --tiny_test
[ PASSED ] :     24 seconds : femur_mesh --tiny_test
[ PASSED ] :     71 seconds : femur_cut --tiny_test
[ PASSED ] :     90 seconds : femur_cut --groom_images --tiny_test
[ PASSED ] :     88 seconds : deep_ssm --tiny_test
[ PASSED ] :     36 seconds : supershapes_1mode_contour --tiny_test
[ PASSED ] :     16 seconds : thin_cavity_bean --tiny_test
[ PASSED ] :     31 seconds : ellipsoid_multiple_domain --tiny_test
[ PASSED ] :     23 seconds : ellipsoid_multiple_domain_mesh --tiny_test
[ PASSED ] :     12 seconds : ellipsoid_pca --tiny_test
[ PASSED ] :     83 seconds : ellipsoid --verify
[ PASSED ] :     78 seconds : ellipsoid_cut --verify
[ PASSED ] :     36 seconds : ellipsoid_fd --verify
[ PASSED ] :     41 seconds : ellipsoid_mesh --verify
[ PASSED ] :    410 seconds : ellipsoid_multiple_domain --verify
[ PASSED ] :    184 seconds : ellipsoid_multiple_domain_mesh --verify
[ PASSED ] :   7636 seconds : femur --verify
[ PASSED ] :   8507 seconds : femur --groom_images --verify
[ PASSED ] :    525 seconds : femur_mesh --verify
[ PASSED ] :   7433 seconds : femur_cut --verify
[ PASSED ] :   6244 seconds : femur_cut --groom_images --verify
[ PASSED ] :   4341 seconds : left_atrium --verify
[ PASSED ] :   3119 seconds : left_atrium --groom_images --verify
[ PASSED ] :    429 seconds : lumps --verify
[ PASSED ] :   1658 seconds : thin_cavity_bean --verify
[ PASSED ] :     36 seconds : supershapes_1mode_contour --verify

---------------------------------------------

100.00% tests passed, 0 failed out of 40

Total Test time: 41687 seconds
akenmorris commented 3 years ago

RC6 on Mac:

[ PASSED ] : 26 seconds : ellipsoid_fd --tiny_test [ PASSED ] : 16 seconds : ellipsoid_cut --tiny_test [ PASSED ] : 12 seconds : lumps --tiny_test [ PASSED ] : 38 seconds : left_atrium --tiny_test [ PASSED ] : 55 seconds : femur --tiny_test [ PASSED ] : 14 seconds : ellipsoid --tiny_test [ PASSED ] : 10 seconds : ellipsoid_mesh --tiny_test [ PASSED ] : 24 seconds : ellipsoid_fd --tiny_test [ PASSED ] : 14 seconds : ellipsoid_cut --tiny_test [ PASSED ] : 10 seconds : lumps --tiny_test [ PASSED ] : 28 seconds : left_atrium --tiny_test [ PASSED ] : 21 seconds : femur --tiny_test [ PASSED ] : 71 seconds : femur --groom_images --tiny_test [ PASSED ] : 22 seconds : femur_mesh --tiny_test [ PASSED ] : 54 seconds : femur_cut --tiny_test [ PASSED ] : 74 seconds : femur_cut --groom_images --tiny_test [ PASSED ] : 90 seconds : deep_ssm --tiny_test [ PASSED ] : 34 seconds : supershapes_1mode_contour --tiny_test [ PASSED ] : 10 seconds : thin_cavity_bean --tiny_test [ PASSED ] : 18 seconds : ellipsoid_multiple_domain --tiny_test [ PASSED ] : 12 seconds : ellipsoid_multiple_domain_mesh --tiny_test [ PASSED ] : 4 seconds : ellipsoid_pca --tiny_test [ PASSED ] : 90 seconds : ellipsoid --verify [ PASSED ] : 73 seconds : ellipsoid_cut --verify [ PASSED ] : 24 seconds : ellipsoid_fd --verify [ PASSED ] : 44 seconds : ellipsoid_mesh --verify [ PASSED ] : 537 seconds : ellipsoid_multiple_domain --verify [ PASSED ] : 334 seconds : ellipsoid_multiple_domain_mesh --verify [ PASSED ] : 11024 seconds : femur --verify [ PASSED ] : 11734 seconds : femur --groom_images --verify [ PASSED ] : 917 seconds : femur_mesh --verify [ PASSED ] : 11018 seconds : femur_cut --verify [ PASSED ] : 12882 seconds : femur_cut --groom_images --verify [ PASSED ] : 5271 seconds : left_atrium --verify [ PASSED ] : 5340 seconds : left_atrium --groom_images --verify [ PASSED ] : 783 seconds : lumps --verify [ PASSED ] : 4715 seconds : thin_cavity_bean --verify [ PASSED ] : 46 seconds : supershapes_1mode_contour --verify

100.00% tests passed, 0 failed out of 40

Total Test time: 65601 seconds

akenmorris commented 3 years ago

RC6 on Windows:

---------------------------------------------
Testing Results:
---------------------------------------------
The following use cases passed (40)
[ PASSED ] :     18 seconds : ellipsoid --tiny_test
[ PASSED ] :     13 seconds : ellipsoid_mesh --tiny_test
[ PASSED ] :     32 seconds : ellipsoid_fd --tiny_test
[ PASSED ] :     21 seconds : ellipsoid_cut --tiny_test
[ PASSED ] :     14 seconds : lumps --tiny_test
[ PASSED ] :     40 seconds : left_atrium --tiny_test
[ PASSED ] :     22 seconds : femur --tiny_test
[ PASSED ] :     15 seconds : ellipsoid --tiny_test
[ PASSED ] :     11 seconds : ellipsoid_mesh --tiny_test
[ PASSED ] :     24 seconds : ellipsoid_fd --tiny_test
[ PASSED ] :     17 seconds : ellipsoid_cut --tiny_test
[ PASSED ] :     10 seconds : lumps --tiny_test
[ PASSED ] :     27 seconds : left_atrium --tiny_test
[ PASSED ] :     21 seconds : femur --tiny_test
[ PASSED ] :     69 seconds : femur --groom_images --tiny_test
[ PASSED ] :     16 seconds : femur_mesh --tiny_test
[ PASSED ] :     80 seconds : femur_cut --tiny_test
[ PASSED ] :     70 seconds : femur_cut --groom_images --tiny_test
[ PASSED ] :     77 seconds : deep_ssm --tiny_test
[ PASSED ] :     40 seconds : supershapes_1mode_contour --tiny_test
[ PASSED ] :     11 seconds : thin_cavity_bean --tiny_test
[ PASSED ] :     20 seconds : ellipsoid_multiple_domain --tiny_test
[ PASSED ] :     14 seconds : ellipsoid_multiple_domain_mesh --tiny_test
[ PASSED ] :      3 seconds : ellipsoid_pca --tiny_test
[ PASSED ] :     92 seconds : ellipsoid --verify
[ PASSED ] :     95 seconds : ellipsoid_cut --verify
[ PASSED ] :     27 seconds : ellipsoid_fd --verify
[ PASSED ] :     51 seconds : ellipsoid_mesh --verify
[ PASSED ] :    527 seconds : ellipsoid_multiple_domain --verify
[ PASSED ] :    314 seconds : ellipsoid_multiple_domain_mesh --verify
[ PASSED ] :  10425 seconds : femur --verify
[ PASSED ] :  10957 seconds : femur --groom_images --verify
[ PASSED ] :   1202 seconds : femur_mesh --verify
[ PASSED ] :  11932 seconds : femur_cut --verify
[ PASSED ] :  12688 seconds : femur_cut --groom_images --verify
[ PASSED ] :   5157 seconds : left_atrium --verify
[ PASSED ] :   5517 seconds : left_atrium --groom_images --verify
[ PASSED ] :   1005 seconds : lumps --verify
[ PASSED ] :   8059 seconds : thin_cavity_bean --verify
[ PASSED ] :     44 seconds : supershapes_1mode_contour --verify

---------------------------------------------

100.00% tests passed, 0 failed out of 40

Total Test time: 68778 seconds
akenmorris commented 3 years ago

RC8 Windows:

Testing Results:

The following use cases passed (40) [ PASSED ] : 45 seconds : ellipsoid --tiny_test [ PASSED ] : 15 seconds : ellipsoid_mesh --tiny_test [ PASSED ] : 34 seconds : ellipsoid_fd --tiny_test [ PASSED ] : 22 seconds : ellipsoid_cut --tiny_test [ PASSED ] : 17 seconds : lumps --tiny_test [ PASSED ] : 50 seconds : left_atrium --tiny_test [ PASSED ] : 107 seconds : femur --tiny_test [ PASSED ] : 16 seconds : ellipsoid --tiny_test [ PASSED ] : 11 seconds : ellipsoid_mesh --tiny_test [ PASSED ] : 25 seconds : ellipsoid_fd --tiny_test [ PASSED ] : 17 seconds : ellipsoid_cut --tiny_test [ PASSED ] : 11 seconds : lumps --tiny_test [ PASSED ] : 27 seconds : left_atrium --tiny_test [ PASSED ] : 22 seconds : femur --tiny_test [ PASSED ] : 70 seconds : femur --groom_images --tiny_test [ PASSED ] : 17 seconds : femur_mesh --tiny_test [ PASSED ] : 105 seconds : femur_cut --tiny_test [ PASSED ] : 71 seconds : femur_cut --groom_images --tiny_test [ PASSED ] : 118 seconds : deep_ssm --tiny_test [ PASSED ] : 40 seconds : supershapes_1mode_contour --tiny_test [ PASSED ] : 11 seconds : thin_cavity_bean --tiny_test [ PASSED ] : 20 seconds : ellipsoid_multiple_domain --tiny_test [ PASSED ] : 14 seconds : ellipsoid_multiple_domain_mesh --tiny_test [ PASSED ] : 3 seconds : ellipsoid_pca --tiny_test [ PASSED ] : 104 seconds : ellipsoid --verify [ PASSED ] : 103 seconds : ellipsoid_cut --verify [ PASSED ] : 30 seconds : ellipsoid_fd --verify [ PASSED ] : 58 seconds : ellipsoid_mesh --verify [ PASSED ] : 593 seconds : ellipsoid_multiple_domain --verify [ PASSED ] : 344 seconds : ellipsoid_multiple_domain_mesh --verify [ PASSED ] : 11822 seconds : femur --verify [ PASSED ] : 11077 seconds : femur --groom_images --verify [ PASSED ] : 1201 seconds : femur_mesh --verify [ PASSED ] : 12112 seconds : femur_cut --verify [ PASSED ] : 12766 seconds : femur_cut --groom_images --verify [ PASSED ] : 5452 seconds : left_atrium --verify [ PASSED ] : 5744 seconds : left_atrium --groom_images --verify [ PASSED ] : 841 seconds : lumps --verify [ PASSED ] : 8487 seconds : thin_cavity_bean --verify [ PASSED ] : 51 seconds : supershapes_1mode_contour --verify

100.00% tests passed, 0 failed out of 40

Total Test time: 71674 seconds

sheryjoe commented 3 years ago

@akenmorris are we ready to release 6.2?

akenmorris commented 3 years ago

The use cases appear to all work according to the verify script. I have not examined the shape models in Studio yet.

Someone needs to re-test all of the jupyter notebooks on all three platforms.

We also need to resolve #1580.

sheryjoe commented 3 years ago

@jadie1 @iyerkrithika21 could you re-test the notebooks?

akenmorris commented 3 years ago

Linux RC8:

Testing Results:

The following use cases passed (40) [ PASSED ] : 23 seconds : ellipsoid --tiny_test [ PASSED ] : 15 seconds : ellipsoid_mesh --tiny_test [ PASSED ] : 30 seconds : ellipsoid_fd --tiny_test [ PASSED ] : 29 seconds : ellipsoid_cut --tiny_test [ PASSED ] : 14 seconds : lumps --tiny_test [ PASSED ] : 36 seconds : left_atrium --tiny_test [ PASSED ] : 56 seconds : femur --tiny_test [ PASSED ] : 19 seconds : ellipsoid --tiny_test [ PASSED ] : 13 seconds : ellipsoid_mesh --tiny_test [ PASSED ] : 26 seconds : ellipsoid_fd --tiny_test [ PASSED ] : 26 seconds : ellipsoid_cut --tiny_test [ PASSED ] : 12 seconds : lumps --tiny_test [ PASSED ] : 26 seconds : left_atrium --tiny_test [ PASSED ] : 22 seconds : femur --tiny_test [ PASSED ] : 67 seconds : femur --groom_images --tiny_test [ PASSED ] : 16 seconds : femur_mesh --tiny_test [ PASSED ] : 55 seconds : femur_cut --tiny_test [ PASSED ] : 70 seconds : femur_cut --groom_images --tiny_test [ PASSED ] : 70 seconds : deep_ssm --tiny_test [ PASSED ] : 31 seconds : supershapes_1mode_contour --tiny_test [ PASSED ] : 10 seconds : thin_cavity_bean --tiny_test [ PASSED ] : 23 seconds : ellipsoid_multiple_domain --tiny_test [ PASSED ] : 16 seconds : ellipsoid_multiple_domain_mesh --tiny_test [ PASSED ] : 4 seconds : ellipsoid_pca --tiny_test [ PASSED ] : 69 seconds : ellipsoid --verify [ PASSED ] : 81 seconds : ellipsoid_cut --verify [ PASSED ] : 27 seconds : ellipsoid_fd --verify [ PASSED ] : 35 seconds : ellipsoid_mesh --verify [ PASSED ] : 297 seconds : ellipsoid_multiple_domain --verify [ PASSED ] : 169 seconds : ellipsoid_multiple_domain_mesh --verify [ PASSED ] : 5889 seconds : femur --verify [ PASSED ] : 7203 seconds : femur --groom_images --verify [ PASSED ] : 521 seconds : femur_mesh --verify [ PASSED ] : 7879 seconds : femur_cut --verify [ PASSED ] : 8087 seconds : femur_cut --groom_images --verify [ PASSED ] : 2984 seconds : left_atrium --verify [ PASSED ] : 3010 seconds : left_atrium --groom_images --verify [ PASSED ] : 437 seconds : lumps --verify [ PASSED ] : 1690 seconds : thin_cavity_bean --verify [ PASSED ] : 28 seconds : supershapes_1mode_contour --verify

100.00% tests passed, 0 failed out of 40

Total Test time: 39113 seconds

akenmorris commented 3 years ago

Mac RC8:

Testing Results:

The following use cases passed (40) [ PASSED ] : 48 seconds : ellipsoid --tiny_test [ PASSED ] : 12 seconds : ellipsoid_mesh --tiny_test [ PASSED ] : 26 seconds : ellipsoid_fd --tiny_test [ PASSED ] : 15 seconds : ellipsoid_cut --tiny_test [ PASSED ] : 12 seconds : lumps --tiny_test [ PASSED ] : 39 seconds : left_atrium --tiny_test [ PASSED ] : 67 seconds : femur --tiny_test [ PASSED ] : 14 seconds : ellipsoid --tiny_test [ PASSED ] : 10 seconds : ellipsoid_mesh --tiny_test [ PASSED ] : 25 seconds : ellipsoid_fd --tiny_test [ PASSED ] : 13 seconds : ellipsoid_cut --tiny_test [ PASSED ] : 10 seconds : lumps --tiny_test [ PASSED ] : 29 seconds : left_atrium --tiny_test [ PASSED ] : 21 seconds : femur --tiny_test [ PASSED ] : 74 seconds : femur --groom_images --tiny_test [ PASSED ] : 36 seconds : femur_mesh --tiny_test [ PASSED ] : 65 seconds : femur_cut --tiny_test [ PASSED ] : 75 seconds : femur_cut --groom_images --tiny_test [ PASSED ] : 97 seconds : deep_ssm --tiny_test [ PASSED ] : 34 seconds : supershapes_1mode_contour --tiny_test [ PASSED ] : 10 seconds : thin_cavity_bean --tiny_test [ PASSED ] : 18 seconds : ellipsoid_multiple_domain --tiny_test [ PASSED ] : 12 seconds : ellipsoid_multiple_domain_mesh --tiny_test [ PASSED ] : 4 seconds : ellipsoid_pca --tiny_test [ PASSED ] : 90 seconds : ellipsoid --verify [ PASSED ] : 74 seconds : ellipsoid_cut --verify [ PASSED ] : 25 seconds : ellipsoid_fd --verify [ PASSED ] : 44 seconds : ellipsoid_mesh --verify [ PASSED ] : 548 seconds : ellipsoid_multiple_domain --verify [ PASSED ] : 346 seconds : ellipsoid_multiple_domain_mesh --verify [ PASSED ] : 11130 seconds : femur --verify [ PASSED ] : 11715 seconds : femur --groom_images --verify [ PASSED ] : 887 seconds : femur_mesh --verify [ PASSED ] : 11206 seconds : femur_cut --verify [ PASSED ] : 12559 seconds : femur_cut --groom_images --verify [ PASSED ] : 5284 seconds : left_atrium --verify [ PASSED ] : 5331 seconds : left_atrium --groom_images --verify [ PASSED ] : 797 seconds : lumps --verify [ PASSED ] : 4609 seconds : thin_cavity_bean --verify [ PASSED ] : 45 seconds : supershapes_1mode_contour --verify

100.00% tests passed, 0 failed out of 40

Total Test time: 65453 seconds

archanasri commented 3 years ago

I can test the notebooks on Mac

akenmorris commented 3 years ago

You might hold off testing notebooks until after #1580 is resolved. Fixing all the incompatible mess of versions we have of conda/pip packages may end up breaking things. The notebooks would be the most likely to break since they use these extremely brittle packages like pyvista.

archanasri commented 2 years ago

Mac RC9:

Testing Results:

The following use cases passed (40) [ PASSED ] : 69 seconds : ellipsoid --tiny_test [ PASSED ] : 31 seconds : ellipsoid_mesh --tiny_test [ PASSED ] : 26 seconds : ellipsoid_fd --tiny_test [ PASSED ] : 33 seconds : ellipsoid_cut --tiny_test [ PASSED ] : 30 seconds : lumps --tiny_test [ PASSED ] : 251 seconds : left_atrium --tiny_test [ PASSED ] : 907 seconds : femur --tiny_test [ PASSED ] : 32 seconds : ellipsoid --tiny_test [ PASSED ] : 23 seconds : ellipsoid_mesh --tiny_test [ PASSED ] : 19 seconds : ellipsoid_fd --tiny_test [ PASSED ] : 25 seconds : ellipsoid_cut --tiny_test [ PASSED ] : 14 seconds : lumps --tiny_test [ PASSED ] : 44 seconds : left_atrium --tiny_test [ PASSED ] : 40 seconds : femur --tiny_test [ PASSED ] : 96 seconds : femur --groom_images --tiny_test [ PASSED ] : 48 seconds : femur_mesh --tiny_test [ PASSED ] : 949 seconds : femur_cut --tiny_test [ PASSED ] : 102 seconds : femur_cut --groom_images --tiny_test [ PASSED ] : 309 seconds : deep_ssm --tiny_test [ PASSED ] : 36 seconds : supershapes_1mode_contour --tiny_test [ PASSED ] : 25 seconds : thin_cavity_bean --tiny_test [ PASSED ] : 31 seconds : ellipsoid_multiple_domain --tiny_test [ PASSED ] : 27 seconds : ellipsoid_multiple_domain_mesh --tiny_test [ PASSED ] : 3 seconds : ellipsoid_pca --tiny_test [ PASSED ] : 78 seconds : ellipsoid --verify [ PASSED ] : 80 seconds : ellipsoid_cut --verify [ PASSED ] : 20 seconds : ellipsoid_fd --verify [ PASSED ] : 41 seconds : ellipsoid_mesh --verify [ PASSED ] : 556 seconds : ellipsoid_multiple_domain --verify [ PASSED ] : 353 seconds : ellipsoid_multiple_domain_mesh --verify [ PASSED ] : 11488 seconds : femur --verify [ PASSED ] : 10850 seconds : femur --groom_images --verify [ PASSED ] : 783 seconds : femur_mesh --verify [ PASSED ] : 10748 seconds : femur_cut --verify [ PASSED ] : 11377 seconds : femur_cut --groom_images --verify [ PASSED ] : 8340 seconds : left_atrium --verify [ PASSED ] : 5377 seconds : left_atrium --groom_images --verify [ PASSED ] : 750 seconds : lumps --verify [ PASSED ] : 5241 seconds : thin_cavity_bean --verify [ PASSED ] : 34 seconds : supershapes_1mode_contour --verify

100.00% tests passed, 0 failed out of 40

Total Test time: 69289 seconds

akenmorris commented 2 years ago

RC9 on Linux:

Testing Results:

The following use cases passed (40) [ PASSED ] : 48 seconds : ellipsoid --tiny_test [ PASSED ] : 22 seconds : ellipsoid_mesh --tiny_test [ PASSED ] : 73 seconds : ellipsoid_fd --tiny_test [ PASSED ] : 35 seconds : ellipsoid_cut --tiny_test [ PASSED ] : 29 seconds : lumps --tiny_test [ PASSED ] : 54 seconds : left_atrium --tiny_test [ PASSED ] : 35 seconds : femur --tiny_test [ PASSED ] : 23 seconds : ellipsoid --tiny_test [ PASSED ] : 17 seconds : ellipsoid_mesh --tiny_test [ PASSED ] : 44 seconds : ellipsoid_fd --tiny_test [ PASSED ] : 25 seconds : ellipsoid_cut --tiny_test [ PASSED ] : 15 seconds : lumps --tiny_test [ PASSED ] : 30 seconds : left_atrium --tiny_test [ PASSED ] : 23 seconds : femur --tiny_test [ PASSED ] : 100 seconds : femur --groom_images --tiny_test [ PASSED ] : 16 seconds : femur_mesh --tiny_test [ PASSED ] : 31 seconds : femur_cut --tiny_test [ PASSED ] : 103 seconds : femur_cut --groom_images --tiny_test [ PASSED ] : 66 seconds : deep_ssm --tiny_test [ PASSED ] : 42 seconds : supershapes_1mode_contour --tiny_test [ PASSED ] : 10 seconds : thin_cavity_bean --tiny_test [ PASSED ] : 25 seconds : ellipsoid_multiple_domain --tiny_test [ PASSED ] : 15 seconds : ellipsoid_multiple_domain_mesh --tiny_test [ PASSED ] : 5 seconds : ellipsoid_pca --tiny_test [ PASSED ] : 115 seconds : ellipsoid --verify [ PASSED ] : 103 seconds : ellipsoid_cut --verify [ PASSED ] : 39 seconds : ellipsoid_fd --verify [ PASSED ] : 37 seconds : ellipsoid_mesh --verify [ PASSED ] : 383 seconds : ellipsoid_multiple_domain --verify [ PASSED ] : 241 seconds : ellipsoid_multiple_domain_mesh --verify [ PASSED ] : 6726 seconds : femur --verify [ PASSED ] : 6680 seconds : femur --groom_images --verify [ PASSED ] : 493 seconds : femur_mesh --verify [ PASSED ] : 6035 seconds : femur_cut --verify [ PASSED ] : 6445 seconds : femur_cut --groom_images --verify [ PASSED ] : 3444 seconds : left_atrium --verify [ PASSED ] : 3145 seconds : left_atrium --groom_images --verify [ PASSED ] : 471 seconds : lumps --verify [ PASSED ] : 1729 seconds : thin_cavity_bean --verify [ PASSED ] : 33 seconds : supershapes_1mode_contour --verify

100.00% tests passed, 0 failed out of 40

Total Test time: 37005 seconds

sheryjoe commented 2 years ago

6.2 has been released. Thanks everyone!