Release notes for 6.0 - Githubissues

sheryjoe commented 3 years ago

Please add to this issue a paragraph (plus videos/snapshots for illustration) about each new/improved/bug-fix included in this release. I will consolidate and add to the documentation.

akenmorris commented 3 years ago

Studio: • Added support for mesh inputs with minimal grooming. • Added a new surface reconstruction method with support for both mesh or image inputs. This method is much faster and is the new default. • Added support for loading and displaying scalar values from mesh inputs • Improved particle shape statistics computation speed • Improved UI responsiveness during optimization • Added UI tooltips • Added Help->Keyboard shortcuts • Show current iteration, number of particles in statusbar during optimization • Added ability to choose orientation marker (medical, triad) and location (corner) • Added ability to use 2 viewer (inbetween 1 and 4) • Added ability to abort grooming step • Added new checkbox for automatic glyph sizing • Added support for drag and drop of images and meshes • Improved distance transform loading for surface reconstruction • Fixed various bugs (#848, #856, #880, #892, #265, #963, #959, #961)

Optimizer: • Replaced backend mesh library to fix bugs (#925) • Switched from OpenMP to TBB allowing multithreading on Mac (#788)

Screenshot showing open meshes in Studio:

sheryjoe commented 3 years ago

Thanks @akenmorris! Could you add a "fancy" snapshot that showcase mesh and feature support in Studio?

medakk commented 3 years ago

Support for use_normals in mesh domain: this results in improved shape models on thin domains:
Gradient of normals for image domain: this corrects a long standing bug where we used the hessian in place of the gradient of the normal. If you have an existing use case with use_normals enabled, the normal weighting may have to be adjusted
Performance improvements in shapeworks optimize:
- ~20% faster optimization in all domains, from replacing certain data structures (that were originally in place to allow interactive removal of particles during the optimization)
- ~50% faster optimization in mesh domain, from caching nearest-triangle lookups

sheryjoe commented 3 years ago

@medakk thanks!

could you add a snapshot for a couple of thin ellispsoid samples with their partices? the view should demonstrate the thin aspect of it.
for performance improvement, i suppose this is memory improvement, right?

medakk commented 3 years ago

@sheryjoe

could you add a snapshot

Sure!

i suppose this is memory improvement, right?

No, its faster runtime.

I updated my original comment

jadie1 commented 3 years ago

DeepSSM:

DeepSSM now saves both the "best" and "final" model. The final model is saved after all training epochs have run. The best model is saved after the epoch which had the lowest prediction error on the validation set. The best model makes use of early stopping to prevent overfitting.
The error meshes that are output from running the DeepSSM use case can now be visualized in Studio. These meshes have a distance scalar field on them which captures the distance between the true and predicted mesh. To view in studio simply run: ShapeWorksStudio path/to/error/mesh.vtk.

Data Augmentation:

In data augmentation the user can now either specify how many PCA components to retain in embedding OR what percentage of population variability to retain. For example, if the user specifies that 95% of population variability should be kept, then the number of components will be automatically selected such that less than 5% of shape variation is lost in embedding.
Data augmentation can now be run on a dataset for which Procrustes was used in optimization. When both the local and world .particle files are passed as arguments for data augmentation, the translation is accounted for in the augmented data.
A jupyter notebook which demonstrates the data augmentation process has been added (ShapeWorks/DeepSSMViz/Examples/Python/notebooks/tutorials/getting-started-with-data-augmentation.ipynb). In this notebook, parallel violin plots are used to visually compare the distribution of real and augmented data.

Use cases:

All use cases now have a tiny test which can be run using the--tiny_test option. When the tiny test is run, only the data necessary for the test is downloaded rather than all of the data.
All of the use cases (mesh or segmentation based) can now be run on a subset of the data using the --use_subsample option. Note that the entire dataset is downloaded in this case so that a subset which is representative of the entire dataset can be selected.
GroomUtils.pynow makes use of Python binding rather than calling command line tools.

Generating shape cohorts:

Example shape cohorts can now be generated using the ShapeWorks package GenerateShapeCohort. Currently cohorts of parameterized ellipsoids or supershapes can be generated. Options are available to specify the degree to which the cohort is groomed (i.e. a cohort can be generated to be in alignment or misaligned in various ways).
A notebook was added (ShapeWorks/DeepSSMViz/Examples/Python/notebooks/tutorials/getting-started-with-shape-cohort-generation.ipynb) which demonstrates how to use GenerateShapeCohort.

sheryjoe commented 3 years ago

Please add/update this issue with release notes (including bullet points, snapshots/videos where possible) pertaining to issues you have resolved and merged to alpha/master. @cchriste @akenmorris @archanasri @riddhishb @medakk @iyerkrithika21 @jadie1 @HeavenlyBerserker

archanasri commented 3 years ago

Updated ShapeWorks API: Consolidation of mesh-based grooming tools (smooth, decimate, invert normals, reflect, alignment, fill holes, probe volume at mesh vertices, clip, translate, scale, bounding box, quality control, surface to surface distance, to image and to distance transform). Addition of mesh-based query tools (center, center of mass, number of points, number of faces, get field names, set field, get field, set field value, get field value, get field range, get field mean, get field std and comparison). This includes a full complement of unit tests.

New Python ShapeWorks API: In addition to command line and C++ interface, there is Python interface for existing image-based, segmentation-based and mesh-based grooming tools and query tools. This includes a full complement of unit tests.

iyerkrithika21 commented 3 years ago

DeepSSM:

* DeepSSM now saves both the "best" and "final" model. The final model is saved after all training epochs have run. The best model is saved after the epoch which had the lowest prediction error on the validation set. The best model makes use of early stopping to prevent overfitting.

* The error meshes that are output from running the DeepSSM use case can now be visualized in Studio. These meshes have a distance scalar field on them which captures the distance between the true and predicted mesh. To view in studio simply run: `ShapeWorksStudio path/to/error/mesh.vtk`.

Data Augmentation:

* In data augmentation the user can now either specify how many PCA components to retain in embedding OR what percentage of population variability to retain.  For example, if the user specifies that 95% of population variability should be kept, then the number of components will be automatically selected such that less than 5% of shape variation is lost in embedding.

* Data augmentation can now be run on a dataset for which Procrustes was used in optimization.  When both the local and world .particle files are passed as arguments for data augmentation, the translation is accounted for in the augmented data.

* A jupyter notebook which demonstrates the data augmentation process has been added (`ShapeWorks/DeepSSMViz/Examples/Python/notebooks/tutorials/getting-started-with-data-augmentation.ipynb`). In this notebook, parallel violin plots are used to visually compare the distribution of real and augmented data.

Use cases:

* All use cases now have a tiny test which can be run using the` --tiny_test` option. When the tiny test is run, only the data necessary for the test is downloaded rather than all of the data.

* All of the use cases (mesh or segmentation based) can now be run on a subset of the data using the `--use_subsample` option. Note that the entire dataset is downloaded in this case so that a subset which is representative of the entire dataset can be selected.

* `GroomUtils.py `now makes use of Python binding rather than calling command line tools.

Generating shape cohorts:

* Example shape cohorts can now be generated using the ShapeWorks package `GenerateShapeCohort`.  Currently cohorts of parameterized ellipsoids or supershapes can be generated. Options are available to specify the degree to which the cohort is groomed (i.e. a cohort can be generated to be in alignment or misaligned in various ways).

* A notebook was added (`ShapeWorks/DeepSSMViz/Examples/Python/notebooks/tutorials/getting-started-with-shape-cohort-generation.ipynb`) which demonstrates how to use `GenerateShapeCohort`.

Videos and images for shape cohort generator:

ellipsoid_segs supershapes_Segs supershaped_generate_shapes

ellipsoid_generate_shapes

sheryjoe commented 3 years ago

• Added support for loading and displaying scalar values from feature map volumes

@akenmorris wasn't this already included in release 5.5? what is new in release 6?

sheryjoe commented 3 years ago

DeepSSM:

DeepSSM now saves both the "best" and "final" model. The final model is saved after all training epochs have run. The best model is saved after the epoch which had the lowest prediction error on the validation set. The best model makes use of early stopping to prevent overfitting.

The error meshes that are output from running the DeepSSM use case can now be visualized in Studio. These meshes have a distance scalar field on them which captures the distance between the true and predicted mesh. To view in studio simply run: ShapeWorksStudio path/to/error/mesh.vtk.

Data Augmentation:

In data augmentation the user can now either specify how many PCA components to retain in embedding OR what percentage of population variability to retain. For example, if the user specifies that 95% of population variability should be kept, then the number of components will be automatically selected such that less than 5% of shape variation is lost in embedding.

Data augmentation can now be run on a dataset for which Procrustes was used in optimization. When both the local and world .particle files are passed as arguments for data augmentation, the translation is accounted for in the augmented data.

A jupyter notebook which demonstrates the data augmentation process has been added (ShapeWorks/DeepSSMViz/Examples/Python/notebooks/tutorials/getting-started-with-data-augmentation.ipynb). In this notebook, parallel violin plots are used to visually compare the distribution of real and augmented data.

@jadie1 did you update documentation to reflect the above?

akenmorris commented 3 years ago

• Added support for loading and displaying scalar values from feature map volumes

@akenmorris wasn't this already included in release 5.5? what is new in release 6?

Yes, you're right, I think it got included because we didn't close the issue until just recently. I will update the list above.

sheryjoe commented 3 years ago

@sheryjoe add fix in #1123 to the release notes.

jadie1 commented 3 years ago

DeepSSM:

DeepSSM now saves both the "best" and "final" model. The final model is saved after all training epochs have run. The best model is saved after the epoch which had the lowest prediction error on the validation set. The best model makes use of early stopping to prevent overfitting.

The error meshes that are output from running the DeepSSM use case can now be visualized in Studio. These meshes have a distance scalar field on them which captures the distance between the true and predicted mesh. To view in studio simply run: ShapeWorksStudio path/to/error/mesh.vtk.

Data Augmentation:

In data augmentation the user can now either specify how many PCA components to retain in embedding OR what percentage of population variability to retain. For example, if the user specifies that 95% of population variability should be kept, then the number of components will be automatically selected such that less than 5% of shape variation is lost in embedding.

Data augmentation can now be run on a dataset for which Procrustes was used in optimization. When both the local and world .particle files are passed as arguments for data augmentation, the translation is accounted for in the augmented data.

A jupyter notebook which demonstrates the data augmentation process has been added (ShapeWorks/DeepSSMViz/Examples/Python/notebooks/tutorials/getting-started-with-data-augmentation.ipynb). In this notebook, parallel violin plots are used to visually compare the distribution of real and augmented data.

@jadie1 did you update documentation to reflect the above?

@sheryjoe The documentation was updated for everything except the studio visualization. I just added that on the mkdocs branch: https://github.com/SCIInstitute/ShapeWorks/blob/mkdocs/docs/deep-learning/deep-ssm.md

sheryjoe commented 3 years ago

@akenmorris @medakk @jadie1 @iyerkrithika21 @HeavenlyBerserker @riddhishb

What do you think of this as the graphical abstract for R6?

medakk commented 3 years ago

@sheryjoe This looks great! (I still have the same nitpick as last time, I'd prefer that the "ShapeWorks 6.0" text not obscure the logo)

sheryjoe commented 3 years ago

@sheryjoe This looks great! (I still have the same nitpick as last time, I'd prefer that the "ShapeWorks 6.0" text not obscure the logo)

@akenmorris @medakk @jadie1 @iyerkrithika21 @HeavenlyBerserker @riddhishb

Whenever I move the logo or the text, symmetries are lost. How about this one? I am open to any suggestions here.

akenmorris commented 3 years ago

I think the earlier one was better. The text/logo doesn't bother me the way it does @medakk :)

sheryjoe commented 3 years ago

1:1 now :) ... any other voters? comments? suggestions?

iyerkrithika21 commented 3 years ago

2:1 :-D I felt the earlier version was better as the logo was visible. In the second version, the logo watermark is too faint in the background.

jadie1 commented 3 years ago

I agree, 3:1

archanasri commented 3 years ago

3:2

sheryjoe commented 3 years ago

not a consensus yet, need more voters. @HeavenlyBerserker @riddhishb ?

medakk commented 3 years ago

I felt the earlier version was better as the logo was visible. In the second version, the logo watermark is too faint in the background.

I agree with this, I prefer the first one too. Maybe make the whole image a little bigger so we can fit logo+name separately

sheryjoe commented 3 years ago

Final call for voters!!

@akenmorris @archanasri @cchriste @medakk @jadie1 @iyerkrithika21 @HeavenlyBerserker @riddhishb [A]

veruss [B]

akenmorris commented 3 years ago

A

Also, if you make them separate comments we can vote with thumbs up on each.

jadie1 commented 3 years ago

A

On Wed, Mar 10, 2021 at 9:05 AM Alan Morris @.***> wrote:

A

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/SCIInstitute/ShapeWorks/issues/893#issuecomment-795660228, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM65AFOX7OS3DMGNFHQZNELTC6DFJANCNFSM4WA33UBQ .

medakk commented 3 years ago

B

iyerkrithika21 commented 3 years ago

B

archanasri commented 3 years ago

B

sheryjoe commented 3 years ago

2:3 @HeavenlyBerserker @cchriste @riddhishb , please VOTE!!

HeavenlyBerserker commented 3 years ago

A

riddhishb commented 3 years ago

A

sheryjoe commented 3 years ago

Update release notes and abstract to de-emphasize python stuff. See #1168

sheryjoe commented 3 years ago

done.

SCIInstitute / ShapeWorks

Release notes for 6.0 #893