2024 GPU Hackathon Integration

[garrettwrong]

During the hackathon we were able to implement significant speedups for FLE2D, FFB2D, and FFB3D in ASPIRE for target problem sizes (256pixel/voxel). Oscar's FLE3D code also improved but in a separate code base for his research purposes. Below is an outline of work required to integrate into our codebase.

• [x] Enable cupy for numeric and fft backend for the GPU CI jobs
• [x] Fix failing unit tests in this configuration. (to be posted below)
• [x] Code cleanup to match style/lint checks etc
• [x] Remove deprecated GPU env extensions from pyproject
• [x] Add a section to the config tutorial demonstrating this setup, add or link this to doc page.
• [x] Re-Review hackathon changes to nufft and xp wrapper (might need to add new tests)
• [ ] Resolve cupy memory/cache issue in a more robust way

[garrettwrong]

This looks like a lot, but I think many of them are the same small problems.

=============================== warnings summary ===============================
tests/test_FLEbasis2D.py::testRadialConvolution
tests/test_simulation.py::SimTestCase::testSimulationSaveFile
  /scratch/gbwright/miniconda3/envs/hack24/lib/python3.8/site-packages/cufinufft/_plan.py:336: UserWarning: Argument `data` does not satisfy the following requirement: C. Copying array (this may reduce performance)
    warnings.warn(f"Argument `{name}` does not satisfy the "

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests/test_CtfEstimate.py::test_estimate_CTF - TypeError: The input ar...
FAILED tests/test_CtfEstimate.py::testRectangularMicrograph[slice_range0] - T...
FAILED tests/test_CtfEstimate.py::testRectangularMicrograph[slice_range1] - T...
FAILED tests/test_FLEbasis2D.py::testRadialConvolution - ValueError: non-scal...
FAILED tests/test_array_image_source.py::ImageTestCase::testArrayImageSourceMeanVol
FAILED tests/test_averager2d.py::ReddyChatterjiAverager2DTestCase::testAverager
FAILED tests/test_averager2d.py::ReddyChatterjiAverager2DTestCase::test_attributes
FAILED tests/test_averager2d.py::BFSReddyChatterjiAverager2DTestCase::testAverager
FAILED tests/test_averager2d.py::BFSReddyChatterjiAverager2DTestCase::test_attributes
FAILED tests/test_coor_trans.py::UtilsTestCase::testCrop2DFillValue - ValueEr...
FAILED tests/test_coor_trans.py::UtilsTestCase::testRectPad2D - ValueError: n...
FAILED tests/test_coor_trans.py::UtilsTestCase::testSquarePad2D - ValueError:...
FAILED tests/test_covar2d_denoiser.py::test_batched_rotcov2d_MSE[basis=<class 'aspire.basis.fle_2d.FLEBasis2D'>]
FAILED tests/test_covar2d_denoiser.py::test_source_mismatch[basis=<class 'aspire.basis.fle_2d.FLEBasis2D'>]
FAILED tests/test_covar2d_denoiser.py::test_filter_to_basis_mat_id[basis=<class 'aspire.basis.fle_2d.FLEBasis2D'>]
FAILED tests/test_covar2d_denoiser.py::test_filter_to_basis_mat_ctf[basis=<class 'aspire.basis.fle_2d.FLEBasis2D'>]
FAILED tests/test_fourier_correlation.py::test_2d_stack_plot_raise - TypeErro...
FAILED tests/test_fourier_correlation.py::test_3d_stack_plot_raise - TypeErro...
FAILED tests/test_fourier_correlation.py::test_multiple_stack_plot_raise - Ty...
FAILED tests/test_fourier_kernel.py::FourierKernelTestCase::test1 - TypeError...
FAILED tests/test_mean_estimator.py::test_estimate[preconditioner=None-L=8-dtype=<class 'numpy.float32'>]
FAILED tests/test_mean_estimator.py::test_checkpoint[preconditioner=None-L=8-dtype=<class 'numpy.float32'>]
FAILED tests/test_mean_estimator.py::test_estimate[preconditioner=None-L=8-dtype=<class 'numpy.float64'>]
FAILED tests/test_mean_estimator.py::test_checkpoint[preconditioner=None-L=8-dtype=<class 'numpy.float64'>]
FAILED tests/test_mean_estimator.py::test_estimate[preconditioner=None-L=9-dtype=<class 'numpy.float64'>]
FAILED tests/test_mean_estimator.py::test_checkpoint[preconditioner=None-L=9-dtype=<class 'numpy.float64'>]
FAILED tests/test_mean_estimator.py::test_estimate[preconditioner=circulant-L=9-dtype=<class 'numpy.float64'>]
FAILED tests/test_mean_estimator.py::test_checkpoint[preconditioner=circulant-L=9-dtype=<class 'numpy.float64'>]
FAILED tests/test_mean_estimator.py::test_estimate[preconditioner=circulant-L=9-dtype=<class 'numpy.float32'>]
FAILED tests/test_mean_estimator.py::test_checkpoint[preconditioner=circulant-L=9-dtype=<class 'numpy.float32'>]
FAILED tests/test_mean_estimator.py::test_estimate[preconditioner=circulant-L=8-dtype=<class 'numpy.float64'>]
FAILED tests/test_mean_estimator.py::test_checkpoint[preconditioner=circulant-L=8-dtype=<class 'numpy.float64'>]
FAILED tests/test_mean_estimator.py::test_estimate[preconditioner=circulant-L=8-dtype=<class 'numpy.float32'>]
FAILED tests/test_mean_estimator.py::test_checkpoint[preconditioner=circulant-L=8-dtype=<class 'numpy.float32'>]
FAILED tests/test_mean_estimator.py::test_estimate[preconditioner=None-L=9-dtype=<class 'numpy.float32'>]
FAILED tests/test_mean_estimator.py::test_checkpoint[preconditioner=None-L=9-dtype=<class 'numpy.float32'>]
FAILED tests/test_mean_estimator_boosting.py::test_weighted_volumes[volume=<class 'aspire.volume.volume_synthesis.AsymmetricVolume'>, order=None-resolution=32-dtype=<class 'numpy.float32'>]
FAILED tests/test_mean_estimator_boosting.py::test_weighted_volumes[volume=<class 'aspire.volume.volume_synthesis.CnSymmetricVolume'>, order=4-resolution=32-dtype=<class 'numpy.float32'>]
FAILED tests/test_mean_estimator_boosting.py::test_weighted_volumes[volume=<class 'aspire.volume.volume_synthesis.CnSymmetricVolume'>, order=5-resolution=32-dtype=<class 'numpy.float32'>]
FAILED tests/test_numeric_sparse.py::test_eigsh[cupy] - AssertionError: 
FAILED tests/test_oriented_source.py::test_rotations[Orientation estimator: <class 'aspire.abinitio.commonline_sync.CLSyncVoting'>, Symmetry: None]
FAILED tests/test_oriented_source.py::test_angles[Orientation estimator: <class 'aspire.abinitio.commonline_sync.CLSyncVoting'>, Symmetry: None]
FAILED tests/test_oriented_source.py::test_lazy_evaluation[Orientation estimator: <class 'aspire.abinitio.commonline_sync.CLSyncVoting'>, Symmetry: None]
FAILED tests/test_relion_interop.py::test_projections_frc[rln_proj_65.star]
FAILED tests/test_relion_interop.py::test_projections_frc[rln_proj_64.star]
FAILED tests/test_simulation.py::SimTestCase::testSimulationSaveFile - Assert...
FAILED tests/test_weighted_mean_estimator.py::test_estimate[preconditioner=None-L=8-dtype=<class 'numpy.float32'>]
FAILED tests/test_weighted_mean_estimator.py::test_checkpoint[preconditioner=None-L=8-dtype=<class 'numpy.float32'>]
FAILED tests/test_weighted_mean_estimator.py::test_estimate[preconditioner=None-L=8-dtype=<class 'numpy.float64'>]
FAILED tests/test_weighted_mean_estimator.py::test_checkpoint[preconditioner=None-L=8-dtype=<class 'numpy.float64'>]
FAILED tests/test_weighted_mean_estimator.py::test_estimate[preconditioner=None-L=9-dtype=<class 'numpy.float64'>]
FAILED tests/test_weighted_mean_estimator.py::test_checkpoint[preconditioner=None-L=9-dtype=<class 'numpy.float64'>]
FAILED tests/test_weighted_mean_estimator.py::test_estimate[preconditioner=circulant-L=9-dtype=<class 'numpy.float64'>]
FAILED tests/test_weighted_mean_estimator.py::test_checkpoint[preconditioner=circulant-L=9-dtype=<class 'numpy.float64'>]
FAILED tests/test_weighted_mean_estimator.py::test_estimate[preconditioner=circulant-L=9-dtype=<class 'numpy.float32'>]
FAILED tests/test_weighted_mean_estimator.py::test_checkpoint[preconditioner=circulant-L=9-dtype=<class 'numpy.float32'>]
FAILED tests/test_weighted_mean_estimator.py::test_estimate[preconditioner=circulant-L=8-dtype=<class 'numpy.float64'>]
FAILED tests/test_weighted_mean_estimator.py::test_checkpoint[preconditioner=circulant-L=8-dtype=<class 'numpy.float64'>]
FAILED tests/test_weighted_mean_estimator.py::test_estimate[preconditioner=circulant-L=8-dtype=<class 'numpy.float32'>]
FAILED tests/test_weighted_mean_estimator.py::test_checkpoint[preconditioner=circulant-L=8-dtype=<class 'numpy.float32'>]
FAILED tests/test_weighted_mean_estimator.py::test_estimate[preconditioner=None-L=9-dtype=<class 'numpy.float32'>]
FAILED tests/test_weighted_mean_estimator.py::test_checkpoint[preconditioner=None-L=9-dtype=<class 'numpy.float32'>]
ERROR tests/test_covar3d.py::Covar3DTestCase::testClustering - TypeError: Imp...
ERROR tests/test_covar3d.py::Covar3DTestCase::testCovar3D1 - TypeError: Impli...
ERROR tests/test_covar3d.py::Covar3DTestCase::testCovar3D2 - TypeError: Impli...
ERROR tests/test_covar3d.py::Covar3DTestCase::testCovarEvaluation - TypeError...
ERROR tests/test_covar3d.py::Covar3DTestCase::testEigsEvaluation - TypeError:...
ERROR tests/test_covar3d.py::Covar3DTestCase::testMeanEvaluation - TypeError:...
ERROR tests/test_fourier_correlation.py::test_frc_id[method=fft-img_size=64-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_id[method=fft-img_size=64-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_frc_id[method=fft-img_size=65-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_id[method=fft-img_size=65-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_frc_id[method=nufft-img_size=64-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_id[method=nufft-img_size=64-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_frc_id[method=nufft-img_size=65-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_id[method=nufft-img_size=65-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_frc_trunc[method=fft-img_size=64-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_trunc[method=fft-img_size=64-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_frc_trunc[method=fft-img_size=65-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_trunc[method=fft-img_size=65-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_frc_trunc[method=nufft-img_size=64-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_trunc[method=nufft-img_size=64-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_frc_trunc[method=nufft-img_size=65-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_trunc[method=nufft-img_size=65-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_frc_noise[method=fft-img_size=64-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_noise[method=fft-img_size=64-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_frc_noise[method=fft-img_size=65-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_noise[method=fft-img_size=65-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_frc_noise[method=nufft-img_size=64-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_noise[method=nufft-img_size=64-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_frc_noise[method=nufft-img_size=65-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_noise[method=nufft-img_size=65-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_frc_img_plot[img_size=64-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_img_plot[img_size=64-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_frc_img_plot[img_size=65-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_img_plot[img_size=65-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_frc_plot[method=fft-img_size=64-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_plot[method=fft-img_size=64-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_frc_plot[method=fft-img_size=65-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_plot[method=fft-img_size=65-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_frc_plot[method=nufft-img_size=64-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_plot[method=nufft-img_size=64-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_frc_plot[method=nufft-img_size=65-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_plot[method=nufft-img_size=65-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_fsc_id[method=fft-img_size=64-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_fsc_id[method=fft-img_size=64-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_fsc_id[method=fft-img_size=65-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_fsc_id[method=fft-img_size=65-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_fsc_id[method=nufft-img_size=64-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_fsc_id[method=nufft-img_size=64-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_fsc_id[method=nufft-img_size=65-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_fsc_id[method=nufft-img_size=65-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_fsc_trunc[method=fft-img_size=64-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_fsc_trunc[method=fft-img_size=64-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_fsc_trunc[method=fft-img_size=65-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_fsc_trunc[method=fft-img_size=65-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_fsc_trunc[method=nufft-img_size=64-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_fsc_trunc[method=nufft-img_size=64-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_fsc_trunc[method=nufft-img_size=65-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_fsc_trunc[method=nufft-img_size=65-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_fsc_vol_plot[img_size=64-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_fsc_vol_plot[img_size=64-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_fsc_vol_plot[img_size=65-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_fsc_vol_plot[img_size=65-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_fsc_plot[method=fft-img_size=64-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_fsc_plot[method=fft-img_size=64-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_fsc_plot[method=fft-img_size=65-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_fsc_plot[method=fft-img_size=65-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_fsc_plot[method=nufft-img_size=64-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_fsc_plot[method=nufft-img_size=64-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_fsc_plot[method=nufft-img_size=65-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_fsc_plot[method=nufft-img_size=65-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_frc_id_bcast[method=fft-img_size=64-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_id_bcast[method=fft-img_size=64-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_frc_id_bcast[method=fft-img_size=65-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_id_bcast[method=fft-img_size=65-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_frc_id_bcast[method=nufft-img_size=64-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_id_bcast[method=nufft-img_size=64-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_frc_id_bcast[method=nufft-img_size=65-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_id_bcast[method=nufft-img_size=65-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_fsc_id_bcast[method=fft-img_size=64-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_fsc_id_bcast[method=fft-img_size=64-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_fsc_id_bcast[method=fft-img_size=65-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_fsc_id_bcast[method=fft-img_size=65-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_fsc_id_bcast[method=nufft-img_size=64-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_fsc_id_bcast[method=nufft-img_size=64-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_fsc_id_bcast[method=nufft-img_size=65-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_fsc_id_bcast[method=nufft-img_size=65-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_frc_img_plot_bcast[img_size=64-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_img_plot_bcast[img_size=64-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_frc_img_plot_bcast[img_size=65-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_frc_img_plot_bcast[img_size=65-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_plot_bad_bcast[img_size=64-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_plot_bad_bcast[img_size=64-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_plot_bad_bcast[img_size=65-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_plot_bad_bcast[img_size=65-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_plot_labels[img_size=64-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_plot_labels[img_size=64-dtype=<class 'numpy.float32'>]
ERROR tests/test_fourier_correlation.py::test_plot_labels[img_size=65-dtype=<class 'numpy.float64'>]
ERROR tests/test_fourier_correlation.py::test_plot_labels[img_size=65-dtype=<class 'numpy.float32'>]
ERROR tests/test_mean_estimator_boosting.py::test_fsc[volume=<class 'aspire.volume.volume_synthesis.AsymmetricVolume'>, order=None-resolution=32-dtype=<class 'numpy.float32'>]
ERROR tests/test_mean_estimator_boosting.py::test_mse[volume=<class 'aspire.volume.volume_synthesis.AsymmetricVolume'>, order=None-resolution=32-dtype=<class 'numpy.float32'>]
ERROR tests/test_mean_estimator_boosting.py::test_total_energy[volume=<class 'aspire.volume.volume_synthesis.AsymmetricVolume'>, order=None-resolution=32-dtype=<class 'numpy.float32'>]
ERROR tests/test_mean_estimator_boosting.py::test_boost_flag[volume=<class 'aspire.volume.volume_synthesis.AsymmetricVolume'>, order=None-resolution=32-dtype=<class 'numpy.float32'>]
ERROR tests/test_mean_estimator_boosting.py::test_fsc[volume=<class 'aspire.volume.volume_synthesis.CnSymmetricVolume'>, order=4-resolution=32-dtype=<class 'numpy.float32'>]
ERROR tests/test_mean_estimator_boosting.py::test_mse[volume=<class 'aspire.volume.volume_synthesis.CnSymmetricVolume'>, order=4-resolution=32-dtype=<class 'numpy.float32'>]
ERROR tests/test_mean_estimator_boosting.py::test_total_energy[volume=<class 'aspire.volume.volume_synthesis.CnSymmetricVolume'>, order=4-resolution=32-dtype=<class 'numpy.float32'>]
ERROR tests/test_mean_estimator_boosting.py::test_boost_flag[volume=<class 'aspire.volume.volume_synthesis.CnSymmetricVolume'>, order=4-resolution=32-dtype=<class 'numpy.float32'>]
ERROR tests/test_mean_estimator_boosting.py::test_fsc[volume=<class 'aspire.volume.volume_synthesis.CnSymmetricVolume'>, order=5-resolution=32-dtype=<class 'numpy.float32'>]
ERROR tests/test_mean_estimator_boosting.py::test_mse[volume=<class 'aspire.volume.volume_synthesis.CnSymmetricVolume'>, order=5-resolution=32-dtype=<class 'numpy.float32'>]
ERROR tests/test_mean_estimator_boosting.py::test_total_energy[volume=<class 'aspire.volume.volume_synthesis.CnSymmetricVolume'>, order=5-resolution=32-dtype=<class 'numpy.float32'>]
ERROR tests/test_mean_estimator_boosting.py::test_boost_flag[volume=<class 'aspire.volume.volume_synthesis.CnSymmetricVolume'>, order=5-resolution=32-dtype=<class 'numpy.float32'>]
ERROR tests/test_mean_estimator_boosting.py::test_fsc[volume=<class 'aspire.volume.volume_synthesis.DnSymmetricVolume'>, order=2-resolution=32-dtype=<class 'numpy.float32'>]
ERROR tests/test_mean_estimator_boosting.py::test_mse[volume=<class 'aspire.volume.volume_synthesis.DnSymmetricVolume'>, order=2-resolution=32-dtype=<class 'numpy.float32'>]
ERROR tests/test_mean_estimator_boosting.py::test_total_energy[volume=<class 'aspire.volume.volume_synthesis.DnSymmetricVolume'>, order=2-resolution=32-dtype=<class 'numpy.float32'>]
ERROR tests/test_mean_estimator_boosting.py::test_boost_flag[volume=<class 'aspire.volume.volume_synthesis.DnSymmetricVolume'>, order=2-resolution=32-dtype=<class 'numpy.float32'>]
========= 62 failed, 122 deselected, 2 warnings, 114 errors in 44.24s ==========

[garrettwrong]

So far I have resolved all but the following:

FAILED tests/test_covar2d_denoiser.py::test_batched_rotcov2d_MSE[basis=<class 'aspire.basis.fle_2d.FLEBasis2D'>] - ValueError: matmul: Input operand 0 does not have enough dimensions (has 0, gufunc core with signature (n?,k),(k,m?)->(n?,m?) requires 1)
FAILED tests/test_covar2d_denoiser.py::test_source_mismatch[basis=<class 'aspire.basis.fle_2d.FLEBasis2D'>] - ValueError: matmul: Input operand 0 does not have enough dimensions (has 0, gufunc core with signature (n?,k),(k,m?)->(n?,m?) requires 1)
FAILED tests/test_covar2d_denoiser.py::test_filter_to_basis_mat_id[basis=<class 'aspire.basis.fle_2d.FLEBasis2D'>] - ValueError: matmul: Input operand 0 does not have enough dimensions (has 0, gufunc core with signature (n?,k),(k,m?)->(n?,m?) requires 1)
FAILED tests/test_covar2d_denoiser.py::test_filter_to_basis_mat_ctf[basis=<class 'aspire.basis.fle_2d.FLEBasis2D'>] - ValueError: matmul: Input operand 0 does not have enough dimensions (has 0, gufunc core with signature (n?,k),(k,m?)->(n?,m?) requires 1)
FAILED tests/test_simulation.py::SimTestCase::testSimulationSaveFile - AssertionError: False is not true

Some of them can be optimized, but before I get further into that I want to cover the initial tasks in this issue... so that we can actually take the changes...

[janden]

Some of them can be optimized, but before I get further into that I want to cover the initial tasks in this issue...

I'm not following. How is this about optimization?

[garrettwrong]

Some of them can be optimized, but before I get further into that I want to cover the initial tasks in this issue...

I'm not following. How is this about optimization?

Optimization is the reason to use a GPU... The large list of test failures have a lot of code paths that can be made just to function (ie not crash passing tests and ""run on the gpu""), but requires some code refactoring to use the GPU effectively (speed, less copies etc).

All I'm saying here is that the plan is to first make the things work and then make them better. I think I have almost all the unit tests working now. The issues with that small group above is actually due poor accuracy of Cupy's FFT in single precision which took me most of Friday to figure out. Hoping to check a few more boxes off today.

[garrettwrong]

Happy to report the unit tests appear to be functional now in both numpy and cupy modes, with cupy mode now used for the ampere_gpu CI workflow.