search

ptycho / ptypy

Ptypy - main repository
Other
36 stars 16 forks source link

Using stochastic reconstruction method (EPIE) with PyCUDA problem #523

Closed ltang320 closed 8 months ago

ltang320 commented 9 months ago

Hi, I encountered an issue when I used EPIE to do the reconstruction.

p.engines.engine00 = u.Param() p.engines.engine00.name = 'EPIE_pycuda' p.engines.engine00.numiter = 1500 # number of iterations

The error told me, "The stochastic engines are incompatible with MPI." When I check the code in /type/type/accelerate/cuda_pycuda/ engines/stochastic.py, EPIE supports GPU. However, it doesn't support MPI.

MPI = False

Is it possible to make EPIE to support MPI? @bjoernenders @daurer Thanks a lot for your help!

Traceback (most recent call last):
  File "/asap3/petra3/gpfs/p06/2023/data/11017456/processed/macros/ptypy/TL/rec_TL_multi_3x4_initProb_GPU_tomo_01_epie.py", line 332, in <module>
Traceback (most recent call last):
  File "/asap3/petra3/gpfs/p06/2023/data/11017456/processed/macros/ptypy/TL/rec_TL_multi_3x4_initProb_GPU_tomo_01_epie.py", line 332, in <module>
Traceback (most recent call last):
  File "/asap3/petra3/gpfs/p06/2023/data/11017456/processed/macros/ptypy/TL/rec_TL_multi_3x4_initProb_GPU_tomo_01_epie.py", line 332, in <module>
Traceback (most recent call last):
  File "/asap3/petra3/gpfs/p06/2023/data/11017456/processed/macros/ptypy/TL/rec_TL_multi_3x4_initProb_GPU_tomo_01_epie.py", line 332, in <module>
    P = Ptycho(p, level=4)
        ^^^^^^^^^^^^^^^^^^
  File "/home/litang/ptypy_p06/build/lib/ptypy/core/ptycho.py", line 382, in __init__
    P = Ptycho(p, level=4)
    P = Ptycho(p, level=4)
        ^^^^^^^^^^^^^^^^^^
  File "/home/litang/ptypy_p06/build/lib/ptypy/core/ptycho.py", line 382, in __init__
    P = Ptycho(p, level=4)
        ^^^^^^^^^^^^^^^^^^
  File "/home/litang/ptypy_p06/build/lib/ptypy/core/ptycho.py", line 382, in __init__
        ^^^^^^^^^^^^^^^^^^
  File "/home/litang/ptypy_p06/build/lib/ptypy/core/ptycho.py", line 382, in __init__
    self.init_engine()
  File "/home/litang/ptypy_p06/build/lib/ptypy/core/ptycho.py", line 593, in init_engine
    self.init_engine()
    self.init_engine()
    self.init_engine()
  File "/home/litang/ptypy_p06/build/lib/ptypy/core/ptycho.py", line 593, in init_engine
  File "/home/litang/ptypy_p06/build/lib/ptypy/core/ptycho.py", line 593, in init_engine
  File "/home/litang/ptypy_p06/build/lib/ptypy/core/ptycho.py", line 593, in init_engine
    self.init_engine(label)
  File "/home/litang/ptypy_p06/build/lib/ptypy/core/ptycho.py", line 578, in init_engine
    self.init_engine(label)
  File "/home/litang/ptypy_p06/build/lib/ptypy/core/ptycho.py", line 578, in init_engine
    self.init_engine(label)
    engine = engine_class(self, pars)
    self.init_engine(label)
  File "/home/litang/ptypy_p06/build/lib/ptypy/core/ptycho.py", line 578, in init_engine
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/litang/ptypy_p06/build/lib/ptypy/accelerate/cuda_pycuda/engines/stochastic.py", line 507, in __init__
  File "/home/litang/ptypy_p06/build/lib/ptypy/core/ptycho.py", line 578, in init_engine
    engine = engine_class(self, pars)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/litang/ptypy_p06/build/lib/ptypy/accelerate/cuda_pycuda/engines/stochastic.py", line 507, in __init__
    engine = engine_class(self, pars)
    engine = engine_class(self, pars)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/litang/ptypy_p06/build/lib/ptypy/accelerate/cuda_pycuda/engines/stochastic.py", line 507, in __init__
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/litang/ptypy_p06/build/lib/ptypy/accelerate/cuda_pycuda/engines/stochastic.py", line 507, in __init__
    _StochasticEnginePycuda.__init__(self, ptycho_parent, pars)
    _StochasticEnginePycuda.__init__(self, ptycho_parent, pars)
    _StochasticEnginePycuda.__init__(self, ptycho_parent, pars)
  File "/home/litang/ptypy_p06/build/lib/ptypy/accelerate/cuda_pycuda/engines/stochastic.py", line 62, in __init__
  File "/home/litang/ptypy_p06/build/lib/ptypy/accelerate/cuda_pycuda/engines/stochastic.py", line 62, in __init__
    _StochasticEnginePycuda.__init__(self, ptycho_parent, pars)
  File "/home/litang/ptypy_p06/build/lib/ptypy/accelerate/cuda_pycuda/engines/stochastic.py", line 62, in __init__
  File "/home/litang/ptypy_p06/build/lib/ptypy/accelerate/cuda_pycuda/engines/stochastic.py", line 62, in __init__
    super().__init__(ptycho_parent, pars)
  File "/home/litang/ptypy_p06/build/lib/ptypy/accelerate/base/engines/stochastic.py", line 51, in __init__
    super().__init__(ptycho_parent, pars)
  File "/home/litang/ptypy_p06/build/lib/ptypy/accelerate/base/engines/stochastic.py", line 51, in __init__
    super().__init__(ptycho_parent, pars)
    super().__init__(ptycho_parent, pars)
  File "/home/litang/ptypy_p06/build/lib/ptypy/accelerate/base/engines/stochastic.py", line 51, in __init__
  File "/home/litang/ptypy_p06/build/lib/ptypy/accelerate/base/engines/stochastic.py", line 51, in __init__
    super().__init__(ptycho_parent, pars)
  File "/home/litang/ptypy_p06/build/lib/ptypy/engines/stochastic.py", line 53, in __init__
    super().__init__(ptycho_parent, pars)
  File "/home/litang/ptypy_p06/build/lib/ptypy/engines/stochastic.py", line 53, in __init__
    super().__init__(ptycho_parent, pars)
  File "/home/litang/ptypy_p06/build/lib/ptypy/engines/stochastic.py", line 53, in __init__
    super().__init__(ptycho_parent, pars)
  File "/home/litang/ptypy_p06/build/lib/ptypy/engines/stochastic.py", line 53, in __init__
    raise NotImplementedError("The stochastic engines are not compatible with MPI")
    raise NotImplementedError("The stochastic engines are not compatible with MPI")
NotImplementedError: The stochastic engines are not compatible with MPI
    raise NotImplementedError("The stochastic engines are not compatible with MPI")
NotImplementedError: The stochastic engines are not compatible with MPI
    raise NotImplementedError("The stochastic engines are not compatible with MPI")
NotImplementedError: The stochastic engines are not compatible with MPI
NotImplementedError: The stochastic engines are not compatible with MPI
bjoernenders commented 9 months ago

Hi, our stochastic engines are not compatible with MPI. You can however split the dataset and have EPIE work on a subset per MPI rank. Andrew Maiden talked about this scheme at the Ptycho Workshop last year. But this is ultimately a different type of engine and doesn't necessarily have the same convergence behavior. I don't recall how Andrew called these engines, maybe it's WASP? https://preprints.opticaopen.org/articles/preprint/WASP_Weighted_Average_of_Sequential_Projections_for_ptychographic_phase_retrieval/24894489 Timothy is working on a pull request for that engine. https://github.com/ptycho/ptypy/pull/522 Maybe @daurer can comment better on that.

ltang320 commented 9 months ago

Hi, our stochastic engines are not compatible with MPI. You can however split the dataset and have EPIE work on a subset per MPI rank. Andrew Maiden talked about this scheme at the Ptycho Workshop last year. But this is ultimately a different type of engine and doesn't necessarily have the same convergence behavior. I don't recall how Andrew called these engines, maybe it's WASP? https://preprints.opticaopen.org/articles/preprint/WASP_Weighted_Average_of_Sequential_Projections_for_ptychographic_phase_retrieval/24894489 Timothy is working on a pull request for that engine. #522 Maybe @daurer can comment better on that.

Oh, I see your point. @ptim0626 Hi Timothy, I see you add the WASP reconstruction engine. Can I have a test of it? By the way, can it be used with multi-modes? Thanks for all your help!

ptim0626 commented 9 months ago

Hi @ltang320, of course you are welcome to try the WASP engine! It supports multi probe and object modes.

I should add the WASP engine codes in #522 is currently under active development and you may find errors and other unexpected behaviour, so please use if with caution and if you encounter any issue, please let us know.