Parallelization of post-propagation

[jam31118]

When the laser is off, each equations for each basis are decoupled. The calculation for these equations may easily distributed to multiple processes. Let's see the inside of wavefunction.propagate() method and figure out how to implement parallelization.

TODO

• [ ] Do all the following tasks denoted by check-boxes

[jam31118]

Although the t-SURFF method is quite efficient algorithm to obtain photoelectron momentum distribution, it sometimes requires long post-propagation to get low-momentum part of the distribution. Often, the post-propagation time is comparable or bigger than the pulse duration, which makes even WINOP method become feasible.

However, post-propagation can take advantage of the decoupled equations, which is highly parallizable compared to coupled case. This can enhance the computing time a lot, especially for the case where the post-propagation time is comparable to pulse duration.

Thus, following routine is suggested:

propagation during pulse duration - with high-speed single core post-propagation - with multiple cores

[jam31118]

Idea

Consider implementing this post-propagation routine as a separate function, because:

• It seems feasible, since the propagation with spherical harmonics without the external field seems really
• Pedagogic for me. It seems to be a good practice.
• Easy to implement parallelization
  • Since it is a separate routine, it is easy to manage each code
  • Easy to call this routine after the external field, with negligible overhead.

TO DO

• [ ] The result of this subroutine should be compared to the original routines
• [ ] See the formula and/or the corresponding code internal (namely, wavefunction.propagate()) of the qprop with external field. Derive the decoupled Schrodinger equations from the external-field case's coupled equations with the external field's strength set to zero. It this derivation succeeded, then the coincidence between this new subroutine and the original one become promised.
• [ ] Use python to quickly implement this routine. Also consider making this routine as an extension. Both the CPU and GPU version and even with MPI seems possible with almost similar performance.
• [ ] Implement CPU-MPI version, assigning each 1D TDSE into each MPI process.
• [ ] Implement GPU version.
• [ ] Implement multiple GPU version.

[jam31118]

Implementing this routine in python with some C (possibly and hopefully also with CUDA) extension may be a one option. Let's check the calculation time for single core for both qprop (thus, pure C++) and this python routine. Compare this single core times with the CPU-parallelized (and possibly also with GPU) routine.

The Python seems actually be nice considering that the rigged-qprop has python wrapper already so that this can easily be the extension of the wrapper and more importantly, user can use it within one programming langauge, python, without consulting for other language.

[jam31118]

• [ ] May be I should modify the end point of the Hamiltonian matrix! It is for matching the boundary condition of the state function at the origin. See Bauer et al, 2017 for further details.
  • Note that this correction delta is for Numerov-boosted propagation. Let's first implement this Numerov boost and then introduce this correction delta.

[jam31118]

TODO

• [ ] [IMPORTANT] The tsurff data files such as:

  • [ ] tsurffpsi.raw
  • [ ] tsurff-dpsidr.raw

• [ ] Update also other required files so that after this post-propagation, all the data files are like just as a normal, pure-qprop propagation.

  • [ ] current-wf.bin
  • [ ] real-prop-obser.dat
  • [ ] real-prop-vecpot.dat
  • [ ] real-prop-wf.dat
  • [ ] real-prop-yield.dat

• [ ] Considering updating even the log files that have been generated such as:

  • [ ] real-prop.log
  • [ ] re.log
  • [ ] (What else?)

[jam31118]

Development Road-map

• [ ] Implement single-core but with working calculation. Also analyze the result and confirm whether the results from PPP(#1) are exactly same as rigged-qprop.

  • [ ] Save the value of imagpot in binary and load it from the python routine so that exactly the same values can be used.

• [ ] Implement MPI-enabled one.

  • [ ] Use mpi4py

• [ ] Boost by C-extension

  • [ ] Implement matrix-vector multiplication and Gaussian elimination for complex-valued vectors in tdse module.
  • [ ] Start from applying this extension to single process, not MPI.
  • [ ] Apply this C-extended routine in MPI.

[jam31118]

Implemented the first working PPP at 4644c99731dc0fe86c82750edc5500f89bbcfe26