Interface to scattering simulations

[bussmann]

How will the interfaces to scattering simulations look like? Specifically, where does the Simex Platform take over from the scattering simulation and vice versa?

[CFGrote]

Hi Michael, thanks for your questions, I'll give a long answer here and refer back in the other issues. The Simex platform is (will be) a modular framework that describes slots and interfaces for the various parts of a simulated experiment: Photon source, focussing optics, photon-matter interaction, scattering / diffraction, and detection. By itself, it does not perform any calculations in any of the above mentioned steps but relies on special "Calculators" to do so. These have to be provided by us and / or by the later users. The platform itself provides abstract classes that define a number of virtual methods for, e.g. IO and for invoking the actual calculation. The specialized calculators derive from these abstracts, have to implement the virtual methods and thus fit into the framework. Details, such as coherent summation of phases etc. are left to the implementation of the special calculator. The virtual methods are called sequentially on each calculator in a "master calculator" to realize the simulation pipeline.

To answer your question: Suppose you have a piece of software that generates the scattering amplitudes from some target. In simex lingo, this is referred to as a backengine. To integrate it into the platform, you would then write a python class that derives from the respective base class (AbstractPhotonMatterInteractor), implement the required methods and hand it over to the master calculator. It is then integrated into the platform in the sense that it can take the incoming photon field as input, write out the scattering amplitudes as output, which will be taken as input by the next calculator in line.

All this being said, I'm building the platform on the basis of the simS2E toolbox for coherent diffractive single particle imaging and the design of the interfaces follows the simS2E paradigm relatively closely. E.g. IO is entirely based on hdf5 files (on disk). This is by no means meant as an end solution, rather as a first iteration from the special simS2E toolchain to something more general. My idea is that as we go along and integrate more and more of these backengines, we can identifiy common patterns, make them part of the abstract base classes and leave the tricky details that are special to each tool to the specializations. But I'm very open to any alternative suggestions.

I'm quite near to a stage where I can demonstrate the platform and the specialized calculators for CDI which will hopefully answer many of your questions.

Thanks, Carsten

[bussmann]

I like this approach very much as it allows for clean interfacing and the Python integration makes it easy to use.

I am however a bit worried that a few choices in design that seem very sensible will make it later quite hard to do some interesting things with it.

One of my main concerns is data transport between the various pieces of the chain. File I/O severly limits the performance, although it makes interfacing very easy and flexible. We should not forget the possibiliy of in-memory data transfer, especiall if we think that in the end we want (and need) to propagte photons through the simulated beamline rather than fields.

[ax3l]

Just for completeness: a performing in-memory way for large data streaming between applications can be done via ADIOS staging transports via the methods FlexPath or DataSpaces. (Doesn't change the representation we use with openPMD.)