Implicit coupling?

[transp]

Is there any support for implicit coupling of components in IPS? I.e., the ability to iterate (Newton or Picard) over a time step until two coupled components agree on the final state.

My immediate need is to make a TRANSP simulation including GENRAY and CQL3D numerically stable. The driven LH current in CQL3D oscillates wildly from one time step to next and reducing the coupling time step only delays the onset.

Longer term, we are moving towards looser coupling in TRANSP and are currently considering the best options for achieving that.

Thanks, Johan

[elwasif]

Well, it depends on the definition of the word "support" :-)

Long time ago, Lee Berry and Fred Jaeger implemented a Picard scheme (AFAIR between CQL3d and TORIC). @dlg0 may know more about that. That effort didn't actually use the IPS, but the IPS DOES offer support for such an implementation.

There are actually more than one way this can be done in the IPS. One scenario leverages the IPS asynchronous event service. In such a scenario the two participating components will be launched concurrently. The components (i.e. Python wrappers) will go into an iterative loop, launching the underlying codes and exchanging relevant state data that control how the next invocation of the "other" code will run. Determining convergence can be done where it makes the most sense.

Another alternative would be to have a more "traditional" time stepped driver that goes into an "inner" iterative loop calling the step() method of the two components until they converge, before moving on to other codes in the coupled simulation (if they exist).

So this is doable (i's just a matter of coding). The real question then becomes how efficient the whole scheme will be with binaries starting and stopping and file-based data exchange. If the run time for each invocation is long enough, then that overhead may not be an issue.

[bernhold]

I think there's a paper describing Lee and Fred's work.

I also think that Ed D'Azevedo was involved in the original work and may be someone we can tap into if needed to implement it now.

On 3/23/2016 12:15 PM, Wael Elwasif wrote:

Well, it depends on the definition of the word "support" :-)

Long time ago, Lee Berry and Fred Jaeger implemented a Picard scheme (AFAIR between CQL3d and TORIC). @dlg0 https://github.com/dlg0 may know more about that. That effort didn't actually use the IPS, but the IPS DOES offer support for such an implementation.

There are actually more than one way this can be done in the IPS. One scenario beverages the IPS asynchronous event service. In such a scenario the two participating components will be launched concurrently. The components (e.g. Python wrappers) will go into an iterative loop, launching the underlying codes and exchanging relevant state data that control how the next invocation of the "other" code will run. Determining convergence can be done where it makes the most sense.

Another alternative would be to have a more "traditional" time stepped driver that goes into an "inner" iterative loop calling the step() method of the two components until they converge, before moving on to other codes in the coupled simulation (if they exist).

So this is doable (i's just a matter of coding). The real question then becomes how efficient the whole scheme will be with binaries starting and stopping and file-based data exchange. If the run time for each invocation is long enough, then that overhead may not be an issue.

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/ORNL-Fusion/ips-atom/issues/42#issuecomment-200418727

[transp]

Thanks, that's good news! I got a bit worried when I didn't see an obvious way to undo a time step for each individual component, but that's probably not necessary for implicit coupling then?

Since @elwasif brought it up, are there any plans for memory -based data exchange?

And thanks for the PlasmaState Python interface, I will try it out for TRANSP! I've been using the netCDF4 module to load PS CDF files into Python, but it has its limitations.

[elwasif]

Well, the memory based data exchange would violate one of the basic and early design decisions of the IPS, to not make changes to the individual codes. That, and having a "general" in-memory coupling solution (especially with one or both sides running in parallel) is really hard, so we've decided to punt on that. Having said that, if codes have an out-of-band in-memory data coupling solution (e.g. using ADIOS or some other mechanism), then the IPS will not stand in their way.

[bernhold]

I would put this differently.

We want the IPS to be able to support components of all kinds. At the beginning, we needed to be able to support encapsulating codes non-intrusively. But we never intended that that be limiting. In fact, Wael may remember that we expected to have to do memory-based coupling before the end of the SWIM project. Its just that the need never arose in the end.

If there is a use case today that would motivate it, we (Wael too, whether he likes it or not) would be happy to talk to you in greater detail. I can imagine a wide range of requirements, each of which could lead to qualitatively different solutions. ADIOS offers one possibility, but I wouldn't want to preclude any paths before we talk about your needs.

On 3/23/2016 2:22 PM, Wael Elwasif wrote:

Well, the memory based data exchange would violate one of the basic and early design decisions of the IPS, to not make changes to the individual codes. That, and having a "general" in-memory coupling solution (especially with one or both sides running in parallel) is really hard, so we've decided to punt on that. Having said that, if codes have an out-of-band in-memory data coupling solution (e.g. using ADIOS or some other mechanism), then the IPS will not stand in their way.

— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/ORNL-Fusion/ips-atom/issues/42#issuecomment-200478436

[transp]

@elwasif and @bernhold, thanks!

I will spend some time familiarizing myself with IPS and tap into the institutional experience with it here at PPPL. I'll get back to you.

[batchelordb]

I would say that anything you can program can be done with IPS. The questions is, as Wael said, are the chunks of work big enough that it makes sense to do it in IPS, i.e. are they big parallel codes?.

DBB

On Mar 23, 2016, at 1:55 PM, Johan Carlsson notifications@github.com wrote:

Thanks, that's good news! I got a bit worried when I didn't see an obvious way to undo a time step for each individual component, but that's probably not necessary for implicit coupling then?

Since @elwasif https://github.com/elwasif brought it up, are there any plans for memory -based data exchange?

And thanks for the PlasmaState Python interface, I will try it out for TRANSP! I've been using the netCDF4 module to load PS CDF files into Python, but it has its limitations.

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/ORNL-Fusion/ips-atom/issues/42#issuecomment-200466091