precice / openfoam-adapter

OpenFOAM-preCICE adapter
https://precice.org/adapter-openfoam-overview.html
GNU General Public License v3.0
134 stars 77 forks source link

Re arrange read time according to adjustable dt #298

Closed davidscn closed 7 months ago

davidscn commented 11 months ago

Only the last commit is relevant

Since read data is now affiliated to a time stamp, we need to ensure that the time stamp we read data is consistent with our solver time-step size. However, the solver time-step size changes in different places, depending on the configuration in the controlDict.

functionObject::execute{ writeData() advance(dtN) [readCheckpoint] [writeCheckpoint] dtN1 = adjustDt() readData(dtN1) }


- If we have an adjustable time-step size in our OpenFOAM solver, the time-step size adjustment is triggered in the `adjustTimeStep` routine of the functionObject, which is triggered in the beginning of each time-step. This results in the order:

functionObject::execute{ writeData() advance(dtN) [readCheckpoint] [writeCheckpoint] }

// at the beginning of each (also the first) time step functionObject::adjustTimeStep{ dtN1 = adjustDt() readData(dtN1)



The described data reading is consistent with a backward Euler scheme (or just `ddt Euler` in OpenFOAM terminology) and not with all the time-integration schemes OpenFOAM offers.

This PR ensures the readData logic according to the described scenarios. 

The only thing (feature-wise), which still does _not_ work with the current workflow is the `writeControl    adjustable;` option and only the option `writeControl    runTime;` works as expected. This option only affects the generation of result files, though. However, I think this was already the case before we introduced the changes here, correct @MakisH ? I'm still a bit confused, where and how the first option modifies the time-step size (triggered actually https://github.com/precice/precice/issues/1751 ), but I guess we cannot do a whole lot about it. 

_DB driven development_

TODO list:

- [ ] I updated the documentation in `docs/`
- [x] I added a changelog entry in `changelog-entries/` (create directory if missing)
MakisH commented 11 months ago

The only thing (feature-wise), which still does not work with the current workflow is the writeControl adjustable; option and only the option writeControl runTime; works as expected. This option only affects the generation of result files, though. However, I think this was already the case before we introduced the changes here, correct @MakisH ? I'm still a bit confused, where and how the first option modifies the time-step size (triggered actually precice/precice#1751 ), but I guess we cannot do a whole lot about it.

Here is the respective OpenFOAM code:

void Foam::Time::adjustDeltaT()
 {
     bool adjustTime = false;
     scalar timeToNextWrite = VGREAT;

     if (writeControl_ == wcAdjustableRunTime)
     {
         adjustTime = true;
         timeToNextWrite = max
         (
             0.0,
             (writeTimeIndex_ + 1)*writeInterval_ - (value() - startTime_)
         );
     }

     if (adjustTime)
     {
         scalar nSteps = timeToNextWrite/deltaT_;

         // For tiny deltaT the label can overflow!
         if (nSteps < scalar(labelMax))
         {
             // nSteps can be < 1 so make sure at least 1
             label nStepsToNextWrite = max(1, round(nSteps));

             scalar newDeltaT = timeToNextWrite/nStepsToNextWrite;

             // Control the increase of the time step to within a factor of 2
             // and the decrease within a factor of 5.
             if (newDeltaT >= deltaT_)
             {
                 deltaT_ = min(newDeltaT, 2.0*deltaT_);
             }
             else
             {
                 deltaT_ = max(newDeltaT, 0.2*deltaT_);
             }
         }
     }

     functionObjects_.adjustTimeStep();
 }

writeControl adjustable sets the timeToNextWrite, which then affects deltaT_. deltaT_ is a member variable of the TimeState class. This is used to compute the time step size in the next iteration:

if (adjustTime)
     {
         scalar nSteps = timeToNextWrite/deltaT_;

I think that you could test the behavior by specifying a fixed deltaT that is not perfectly dividing writeInterval (e.g., 0.3 and 1.0). You should end up with the directories 0/ 0.3/ 0.6/ 0.9/ 1.0/.

preCICE does something similar, so the two mechanisms compete. I don't find any particular documentation (code/thesis) regarding writeControl in the adapter. While I remember being puzzled with all the different possibilities, I don't remember this particular aspect being an unsolved problem (maybe I faced an issue that I later fixed). In case there are issues with that, we should better check and throw an error, as this is rather a common setting.

I am struggling a bit to evaluate all paths, but I think the changes make sense, especially since now the read_data() is coupled to the dt.

Questions:

Are you sure that the dt we are determining everywhere is the one matching the read_data()? I am always a bit confused with what is used now, what is used next, and what preCICE needs.

How can we test that the changes still give the same (or more correct) results? What have you checked and what is still open?

Other than these questions, I think that your contribution is on the right rails.

davidscn commented 11 months ago

Main reason why the writeControl setting is problematic now is that quite some assertions were added during waveform implementations (https://github.com/precice/precice/issues/1751). Maybe it would be a good idea to wait until the mentioned issue is resolved, because the assertion messages I get from preCICE don't help to figure out apparent issues. I'm also wondering whether all the assertions in preCICE consider rounding errors and were tested using real solvers. In principle, running everything using release builds works without any error message from preCICE.

MakisH commented 11 months ago

So, status: Looks good, but waiting for #285 and #297 to be merged (and then rebase this branch) and for https://github.com/precice/precice/issues/1751 to be resolved in the library. We will potentially need to document any restrictions regarding writeControl.

MakisH commented 11 months ago

Merged all dependencies into develop and merged develop back to this branch. Also added a changelog entry.

Next step: waiting for https://github.com/precice/precice/issues/1751 to be resolved in the library. We will potentially need to document any restrictions regarding writeControl.

MakisH commented 7 months ago

Update: I triggered https://github.com/precice/precice/issues/1890 while testing the adapter from develop. However, even when using this branch, I run into the same issue when subcycling. We should wait for that to be fixed.

See also https://github.com/precice/tutorials/pull/406, which uses subcycling.