Auxiliary methods for integrators

[szaghi]

Dear all,

some of you (@jacobwilliams and @zbeekman) have expressed the idea that the FOODiE's integrators should provide some auxiliary methods:

It would be wonderful to have an object oriented library of time integrators that could be used, for integration of abstract data type (ADT) calculus (See Rouson et al.) with procedures to query their properties, like their stability locus plots sigma roots, etc. Users could then be guided to picking a good time integrator for their PDE/ODE of interest and it would take a lot of the headache out of picking a suitable time integration scheme to match your spatial scheme when making a PDE solver.

I am now quite confident with the ADT of Rouson (Euler and explicit TVD RK schemes have been implemented) thus, in the meanwhile other schemes are under development, I would like to implement some of the auxiliary methods that you asked for. Here is list tracking issue:

• [ ] stability locus plot;
• [ ] event finding... @jacobwilliams what do you mean?
• [x] supported stages/steps inquiring methods, e.g. tvd_rk%is_supported(s=7);
  • [x] lower and upper bound limits of supported stages/steps, e.g. tvd_rk%max_stages();
• [ ] provide a descriptive summary (varying string?) of the solver or of the class of solvers, e.g. tvd_rk%description();
• [ ] ...

[rouson]

Today is my last day of vacation and my plate will be full when I return to work tomorrow so my input will be sparse, but this is an exciting project so I hope you'll tolerate the occasional random stream of consciousness.

First, since you're using git, I recommend studying the following Git Workflows documentation if you haven't already: https://www.atlassian.com/git/tutorials/comparing-workflows/. It takes a second or even third reading to digest and some experience to fully appreciate. But I'm really glad that @zbeekman pointed me in this direction.

Second, consider adopting a Test-Driven Development approach, including unit testing at the project outset. TDD pays dividends. Tested code is trusted code. If you're willing to tackle building with CMake and test with CTest, then you might find the building and testing set-up for OpenCoarrays (https://github.com/sourceryinstitute/opencoarrays) instructive. I wrote the CMake scripts and would be glad to explain anything that is there. Or you might check out pFUnit for more sophisticated testing tailored to Fortran (http://sourceforge.net/projects/pfunit/).

Third, consider including performance tests in your testing. This is especially important when using the Abstract Calculus pattern as mentioned elsewhere. Users will want to know how it compares to other approaches. Automating the performance analysis by including it in the test suite will make performance analysis a natural part of your workflow. We're not yet doing this with OpenCoarrays, but I've did have performance tests in one other project I worked on so I might be able to provide some advice on this topic too.

Fourth, document as you write. I find there never comes a day when I wake up and say, "This feels like a great day to document that code I wrote a while back."

I hope this is helpful.

[szaghi]

@rouson

Today is my last day of vacation and my plate will be full when I return to work tomorrow so my input will be sparse, but this is an exciting project so I hope you'll tolerate the occasional random stream of consciousness.

I am very happy and onorate to learn the random stream of consciousness coming from you and from all members of the group. All your opinions are very appreciated also in any random forms :-) Thank you all for your help.

First, since you're using git, I recommend studying the following Git Workflows documentation if you haven't already: https://www.atlassian.com/git/tutorials/comparing-workflows/. It takes a second or even third reading to digest and some experience to fully appreciate. But I'm really glad that @zbeekman pointed me in this direction.

Zaak is a wiseman... I never read this tutorial, I will try to digest it soon. Anyhow, I use gitflow that, for a poor Fortran man like me, is a great help for handling git repos in a more sane way.

Second, consider adopting a Test-Driven Development approach, including unit testing at the project outset. TDD pays dividends. Tested code is trusted code. If you're willing to tackle building with CMake and test with CTest, then you might find the building and testing set-up for OpenCoarrays (https://github.com/sourceryinstitute/opencoarrays) instructive. I wrote the CMake scripts and would be glad to explain anything that is there. Or you might check out pFUnit for more sophisticated testing tailored to Fortran (http://sourceforge.net/projects/pfunit/).

You are absolutely right. I started very soon a discussion about the tests facility but my approach is not so formal and not accurately designed as your. I have to learn more about a formal approach for TDD, presently there is the Lorenz test that you use in your great book, but soon they will be also Burgers test and a 1D Riemann solver tester for the compressible Euler conservation laws. Moreover, I used test unit tools in python, but never in fortran, maybe this is a nice occasion to use the pfunit wrapper of @acroucher.

As cmake is concerned, I disagree: for pure Fortran project like FOODiE is designed to be, cmake is not so friendly. I prefer FoBiS :-) However, anyone being an experienced cmake addicted can add support for cmake, I hope that all of you feel free to join FOODiE project.

Third, consider including performance tests in your testing. This is especially important when using the Abstract Calculus pattern as mentioned elsewhere. Users will want to know how it compares to other approaches. Automating the performance analysis by including it in the test suite will make performance analysis a natural part of your workflow. We're not yet doing this with OpenCoarrays, but I've did have performance tests in one other project I worked on so I might be able to provide some advice on this topic too.

I absolutely agree and all the hints that you can give me will be very appreciated. Regarding this topic I hope to obtain the help of @muellermichel and @francescosalvadore (among all the others) that are two fortraners that (like you) I know to have great experience on HPC.

Fourth, document as you write. I find there never comes a day when I wake up and say, "This feels like a great day to document that code I wrote a while back."

I agree. I documented all the sources that I produced up to now, see full API, there are also the bibliography and maths, all produced by means of FORD a great tool of @cmacmackin. However, I will start very soon the wiki for providing a more comprhensive documentation (with tutorials, examples, etc...).

Thank you very much for your help, feel free to write your idea in any forms and frequency :-)

[zbeekman]

Zaak is a wiseman... I never read this tutorial, I will try to digest it soon. Anyhow, I use gitflow that, for a poor Fortran man like me, is a great help for handling git repos in a more sane way.

Fake it, until you make it! That’s my motto!

All joking aside, I am not a fan of git flow. IMO, it takes a flexible, distributed, modern tool, Git, and tries to impose bad practices necessitated by antiquated SCM systems (ahem, Subversion) and pre-CI workflows. I much prefer GitHub Flow which is pretty much the same as Atlasssian’s forking. Here is some information on how this differs from Git Flow.

In the case of a centralized organization’s repository, GitHub Flow works, but isn’t terribly different than the feature branch workflow in practice, as long as people don’t prematurely merge feature branches.

[szaghi]

@zbeekman I have not yet read all the resources you linked, thus I could be wrong, but I think that the gitflow I am using (the Vincent Driessen's branching model) does not impose a centralized-like workflow. The main attractive features of gitflow for me were (as atlassian says in one of your references):

(about gitflow...) This workflow doesn’t add any new concepts or commands beyond what’s required for the Feature Branch Workflow. Instead, it assigns very specific roles to different branches and defines how and when they should interact. In addition to feature branches, it uses individual branches for preparing, maintaining, and recording releases. Of course, you also get to leverage all the benefits of the Feature Branch Workflow: pull requests, isolated experiments, and more efficient collaboration.

For a poor OCD mind like mine, this workflow imposes the devoloper to be clear and polite while preserving all the goodness of a flexible, distributed, modern tool like Git globally. The continuos development workflow of github is surely more light, but less clean. IMO, we can adopt our own workflow, to be defined... I am a new issue :-)

[zbeekman]

@szaghi I’m not suggesting we should move away from gitflow, just expressing my preference :-)

[szaghi]

@zbeekman your preference is important for me, see this :-)

[rouson]

I probably should have made it clear that the specific workflow I've evolved towards is the Forking Workflow (not the GitFlow Workflow). The number one reason is that I want to be careful about what makes it into the central repository. Without forking, branches will be recorded in the central repository unless they are never pushed. With forking, I have to option to keep my branches private if I push them to a private repository. Currently, my fork of OpenCoarrays (which has occupied most of my development time for the past several months) is public (on GitHub), but I like having the option to take them private (which I would probably do by putting it on BitBucket).

One reason for wanting the option for privacy is that I am hopeful that OpenCoarrays will be widely used. Widely used means widely watched. I wouldn't want users to make development decisions based on some half-baked feature branch that I started working on but later abandoned. Hopefully no one would do that, but it just seems safer to prevent it.

Also, I like the idea of users forking their own repositories for developing their own feature branches before submitting a pull request to the maintainer of the central repository. I think this is a great way to contribute without having commit rights to the central repository. In fact, one user has already done this so I'm pretty excited. I'll be reviewing his pull request today.

[szaghi]

@rouson I see that you promote forking like @zbeekman

The number one reason is that I want to be careful about what makes it into the central repository. Without forking, branches will be recorded in the central repository unless they are never pushed.

I use gitflow locally: all the branches I created locally are not pushed to GitHub repository, only the master one is uploaded thus for this specific topic forking and gitflow produce the same result. Into the FOODiE repository there is only gh and master branches not all the dirty ones I created locally.

With forking, I have to option to keep my branches private if I push them to a private repository.

The same is true for gitflow: if I like I can push all my dirty branches to a remote private repository; git flow is an helper for branch managing it does not prevent to use all the goodness of git.

One reason for wanting the option for privacy is that I am hopeful that OpenCoarrays will be widely used. Widely used means widely watched. I wouldn't want users to make development decisions based on some half-baked feature branch that I started working on but later abandoned. Hopefully no one would do that, but it just seems safer to prevent it.

I completely agree, but in my case I almost alone :-( However, I like to take care of future not so lucky poor Fortran people that will try to use my codes... thus your suggestion is absolutely great!

Also, I like the idea of users forking their own repositories for developing their own feature branches before submitting a pull request to the maintainer of the central repository. I think this is a great way to contribute without having commit rights to the central repository. In fact, one user has already done this so I'm pretty excited. I'll be reviewing his pull request today.

As I said here, I think that gitflow and forking are not in contrast. I hope that someone forks very soon FOODiE and correct my terrific mistakes :-)

[szaghi]

@rouson I have added some documentation into the wiki, in particular:

• motivations
• solution

I would like to cite your book, can I cite it? if yes, how?

[rouson]

For citations, I usually use the "Cite" link for the relevant publication on Google Scholar (scholar.google.com). It gives options for various formats, including a BibTeX format in case that's of interest. The MLA format it provides for my book is

Rouson, Damian, Jim Xia, and Xiaofeng Xu. Scientific software design: the object-oriented way. Cambridge University Press, 2011.