Announcement Paper - Githubissues

szaghi commented 9 years ago

As anticipated on gitter, I am planning to write an announcement paper to be published on CPC or similar.

I have just uploaded a skeleton tex with the list of the authors that (for the moment) I would like to include into the paper. No matter the order of names, we can modify it in any order.

I hope others will join us.

See you soon.

sourceryinstitute commented 9 years ago

Thanks for including me in the author list. I probably haven't contributed enough to be considered an author so far. If you let me know your timeline for submitting it, I'll do my best to review the code and the associated discussions to see if I can provide input in a way that be sufficient to be a co-author.

szaghi commented 9 years ago

@sourceryinstitute Well, I do not think so. Your contribution is firstly in your book that inspired us: FOODiE is essentially the concretization of your idea. Secondly, in my opinion, it is not necessary to actively write codes to be considered a co-author: just the contributions to the interesting discussions here is enough, they drive the code development, they teach many new aspects...

I have not planned a precise timeline: my intention is to complete the accurate analysis of the tests results and perform performance profiling. If both will pass the analysis I think that the work will be worth to be published. In this view, considering that I am doing this in my breaks, the time necessary can be measured in month(s).

See you soon.

sourceryinstitute commented 9 years ago

Thanks! This publication puts this on my radar and I'll hope to find time soon to review everything. Just let me know when you get close to submitting. When the time comes, I can probably get through the code fast enough to feel comfortable being a co-author.

Classes start at Stanford on Monday and I'm scheduled to teach a class based on my book so life is a little hectic right now.

milancurcic commented 9 years ago

Thank you @szaghi for pushing this forward!

szaghi commented 9 years ago

Thank you all for your great help, I am learning a lot!

szaghi commented 9 years ago

I have done some steps on with analysis of the oscillation test (besides I am implementing a scripts-based system thus you can reproduce my analysis without efforts).

Almost all seems good as expected, only the TVD/SSP Runge-Kutta class is less accurated than I supposed: with different tests TVD/SSP seems accurate as the low storage counterpart, wheras for the oscillation test, comparing the two 5 stages formal 4th order schemes, the low storage performs better. Maybe this test could highlight some cons of imposing the observance of the TVD or SSP proprierties that limits the overall accuracy: the low storage version does not care of TVD/SSP proprierty, thus it could be more accurate in this smooth test.

Tomorrow I will give you more details.

See you soon.

szaghi commented 9 years ago

I have modified again the Oscillation test: now the errors analysis is embedded into the Fortran source. I was afraid about the previous python script precision of errors computation.

I have added some instructions for you for reproducing the errors analysis (currently working for only Oscillation test).

Besides, I am going a little forward with the manuscript, see the progress here.

See you soon.

szaghi commented 9 years ago

Dear all,

I have just pushed some improvements on the manuscript:

add some "mass" to the general description;
add short introduction to all ODE solvers implemented;
add some (most all) API descriptions;
add the subsections skeleton for the performance section;

When you start reading this draft, please forgive me... my English is an approximation of order 0... moreover I written furiously the draft in my lunch break, I am sorry. Please, focus on the contents rather than on the format.

Very soon I will complete the API discussion and the tests description (at least for the Oscillation and 1D Euler). What is still missing is the benchmark in the parallel frameworks (I have planned OpenMP and MPI tests, but other can be added in case).

See you soon.

P.S. I have added a 4 steps, formally 4th order Adams-Bashforth solver.

szaghi commented 9 years ago

Hi all,

I have completed the API review and discussion. I have also improved the Oscillation test subsection. Giving a more careful view on the Adams-Bashforth results for this test, I am wondering if the results are close to the expected ones. I have reported some of mine concerns into its subsection in red font.

See you soon.

P.S. I have eliminated the dependency of FOODiE main library from IR_Precision.

szaghi commented 9 years ago

Hi all,

very good news! I found a bug into the multi-step solvers (both AB and leapfrog): now all solvers show an observed order of accuracy in agreement with the expected formal orders (at least for the oscillation test).

For resolving the bug I have changed the API: now the previous time steps memory is more sanely handled (as it already happens for the RK stages) and the multi-step solvers automatically take into account for their correct cyclic update. This alleviates client codes from many odds and make the integrand ADT more concise and clear.

Minor updates are also present into the manuscript.

See you soon.

szaghi commented 8 years ago

@rouson @milancurcic @zbeekman

Dear All,

I have booked a Damian's time-slot for tomorrow, November 12th at 5:30 PM (or more clear format for me 17:30) Rome time (I will not try to convert to your local time :-). I booked 1 hour slot because due to my bad English I guess that some minutes will be lost in translation, but I hope we can finish before 6:30 PM. I think that for Milan and Zaak this slot should be good (more or less 6 hours before Rome time I think for Milan and Zaak), but if I am wrong, please let Damian free his time as soon as possible.

To facilitate our discussion, I am going to list here some topics: feel free to add/modify/delete any topics.

Unfortunately, I cannot use phone (to costly), thus I prefer a skype or hangout conference call. My gmail account is stefano.zaghi@gmail.com, while my skype username is stefano.zaghi (I have not fantasy for username...).

Thank you all for dedicating your time to FOODIE.

szaghi commented 8 years ago

conference call main topics

These are my principal issue about the manuscript:

arguments touched: I have made some efforts to clarify the aims of FOODIE, but the result is very poor; I think that the manuscript introduction/background/aims must be strongly improved. I am doing a bibliographic search for similar projects that should be cited. What is probably missing is a clear statement of intended audience: the arguments touched seem to be not uniform enough;
length: I have used the manuscript principally as a "blog" for making you aware of the implementation progress, thus is now very lengthy and boring to read. Different approach/format/factoring seem to be required for making the manuscript more interesting and concise. I have done a very bad work on highlighting key-features of our work; maybe a different organization of section and topic-focusing could help to make the description less boring (I guess that many will stop to read after the abstract in current form);
examples: currently the focus is on the Oscillation equations because it is a very nice toy-test (thank you Milan for making me conscious of that). Its implementation can help on describing FOODIE features and it is very useful for assessing the accuracy of the solvers provided out of the box. However, I think that it could be helpful to insert also a PDE example (for completing the scenario of FOODIE audience): the Euler 1D conservation is completed (mostly used in the parallel performance analysis) and can used to this, but I am worry about its (of examples section) resulting lengthy description. Your suggestions are very welcome.
parallel benchmarks: I am conscious that you are skeptical about the necessity of this topic, but I think that demonstrating the safeness of FOODIE usage into a parallel framework is important; during the implementation of FOODIE I was concerned that the great level of abstraction of Damian's approach could destroy the parallel scaling, but the tests made seem to prove that this is not the case; in my opinion this a valuable key-point. However, for this topic I was concerned about the quality of my work: are the OpenMP/CAF tests enough? Do we need also a MPI test (it should be simple to implement)? What about esoteric (for me) accelerators like GPUs/MIC? are my analysis (strong/weak scaling) meaningful? In this topic you (and many other that should be following this project) have more experience than me. I tried to interest other experts like @francescosalvadore and @muellermichel without much success, thus I think we are alone on this.
prove of FOODIE usefulness for rapid development of new solvers: I have implemented many solvers in very few hours (during my lunch breaks!), this is an indirect prove of the quality of Damian's idea. Maybe an example of how to test new solvers could help to prove the FOODIE usefulness. In this regards we can try to implement a hybrid solver that is not a standard one. @andreadimascio has great ideas on this regard, but I am not sure about the time/interest he will want to devote us. Maybe this topic is too ambitious for this manuscript.
Journal: which is the most on topic Journal? My experience is very limited (as a dear friend remember me yesterday, my H index is very small), I can suggest only CPC: its review process is very nice, they deeply reviewed one of my work making a huge list of critics that finally help to improve my work, however it was very time consuming (about 1 year) and my H index needs to be improved quickly :-) (I am kidding, sorry).

sourceryinstitute commented 8 years ago

Please remind me how to access the manuscript.

D

Sent from my iPhone

On Nov 11, 2015, at 3:15 AM, Stefano Zaghi notifications@github.com wrote:

analysis

szaghi commented 8 years ago

Sorry for my lack!

You can clone the repository:

git clone --recursive https://github.com/Fortran-FOSS-Programmers/FOODIE

then you have access to both the LaTeX sources and the compiled pdf into the directory:

cd FOODIE
cd papers/announcement/
tree -L 1
.
├── caf_benchmark                                                                                                                                                                                                                            
├── elsarticle.cls                                                                                                                                                                                                                           
├── elsarticle-num.bst                                                                                                                                                                                                                       
├── elsarticle-num-names.bst                                                                                                                                                                                                                 
├── errors_analysis                                                                                                                                                                                                                          
├── images                                                                                                                                                                                                                                   
├── manuscript.aux                                                                                                                                                                                                                           
├── manuscript.bbl                                                                                                                                                                                                                           
├── manuscript.blg                                                                                                                                                                                                                           
├── manuscript.fdb_latexmk                                                                                                                                                                                                                   
├── manuscript.fls                                                                                                                                                                                                                           
├── manuscript.log
├── manuscript.out
├── manuscript.pdf
├── manuscript.spl
├── manuscript.synctex.gz
├── manuscript.tex
├── mycpc2.bst
├── openmp_benchmark
├── Reference.bib
├── src
└── utilities

the last PDF is manuscript.pdf, while the LaTeX sources are into src sub-directory.

Alternatively, (if you have not access to git) you can download either the whole project by means of the download ZIP button into the main page of FOODIE or download only the pdf by means of this link https://github.com/Fortran-FOSS-Programmers/FOODIE/blob/master/papers/announcement/manuscript.pdf

If you download the whole ZIP the tree structure of the sub-directories should be similar to the one depicted in the git clone approach.

See you tomorrow!

milancurcic commented 8 years ago

@szaghi Thank you for arranging this, works for me. See you tomorrow. Let me know whether we will use hangouts or skype.

Cheers, milan

On Wed, Nov 11, 2015 at 2:49 AM, Stefano Zaghi notifications@github.com wrote:

@rouson https://github.com/rouson @milancurcic https://github.com/milancurcic @zbeekman https://github.com/zbeekman

Dear All,

I have booked a Damian's time-slot for tomorrow, November 12th at 5:30 PM (or more clear format for me 17:30) Rome time (I will try to convert to your local time :-). I booked 1 hour slot because due to my bad English I guess that some minutes will be lost in translation, but I hope we can finish before 6:30 PM. I think that for Milan and Zaak this slot should be good (more or less 6 hours before Rome time I think for Milan and Zaak), but if I am wrong, please let Damian free his time as soon as possible.

To facilitate our discussion, I am going to list here some topics: feel free to add/modify/delete any topics.

Unfortunately, I cannot use phone (to costly), thus I prefer a skype or hangout conference call. My gmail account is stefano.zaghi@gmail.com, while my skype username is stefano.zaghi (I have not fantasy for username...).

Thank you all for dedicating your time to FOODIE.

— Reply to this email directly or view it on GitHub https://github.com/Fortran-FOSS-Programmers/FOODIE/issues/20#issuecomment-155695458 .

szaghi commented 8 years ago

@milancurcic ok, I think skype is better (maybe we can try a video conference, I do not why but the few times I have used hangout I feel there were some delays), but as you I am good with both. @rouson let us know your preference. The only not valid option for me is phone calling.

See you.

rouson commented 8 years ago

On Nov 11, 2015, at 8:51 AM, Stefano Zaghi notifications@github.com wrote: tree -L 1

Thanks! BTW, I had never seen “tree” before and just installed it after seeing it in your message. I’m love with it! Oh, tree, where have you been all my life?

:D

zbeekman commented 8 years ago

my Skype name is zaak.beekman. My Gmail is zbeekman@gmail.com I should be able to participate, but I haven't had a chance to keep up with the recent developments, so I am going to read through the draft/blog as it is now to try to get up to speed. @szaghi thanks for including me in this.

@rouson I'm so happy you're on GitHub now! IMO it is a much better collaborative space and community. Also I haven't forgotten about our work together... I'm sorry I haven't been more engaged but I am pushing to finally get the thesis finished. I think there's one chapter that will be of interest to you and others here, but I won't go into details just yet.

szaghi commented 8 years ago

@zbeekman great!

szaghi commented 8 years ago

@rouson @sourceryinstitute Hi Damian, do you prefer skype or hangout for our talk? our contacts is

me skype: stefano.zaghi gmail: stefano.zaghi@gmail.com
Zaak skype: zaak.beekman gmail: zbeekman@gmail.com
Milan I have his contacts

rouson commented 8 years ago

My Skype name is damian.rouson

I'm on Skype on my phone now.

D

Sent from my iPhone

On Nov 12, 2015, at 7:47 AM, Izaak Beekman notifications@github.com wrote:

my Skype name is zaak.beekman. My Gmail is zbeekman@gmail.com I should be able to participate, but I haven't had a chance to keep up with the recent developments, so I am going to read through the draft/blog as it is now to try to get up to speed. @szaghi thanks for including me in this.

@rouson I'm so happy you're on GitHub now! IMO it is a much better collaborative space and community. Also I haven't forgotten about our work together... I'm sorry I haven't been more engaged but I am pushing to finally get the thesis finished. I think there's one chapter that will be of interest to you and others here, but I won't go into details just yet.

— Reply to this email directly or view it on GitHub.

zbeekman commented 8 years ago

Notes from conference call:

Increase emphasis on PDEs and applications. Emphasis that this is about implementation, not the actual algorithm.

Journal Candidates for submission:

Computers in Science and Engineering (CiSE) published by American Institute of Physics (AIP) and IEEE (new peer reviewed software engineering track)
ACM Transactions on Mathematical Software
Scientific Programming (Damian has concerns about the poorly defined scope of the journal)
CPC (Computer Physics Communications)
SIAM journal?
If we get rejected everywhere, we could submit ACM Fortran Forum

Make oscillation analysis in line with literature: see Dale Durran - Numerical Methods in GFD

Submit to a conference before journal submission?

Look at different CAF implementations, also Damian has some concerns that ACP might be an anti-pattern.

zbeekman commented 8 years ago

Grant/funding ideas:

Symbiotic/parasitic joining up with a "host" grant ~ up to 25% of total grant... ? maybe?
Good conferences to find collaborators: application conferences
Jeff Carver UA: http://carver.cs.ua.edu/
1. Hosts workshops on HPC/scientific computing/engineering at conferences like "Super Computing"
2. Invites top papers/presenters to submit to special issue

szaghi commented 8 years ago

@zbeekman @milancurcic @rouson Thank you all!

milancurcic commented 8 years ago

Thank you, it was pleasure!!

On Thu, Nov 12, 2015 at 12:57 PM, Stefano Zaghi notifications@github.com wrote:

@zbeekman https://github.com/zbeekman @milancurcic https://github.com/milancurcic @rouson https://github.com/rouson Thank you all!

— Reply to this email directly or view it on GitHub https://github.com/Fortran-FOSS-Programmers/FOODIE/issues/20#issuecomment-156184228 .

Fortran-FOSS-Programmers / FOODIE

Announcement Paper #20

conference call main topics