NOAA-EMC / NCEPLIBS

Top level repo containing submodules for NCEPLIBS and associated dependencies for superproject builds
Other
43 stars 18 forks source link

All NCEPLIBS should have doxygen documentation, docs from latest release on gh-pages branch of repo #121

Closed edwardhartnett closed 1 year ago

edwardhartnett commented 4 years ago

All NCEPLIBS should have good doxygen documentation. A link to documentation for the most recent release is always available from the project README page, and uses GitHub gh-pages branch to publish built documentation.

Getting doxygen docs together will involve the following:

Markdown files can be incorporated into Doxygen docs directly. That is, you don't have to move your .md text into code files. Within the .md files, you can cross-reference any function by using parens after the name, and doxygen will provide the hyperlink. (So nc_create() instead of nc_create will be hyperlinked).

Fortran is a challenge with Doxygen for sure. A good example can be found in the PIO library, documentation here: https://ncar.github.io/ParallelIO/api.html.

edwardhartnett commented 4 years ago

For reference, here's how I build and release documentation for the PIO project:

Update Documentation: The master branch is checked out, and configured with the --enable-docs option. (Doxygen is required). The documentation is built by doing a make in the doc directory. This produces the HTML version of the documentation in the (confusingly named) docs directory. Another clone of the library is checked out and the gh-pages branch checked out. A git rm -rf is done on the gh-pages branch to delete all files. Then the docs directory is copied there and a git add adds the files to the gh-pages branch. This is then committed and GitHub will display it as project documentation. Note that it can take some time after pushing the changes before the web page with the documentation gets updated. (See also https://stackoverflow.com/questions/11577147/how-to-fix-http-404-on-github-pages). The commands to do this:

git clone git@github.com:NCAR/ParallelIO.git p1
cd p1/
git checkout gh-pages
git rm -rf *
cp -R ../ParallelIO/docs/* .
git add *
git commit -m "updated docs" -a && git push
edwardhartnett commented 4 years ago

https://github.com/NOAA-EMC/NOAA-EMC.github.io/tree/master/NCEPLIBS https://noaa-emc.github.io/NCEPLIBS/NCEPLIBS-nemsio/html/ https://noaa-emc.github.io/NCEPLIBS/NCEPLIBS-sfcio/html/ https://noaa-emc.github.io/NCEPLIBS/NCEPLIBS-sigio/html/ https://noaa-emc.github.io/NCEPLIBS/NCEPLIBS-sp/html/ https://noaa-emc.github.io/NCEPLIBS/NCEPLIBS-w3emc/html/ https://noaa-emc.github.io/NCEPLIBS/NCEPLIBS-w3nco/html/

edwardhartnett commented 4 years ago

Copied from https://github.com/NOAA-EMC/NCEPLIBS-bufr/issues/15:

We are going to standardize on doxygen for all NCEPLIBS.

For an example of what this might look like with NCEPLIBS, see the NCEPLIBS-sp project, which is pretty far along in doxygen conversion. https://noaa-emc.github.io/NCEPLIBS-sp/

The old ways of doing things have taken us far, and given us this valuable legacy software. It's our role to now move that forward to 21st Century methods of software development, including more testing, increased continuous integration, better documentation, and more portable builds. In this way, NOAA and the science community can continue to receive benefit from this important software.

The NCEPLIBS team will benefit from doxygen in the following ways:

Learning doxygen is easy, and can be done mostly from example. There are times when you may wish to google a question, but doxygen is the leading documentation tool in the free software world, and the answers are out there.

As we convert to doxygen, code changes that don't support the doxygen documentation should not be merged. In other words, we cannot let anyone break or neglect the docs moving forward. Every function must be properly documented, and we will check that in the CI and in code review.

A note of warning: doxygen does not have great support for Fortran 90! However, as demonstrated by the HDF5, netCDF, parallel-netcdf, and many other scientific fortran libraries, it can do the job, albeit a bit clumsily at times.

edwardhartnett commented 4 years ago

Some comments on authorship:

As is customary with science projects we will allow each contributor to add their name to the finished product.

One purpose of the list of authors in a function is that it is the list of people who take responsibility for that function.

Tracking authorship has benefit besides fairness and historical record. Programmers crave legacy and too often this craving is satisfied by the creation of unneeded code. This adds to project complexity and costs.

Instead, we will allow everyone to take credit for all the needed code they have contributed, and remove unneeded code. In this way, programmers are encouraged to build legacy as they should - by make valid contributions to necessary code.

edwardhartnett commented 4 years ago

More doxygen conversion advice, from another issue:

OK, what I'm seeing in the file is:

!--------------------------------------------------------------------------------
!:sdoc+:
!
! NAME:
!       CRTM_Forward
!
! PURPOSE:
!       Function that calculates top-of-atmosphere (TOA) radiances
!       and brightness temperatures for an input atmospheric profile or
!       profile set and user specified satellites/channels.
!
! CALLING SEQUENCE:
!       Error_Status = CRTM_Forward( Atmosphere       , &
!                                    Surface          , &
!                                    Geometry         , &
!                                    ChannelInfo      , &
!                                    RTSolution       , &
!                                    Options = Options  )
!
! INPUTS:
!       Atmosphere:     Structure containing the Atmosphere data.
!                       UNITS:      N/A
!                       TYPE:       CRTM_Atmosphere_type
!                       DIMENSION:  Rank-1 (n_Profiles)
!                       ATTRIBUTES: INTENT(IN)
!
!       Surface:        Structure containing the Surface data.
!                       UNITS:      N/A
!                       TYPE:       CRTM_Surface_type
!                       DIMENSION:  Same as input Atmosphere structure
!                       ATTRIBUTES: INTENT(IN)
!
!       Geometry:       Structure containing the view geometry
!                       information.
!                       UNITS:      N/A
!                       TYPE:       CRTM_Geometry_type
!                       DIMENSION:  Same as input Atmosphere structure
!                       ATTRIBUTES: INTENT(IN)
!
!       ChannelInfo:    Structure returned from the CRTM_Init() function
!                       that contains the satellite/sensor channel index
!                       information.
!                       UNITS:      N/A
!                       TYPE:       CRTM_ChannelInfo_type
!                       DIMENSION:  Rank-1 (n_Sensors)
!                       ATTRIBUTES: INTENT(IN)
!
! OUTPUTS:
!       RTSolution:     Structure containing the soluition to the RT equation
!                       for the given inputs.
!                       UNITS:      N/A
!                       TYPE:       CRTM_RTSolution_type
!                       DIMENSION:  Rank-2 (n_Channels x n_Profiles)
!                       ATTRIBUTES: INTENT(IN OUT)
!
! OPTIONAL INPUTS:
!       Options:        Options structure containing the optional arguments
!                       for the CRTM.
!                       UNITS:      N/A
!                       TYPE:       CRTM_Options_type
!                       DIMENSION:  Same as input Atmosphere structure
!                       ATTRIBUTES: INTENT(IN), OPTIONAL
!
! FUNCTION RESULT:
!       Error_Status:   The return value is an integer defining the error status.
!                       The error codes are defined in the Message_Handler module.
!                       If == SUCCESS the computation was sucessful
!                          == FAILURE an unrecoverable error occurred
!                       UNITS:      N/A
!                       TYPE:       INTEGER
!                       DIMENSION:  Scalar
!
! COMMENTS:
!       - The Options optional input structure argument contains
!         spectral information (e.g. emissivity) that must have the same
!         spectral dimensionality (the "L" dimension) as the output
!         RTSolution structure.
!
!:sdoc-:
!--------------------------------------------------------------------------------

So, how to change this?

Doxygen comments start with !> and can be continued with !! or !>. (Or C> for F77 code.)

No need for text like NAME or PURPOSE, and no need to provide the CALLING SEQUENCE, doxygen will understand all those things automatically and display them appropriately.

Extra whitespace is ignored, so there's no need or point to line up comments or text.

Lists can be generated like this:

- item1
- item2

Tables like this:

h1 | h2
----|----
c1 | c2

(The above text will be a html table in the doxygen output.)

Parameters need to be designated like this:

!> @param[in] Atmosphere Structure containing the Atmosphere data. To document the return code:

!> @return Error_Status explaination...

Here's what one looks like in NCEPLIBS-sp:

C> Computes the wind components from divergence and vorticity
c> in spectral space.
C> Subprogram speps should be called already.
C> If L is the zonal wavenumber, N is the total wavenumber,
C> <pre>      
C> EPS(L,N) = SQRT((N**2-L**2)/(4*N**2-1))
C> </pre>
C> and A is earth radius,
C> then the zonal wind component U is computed as
C> <pre>
C> U(L,N)=-I*L/(N*(N+1))*A*D(L,N)
C> +EPS(L,N+1)/(N+1)*A*Z(L,N+1)-EPS(L,N)/N*A*Z(L,N-1)
C> </pre>
C> and the meridional wind component V is computed as
C> <pre>
C> V(L,N)=-I*L/(N*(N+1))*A*Z(L,N)
C> -EPS(L,N+1)/(N+1)*A*D(L,N+1)+EPS(L,N)/N*A*D(L,N-1)
C> </pre>
C> where D is divergence and Z is vorticity.
C> U and V are weighted by the cosine of latitude.
C> Cxtra terms are computed over top of the spectral domain.
C> Advantage is taken of the fact that EPS(L,L)=0
C> in order to vectorize over the entire spectral domain.
C>
C> @param I        - INTEGER SPECTRAL DOMAIN SHAPE
C>               (0 FOR TRIANGULAR, 1 FOR RHOMBOIDAL)
C> @param M        - INTEGER SPECTRAL TRUNCATION
C> @param ENN1     - REAL ((M+1)*((I+1)*M+2)/2) N*(N+1)/A**2
C> @param ELONN1   - REAL ((M+1)*((I+1)*M+2)/2) L/(N*(N+1))*A
C> @param EON      - REAL ((M+1)*((I+1)*M+2)/2) EPSILON/N*A
C> @param EONTOP   - REAL (M+1) EPSILON/N*A OVER TOP
C> @param D        - REAL ((M+1)*((I+1)*M+2)) DIVERGENCE
C> @param Z        - REAL ((M+1)*((I+1)*M+2)) VORTICITY
C> @param U        - REAL ((M+1)*((I+1)*M+2)) ZONAL WIND (TIMES COSLAT)
C> @param V        - REAL ((M+1)*((I+1)*M+2)) MERID WIND (TIMES COSLAT)
C> @param UTOP     - REAL (2*(M+1)) ZONAL WIND (TIMES COSLAT) OVER TOP
C> @param VTOP     - REAL (2*(M+1)) MERID WIND (TIMES COSLAT) OVER TOP
C>
      SUBROUTINE SPDZ2UV(I,M,ENN1,ELONN1,EON,EONTOP,D,Z,U,V,UTOP,VTOP)

This produces the output here: https://noaa-emc.github.io/NCEPLIBS-sp/spdz2uv_8f.html

This is the default look for doxygen documentation. It supports stylesheets, so we can make this look more pretty if desired. ;-)

BenjaminTJohnson commented 4 years ago

given my current workload, there’s no way I’ll even be able to touch this issue for at least two weeks.

edwardhartnett commented 4 years ago

Don't worry I'm just capturing this explanation for other projects in NCEPLIBS. You are not under any deadline for getting documentation working. When you get your repo ready to start on doxygen, let me know and I can help.

edwardhartnett commented 3 years ago

Here's how the gh-pages branch needs to be updated every time a new version of a library is released.

What you do is first build the docs:

git clone git@github.com:NOAA-EMC/NCEPLIBS-g2c.git 
cd NCEPLIBS-g2c
mkdir build
cd build
cmake -DENABLE_DOCS=ON ..
make

Now the docs are in NCEPLIBS-g2c/build/docs/html.

Then, take another clone of the repo, switch to the gh-pages branch, delete contents, and add the HTML output:

git clone git@github.com:NOAA-EMC/NCEPLIBS-g2c.git dox_NCEPLIBS-g2c
cd dox_NCEPLIBS-g2c
git checkout gh-pages
git rm -rf *
cp -R NCEPLIBS-g2c/build/docs/html/* .
git add *
git commit -m "updated docs" -a
git push

Take care that you are on the gh-pages branch before doing git rm -rf *!

After you do this, in a few minutes, github will update the documentation at https://noaa-emc.github.io/NCEPLIBS-g2c/. It does take a few minutes to update.

Ask for help if you need it. This will have to be done every release, on every repo. ;-)

edwardhartnett commented 3 years ago

THe status on this is that many projects now have their first draft of documentation.

UFS_UTIL and w3emc are still under development. I do a PR every few days, hopefully both will have complete first drafts of documentation by the end of the year.

I am also doing documentation for UPP, and that is underway. I am waiting for the UPP to merge an outstanding development PR before I do the next round of documetation there. The UPP may take into next year to get the first draft of documentation done.

jbathegit commented 3 years ago

Here are the Doxygen tips and notes from my NCEPLIBS presentation earlier today: https://docs.google.com/document/d/13d1X8GjiFhe72aG6zKoJ5tJfSWeYfQCGVSMNgNpaDvY/edit?usp=sharing

edwardhartnett commented 3 years ago

More tips:

Parameters are just the ones that start with @param.

Another tag that should be on a line by itself is @file.

Also the @brief tag is usually on a line by itself.

Other tags, like @author and @date, may appear on the same line.

The @date tag can be kept if a date is present, if not, don't worry about the date tag. Some functions have a date, some don't. We won't try to figure out the date if it is missing. (But we will figure out the author, which is much more important.)

edwardhartnett commented 3 years ago

Here's a presentation I gave on doxygen: Doxygen and the UFS Code.pdf

edwardhartnett commented 1 year ago

OK, we have converted to doxygen on all active NCEPLIBS projects. Documentation will continue to improve, but I will close this issue.