[EPIC] Free the documentation

untereiner commented 1 year ago

Description

This is a proposal to have two repository: one for the documentation (user manual, tutorials, developer guide, doxygen) and one for the source code. If someone can clone the GEOS source code repository (+ the third party), he should be able to clone a documentation repository. The goal here is to avoid the annoying commit of generated files like the rst files, xsd file or GeosxConfig.hpp Do not hesitate to comment

Work Scope

The work that has to be done is essentially on the CI side. I made a rough sequence diagram with the current workflow (box Build and Test) plus a new step called "actions/artifact" that publishes some files as artifacts of the successful workflow instance. I added a new workflow called (workflow/doc) with the different steps I imagine necessary to use these artifacts and automate the documentation update.

sequenceDiagram
    autonumber
    actor Reviewer
    actor Developer
    Developer->>Github: Open Pull Request
    box Build & Test
    participant Workflow/ci
    participant actions/check_draft
    participant actions/check_assigned
    participant actions/check_submodules
    participant actions/check_code_style
    participant Workflow/linux_builds
    participant actions/artifact
    end        
    Github->>Workflow/ci: Trigger
    Workflow/ci->>actions/check_draft: Check that the PR is not a draft
    actions/check_draft-->>Workflow/ci: is not a draft
    Workflow/ci->>actions/check_assigned:Check that the PR is assigned
    actions/check_assigned-->>Workflow/ci: is assigned
    Workflow/ci->>actions/check_submodules: Check that submodules are up to date
    actions/check_submodules-->>Workflow/ci: submodules are up to date
    Workflow/ci->>actions/check_code_style: Check code style fills the requirements
    actions/check_code_style-->>Workflow/ci: Code style is ok 
    Workflow/ci->>Workflow/linux_builds: Build and test GEOS
    Workflow/linux_builds->>actions/artifact: Publish artifacts (schema.xsd and GeosxConfig.hpp)
    Workflow/linux_builds->>Workflow/ci: Build and tests ended completed successfully
    Reviewer->>Github: Merge Pull Request
    Github->>Workflow/doc: Trigger - Pull Request beeing closed with is merged flag
    Workflow/doc->>dawidd6/action-download-artifact: Retrieve artifacts from workflow/ci
    Workflow/doc->>actions/checkout: Checkout documentation repository
    actions/checkout-->>Workflow/doc: Source code is checked out
    Workflow/doc->>actions/update_rst_tables: Generating schema-dependent RST files
    actions/update_rst_tables-->>Workflow/doc: Files are generated
    Workflow/doc->>actions/generate doc: Generate doc
    actions/generate doc-->>Workflow/doc: Files are genereated
    Workflow/doc->>create pull request: Conditionnaly create a pull request and add it to the merge queue
    create pull request-->>Workflow/doc: Command completed successfully

Issues in this EPIC

[ ] Add a new step to current CI that generates the artifacts
[ ] Create a new CI workflow that generates the files for the documentation and test if all is ok
[ ] Create a new repository with the extracted files and make the CI point to it

TotoGaz commented 1 year ago

I totally support any actions which removes our need to commit generated code.

Shifting the doc in another repository (and syncing them) will surely make more complicated the writing of docs or fixing a typo. Writing doc is often a burden, so we should try not to add anything too painful to write it. Managing PRs with sub-modules is not that straightforward, specially for non-IT people. Sometimes also, we only edit doc files in the web UI. I think it would be more complicated with a submodule.

Also, waiting for a PR to be merged to generate the doc can be a problem sometimes (changing a configuration on readthedocs); so it’s probably better to be able to validate first.

If you publish the artifacts (step 12) during a first job and then trigger a standard readthedocs build, this build will be able to retrieve the artifacts and build the doc. Then it should be OK and this would keep the build as simple as we can?

Eventually, note https://github.com/GEOS-DEV/GEOS/pull/2602 about GeosxConfig.hpp.

untereiner commented 1 year ago

Writing doc is often a burden, so we should try not to add anything too painful to write it.

There are two documentations:

the code itself based on doxygen's tag that will generate html pages
the general project documentation with tutorials on how to use such and such component.

The first one is intrinsic to the code. The config file can be embedded with the code but I think that the process generating this documentation could be a CI target for its own.

IMHO the second one is not in the same workflow as the first one. You will write your tutorial only when your component is compiling and effectively working. For me writing such high-level documentation is like writing a report, a document. It can be done using the web UI in its own "shared document" (e.g. repository). Their is no need to put it with the code.

I think submodules are awful. I am not suggesting using submodules. Their is no need of such dependencies. By the way, for code managed with cmake I think FetchContent is a better tool to bring dependencies into the build.

GEOS-DEV / GEOS