fediverse-devnet / feditest

A testing framework for distributed, heterogeneous systems communicating with complex protocols, such as the Fediverse
https://feditest.org/
MIT License
27 stars 4 forks source link

Added multifile report generation and placeholder templates. #175

Closed steve-bate closed 1 month ago

steve-bate commented 1 month ago

(Documented in the code...)

Generates the Feditest reports into a test matrix and a linked, separate
file per session. It uses a template path so variants of reports can be generated
while sharing common templates. The file_ext can be specified to support generating
other formats like MarkDown.

The generation uses two primary templates. The `test_matrix.jinja2` template is used for
the matrix generation and 'test_session.jinja2' is used for session file generation.

Any files in a directory called "static" in a template folder will be copied verbatim
to the output directory (useful for CSS, etc.). If a file exists in the static folder of more
than one directory in the template path, earlier path entries will overwrite later ones.

The current templates are placeholders. I think it's close to what we need, but I didn't try to complete it because I know you'll want to make changes. The current multifile directory is basically the standalone report. However, we could organize it as something like: multifile-common, multifile-standalone, and multifile-hugo and run the report generator with an argument like --template multifile-hugo,multifile-common to use the shared common templates with overrides for the hugo report generation (in this example).

I added a few features to the reports that I had been requesting. The headings are now duplicated at the bottom of the test matrix table so that context isn't lost when horizontally scrolling the results. Some of the block elements were sized at 100% width, which made them more difficult to read on a large monitor. I modified a few of those to "fit to content" (with added spacing, where needed). I removed the "Session n:" prefix from the session detail pages because it didn't seem to serve a purpose. The per-session file names use the plan session name instead of "session-n.html". It seems like that naming would make it easier to copy or locate a specific session's report from the directory, if needed.

Related to #173