Just what exact reports will be put on the website?

[jernst]

I think we have two audiences:

1. Somebody who is interested in an overview over "how buggy is the Fediverse" as a whole. They don't really need/want to know whether something returned 400 vs 404 in some test for some particular application. Instead they want to see a single picture that conveys an impression, and perhaps which implementations in general are more correct or less correct.

   I think that's what our test matrix is for.

2. The developer of a specific implementation. They want know what exactly they need to fix in their code. So they do need to know whether it's 404 or 400, and why the test tests that. They don't really care very much about issues in other people's code bases.

   I think that's what our sequential report is for.

So I'm thinking we could either generate a single, gigantic web page that has both matrix and sequential, or we could generate one matrix page, and then a separate page per implementation/constellation.

[steve-bate]

we could generate one matrix page, and then a separate page per implementation/constellation.

I like that idea. I can envision the web site section TOC looking something like:

• Webfinger
  • Test Matrix
  • Server 1
  • Server 2
  • Server 3

With the matrix column headings linking to the individual server page with the test details.

[jernst]

Is there any good pattern in Jinja2 for how to generate such multi-page reports? Keeping track of the links etc ... or we just make it up?

[steve-bate]

Is there any good pattern in Jinja2 for how to generate such multi-page reports? Keeping track of the links etc ... or we just make it up?

I don't think it's a jinja2-specific pattern, but I created a little PoC locally with a "matrix" and "session" template. The session template is used for each session and generates a file using the session name in the file name. The matrix file table headings link the session name to the session-specific file (instead of the current internal link). My directory hierarchy looks like:

• matrix.html
• sessions/
  • Session 1.html
  • Session 2.html
  • (etc.)

This uses a base directory for the set of report files instead of a single file path.

[jernst]

If you already have a little PoC, can we use that?

[steve-bate]

If you already have a little PoC, can we use that?

I pushed it to my repo in branch local/multifile-report. It demonstrates some ideas, but it won't be straightforward to merge it directly into the current develop branch. At least, I think that's true.

[jernst]

I'm going t close this in favor of (less to read) #173.