Automatically report on repository status

[etpeterson]

Feature description

A simple report on the repository containing information summarizing the repository like how many submissions it contains and how many are being tested could be useful. It could also contain a little bit of information about each submission, if that is provided.

Describe the solution

A simple html or pdf report that summarizes the repository statistics would be nice to have. The number of submissions, number of wrapped (tested) submissions, and a little bit of information about each would be a good start.

This could initially just be generated as a workflow artifact, but could also be pushed to a github page as well.

I imagine this as being a small python "test" that parses the repository for this information and generates a test artifact. It could be run with each merge to the main branch to update the information every time the repository changes.

Describe alternatives

No response

Additional context

No response

Are you working on this?

None

[kailash360]

@etpeterson

Can I start with this issue?

[etpeterson]

Absolutely! A good place to start would be to read this file as a record of all contributions but this file contains what is being tested. And there's also the structure of the repository with the original code in its own folder and a wrapper file in a different folder. Between these 4 pieces of information we should be able to say what we have, if it's wrapped, and if it's being tested.

I could imagine a file where there's a header with all the contributions in a simple table with this and perhaps some author information and maybe a one line description. Perhaps some of the columns from the code contribution file for example.

There could also be some more information about each contribution in a different section. I'm not sure where this would come from, but we could think about adding it as a method on the wrapper to return a detailed description. I think this could come later, but there might also be some way to scrape docstrings without changing any code.

[abhicodes369]

I tried and got this. please tell if you have any suggestions

[etpeterson]

Hi @abhicodes369, thanks for taking a try at this. @kailash360 also wanted to work on this so I want to be sure everyone gets a shot. Can you assign yourselves to these? I see an assignee gear in the upper right, can you self assign?

Now to your question. They should largely be getting tested. Here's a list that's being read for testing. Maybe there's a better way, but that's a start at least. Why are some missing the algorithm? It seems like it's mostly printing the names of Ivan's algorithms and not others'. Perhaps a table format as well? It's a little hard to read the list because the line lengths are not consistent.

[abhicodes369]

In the algorithm (JSON) file properties of algorithms (tolerance values ) of three more algorithms should be updated, and how should I include these values in the resultant HTML table

[etpeterson]

Thanks @abhicodes369 . That's looking good but is now missing the testing information. Maybe that's where you're going with the json file though.

That file is generated by the Algorithm Analysis workflow. I would expect that to fail if we're missing tolerances though. What algorithms are missing?

[abhicodes369]

PvH_KB_NKI_IVIMfit PV_MUMC_biexp OGC_AmsterdamUMC_biexp Tolerances for these three algorithms are missing

[etpeterson]

@abhicodes369 it took me a while to figure out what you were referring to. Next time a link to the file or more specifics would help. There are default tolerances handled in a few places like here so no need to change that file.

No need to read the tolerances for this though, if the algorithm is listed there it's tested. The originals are listed from the folder, we have the contributors file, and the wrapper files. So just another column for "Tested" would be good.

[abhicodes369]

I tried and got this please tell me if you have any suggestions

[etpeterson]

Probably just a column in the same table for tested yes/no.

Is there code associated with this? It would be good to get a pull request going.

[abhicodes369]

can I send a pull request ? please tell me if anything needs to be modified

[etpeterson]

can I send a pull request ? please tell me if anything needs to be modified

Absolutely, that's the best way to get started.