Add UoD SATRE evaluation

manics commented 1 year ago

:white_check_mark: Checklist

[x] This pull request has a meaningful title.
[x] If your changes are not yet ready to merge, you have marked this pull request as a draft pull request.

:ballot_box_with_check: Maintainers' checklist

[ ] This pull request has had the appropriate labels assigned
[x] This pull request has been added to the SATRE backlog project board
[ ] This pull request has been assigned to one or more maintainers

:arrow_heading_up: Summary

Adds the UoD SATRE evaluation

This adds the CSV (exported from the Excel file)- I can add the Excel if others think that's helpful, but I think it's better to avoid committing opaque binary blobs if possible. Maybe we should upload it to Zenodo.....?

To create the Markdown table I tried https://github.com/sphinx-contrib/datatemplates but it doesn't handle multi-line table cells so I had to fork it: https://github.com/sphinx-contrib/datatemplates/compare/master...sa-tre:sphinx-datatemplates:multiline-table-item?expand=1

I'd like to open a PR against upstream, but it'll need a test to be added which requires more time, ~~so at the moment it's in my own GitHub account~~. If everyone's happy with this approach I'll ~~transfer the repo to the sa-tre org, and~~ hopefully contribute the fix upstream in future.

:closed_umbrella: Related issues

:raising_hand: Acknowledging contributors

[ ] All contributors to this pull request are already named in the table of contributors in the README file.
[ ] The following people should be added to the table of contributors in the README file:

manics commented 1 year ago

@sa-tre/spec-maintainers Could you let me know what you think of this approach? https://satre-specification--297.org.readthedocs.build/en/297/evaluations/dundee_hic.html The formatting isn't ideal, but I've spent enough time fighting with Sphinx and to improve things I think we may want to write our own plugin based on datatemplates. I still think it's better than just making the CSV available.

manics commented 1 year ago

I've added a link to TREEHOOSE though. It's a bit complicated though because our production TRE isn't yet using the current TREEHOOSE code base (it's using a much older forked version of https://github.com/awslabs/service-workbench-on-aws so they have the same root).

edwardchalstrey1 commented 1 year ago

I've added a link to TREEHOOSE though. It's a bit complicated though because our production TRE isn't yet using the current TREEHOOSE code base (it's using a much older forked version of https://github.com/awslabs/service-workbench-on-aws so they have the same root).

Ok good, maybe still worth having a brief explainer at the top on what Dundee HIC is, and that you have an AWS cloud-based TRE set up for research - see what I've written #298

JimMadge commented 1 year ago

The table from csv idea is great.

It feels like much less to ask for a csv than for people to do the evaluation, then type it up in Markdown!

manics commented 1 year ago

I think this is ready now!

sa-tre / satre-specification