for datalad-extensions we rely on CI services to provide badges, and so than when clicked would lead to the detailed log. Generally works great but relies on services to provide badges and logs. logs eventually expire. badges some times flaky to display
testkraken's dashboard is more evolved -- it is an html with JS to navigate that fancy pane to navigate combinatorics of the runs, similar to the one of the http://openneu.ro/metasearch/ . But to visualize it you need webserver to serve it and artifacts (logs), unlike in datalad-extensions we just rely on markdown rendering by github.
Not sure if we could somehow embed JS driven dashboard, but here is an idea (not yet tried/implemented) on how to provide "scalable" badges + storage of details for each component :
details for a particular "test suite" (e.g. core, or per extension) run could reside in a markdown file , which along with all the additional artifacts (logs, prov capture from pydra if using testkraken etc) could reside a commit, detached from the master, with some git ref pointing to it (e.g. run-<testsuite>-<date>), and most uptodate pointed by (e.g. run-<testsuite>-latest) branch (thus serviced by github raw. service etc).
overall status "badge image" could also reside there as .png,
then main (or master) branch README.md would just point to the images and detailed dashboard in that branch
this way we
can keep some collection of refs for previous runs, but remove (and thus github's GC remove them) elderly ones (while some local clone possibly collecting them all for "backup")
this would keep .git/objects manageable even if we keep populating repository with the all recent results (in those detached from master refs)
this idea is somewhat inspired by
git-annex-remote-gitobjects git-annex special remote prototype to (ab)use git objects and refs to provide git-annex special remote
current idea on metadata storage/handling in https://github.com/datalad/datalad-metalad/ (current datalad stores metadata dumps annexed under .datalad/metadata/objects, thus affecting the tree and .git/objects)
for datalad-extensions we rely on CI services to provide badges, and so than when clicked would lead to the detailed log. Generally works great but relies on services to provide badges and logs. logs eventually expire. badges some times flaky to display
testkraken's dashboard is more evolved -- it is an html with JS to navigate that fancy pane to navigate combinatorics of the runs, similar to the one of the http://openneu.ro/metasearch/ . But to visualize it you need webserver to serve it and artifacts (logs), unlike in datalad-extensions we just rely on markdown rendering by github.
Not sure if we could somehow embed JS driven dashboard, but here is an idea (not yet tried/implemented) on how to provide "scalable" badges + storage of details for each component :
refpointing to it (e.g.run-<testsuite>-<date>), and most uptodate pointed by (e.g.run-<testsuite>-latest) branch (thus serviced by github raw. service etc).master) branch README.md would just point to the images and detailed dashboard in that branchthis way we
.git/objectsmanageable even if we keep populating repository with the all recent results (in those detached frommasterrefs)this idea is somewhat inspired by