Reports service for backend api

[jmw52]

Proposal Summary

To create and publish reports with a REST API.

Motivation

We would like to create reports automatically within a pipeline. Adding multiple plots and text in our code.

[ainoam]

Thanks for suggesting @jmw52.

Can you give a toy example of how you see the usage of such an interface?

[jmw52]

A use case could be when creating pipelines which train multiple models, perhaps different hyper-parameters, and in the end you want to see a report with some basic description of the training runs, and a selection of comparison plots. This report could be auto-generated. I could imagine, on a very high-level, an interface as:

report = clearml.Report(name=""My Report", project="Project", description="Some report")
report.add_text_block("# Title\nSome text here")
report.add_plot(metric="loss", variant="train", experiments=["exp_id_1", "exp_id_2"])
report.add_text_block("Some more text here")
report.publish()

[ainoam]

Appreciate the insight @jmw52.

Adding a similar interface is indeed on our roadmap - we'll update as plans solidify.