We currently use cml-publish to let CML add plots of different metrics to our merge requests so we can do a visual check of the training results. Since cml-publish is deprecated, we want to use cml comment create --publishNative instead. At the moment, there are >300 plots. They are arranged in stacked markdown spans so we can keep the irrelevant ones hidden.
cml comment create --publishNative only uploads 49 of the images in the markdown file and its own watermark. The "metric plots" span contains five more spans for different customers that in turn contain multiple plots each. cml comment create --publishNative only properly uploads the plots for the first customer. The other customers just get one plot each with the exception of testcustomerB, who gets 6.
Run report_before_cml_comment.md through a gitlab pipeline using cml comment --publishNative (CML 0.20.0) and have any images in the paths it uses.
Observe it creating a comment with similar content to report_after_cml_comment.md
Expected behaviour
cml comment create --publishNative should upload all images in the markdown file.
If anything prevents it from doing that, it should throw an error instead of failing silently.
Description
We currently use
cml-publishto let CML add plots of different metrics to our merge requests so we can do a visual check of the training results. Sincecml-publishis deprecated, we want to usecml comment create --publishNativeinstead. At the moment, there are >300 plots. They are arranged in stacked markdown spans so we can keep the irrelevant ones hidden.cml comment create --publishNativeonly uploads 49 of the images in the markdown file and its own watermark. The "metric plots" span contains five more spans for different customers that in turn contain multiple plots each.cml comment create --publishNativeonly properly uploads the plots for the first customer. The other customers just get one plot each with the exception of testcustomerB, who gets 6.I have attached an anonymized example: report_before_cml_comment.md report_after_cml_comment.md
Reproduction
Run report_before_cml_comment.md through a gitlab pipeline using
cml comment --publishNative(CML 0.20.0) and have any images in the paths it uses. Observe it creating a comment with similar content to report_after_cml_comment.mdExpected behaviour
cml comment create --publishNativeshould upload all images in the markdown file. If anything prevents it from doing that, it should throw an error instead of failing silently.