A/B test results should reference commit

[fjetter]

Typically, I look at A/B test results by downloading the static-dashboard zip archive. My Downloads folder is riddled with folders like static-dashboard (11).

Opening the HTML page, the results show the name that was given to the AB config files but they are not showing the commit which makes it hard to distinguish the results. Effectively, if I do not rename and order anything the downloads are one-shots because I cannot truly distinguish different runs.

I think we should

• Embed the commit or even the entire config in the HTML
• rename the static-dashboard folder with a commit or other helpful identifier (even the branch would be a start, branch+commit might be pretty helpful)

[crusaderky]

I'd rather upload to s3 instead so that the whole thing is accessible from just a URL that you get from the build log. The s3 bucket should be set up to be public and automatically clean up files after 2 months.

I'd suggest steering clear of gh pages since they make the .git folder gargantuan, don't support multiple branches, and don't feature any built-in cleanup. Of course all of these shortcomings may have already been addressed by some third party action but I couldn't find one.

[fjetter]

Sorry, I think my comment about the download is misleading. I don't truly care about the download. I think uploading to S3 is also a great idea but what I'm looking for in this ticket is that the HTML report is enriched with sufficient information about the run that produced it such that I can identify what it shows me by just looking at the HTML.

I think I would simply like us to embed all the yaml files in the HTML report as a tab or smth.

For instance, the index.html could just include the yaml file verbatim or contain a link to another page with the relevant info.