clearly report gateware timing failure in buildbot

[sbourdeauducq]

Fail the build, or issue a warning. (Sometimes failed-timing bitstreams are still usable)

[whitequark]

Is there a reliable way to detect one?

[sbourdeauducq]

Grep ISE logs for "All constraints were met", which means timing passed. I'm not sure about Vivado, nor if there is a better way than grepping the ISE log. One would have to read the Xilinx docs and experiment.

[jordens]

for vivado if the stdout contains a line that looks like WARNING: [Route \d+-\d+] Router estimated timing not met.. For ISE, \d+ constraint not met..

[whitequark]

Should I extract some block from stdout for ease of analysis? If yes, how is it delimited?

[jordens]

You could look at the xml files vivado and ise leave behind. Maybe they are easier to parse.

[whitequark]

Yes but is it helpful if I give you the xml output? (I have no idea how do people debug timing failures.)

[sbourdeauducq]

Simply having timing pass/fail information on IRC and in the global build logs e.g. with colors (all similar to build pass/fail) would already be helpful.

[whitequark]

Sure, but since the extraction techniques for pass/fail and some extended report info are nearly same, I can do the latter also.

[sbourdeauducq]

The useful information would be the names of the top-3 signals with the worst negative slack in each failed clock domain.

[jordens]

The xml is not helpful for a human. Just if you want to extract stuff it is sometimes easier than plain text. This is in top_timing.rpt (vivado) and top.twr (ise). But they look painful to extract. On ISE we have top.twx with the same info in xml. I think vivado can be made to generate that timing report in xml as well.

[whitequark]

I don't need XML, if there is consistent stdout output, then it is far easier to extract. XML is a pain to parse anyway. I can grab top_timing.rpt from the builder if you want, sure.

[sbourdeauducq]

Let's keep it simple and grep the log as in https://github.com/m-labs/artiq/issues/310#issuecomment-191193856

[sbourdeauducq]

Ping

[whitequark]

Done

[whitequark]

This fails the build, let me know if you'd rather get a warning.