search

tuura / graph-families

Collection of benchmarks for graph families and associated algorithms
Other
0 stars 1 forks source link

Bring processor benchmarks up-to-date and make them reproducible with tuura/scenco #4

Open snowleopard opened 8 years ago

snowleopard commented 8 years ago

At the moment processor benchmarks are hard to reproduce: they require Workcraft and there is a high chance that the reported results are outdated due to on-going development of the SCENCO plugin: https://github.com/tuura/graph-families/tree/master/real/processors.

As soon as https://github.com/tuura/scenco is released we should include precise instructions for reproducing the results directly with scenco (without relying on Workcraft) and make sure the results are up-to-date. We can keep old results for comparison, this will also allow us to see how our encoding tools improve over time.

allegroCoder commented 8 years ago

I have translated all the .work files present in https://github.com/tuura/graph-families/tree/master/real/processors, into a format compatible with scenco. That is:

.scenario name_scenario
Vertex_A Vertex_B
:Boolean_condition Vertex_A
.end

All of them are present inside the test folder, where I am going to insert a file named 90nm.genlib, useful for testing technology mapping related part.

snowleopard commented 8 years ago

@allegroCoder Excellent! That's a good first step.

As far as I understand we still can't reproduce all the reported results, because we are missing cross-platform ABC-based synthesis and technology mapping algorithms?

snowleopard commented 8 years ago

By the way, I think the current text format will eventually become too limiting for us. At some point we will probably switch to the Farfalle language for describing processor instructions.

allegroCoder commented 8 years ago

Last comment: Sure, Farfalle language would be nice for the processor instruction description. What about partial order description? I think that the format listed above can be fine for the partial orders.

In my view, such a file might still be used, even though it can be hidden by the whole framework. It can be used for passing the partial orders to scenco via a temporary file.

And at that point we will use Farfalle language for the description of the tests we run, as you mentioned! :+1:

snowleopard commented 8 years ago

@allegroCoder Good idea about an intermediate format for representing partial orders, and in general graphs. I think we can use an algebraic form, e.g. p = a + b -> (c + d), which is used in @jrbeaumont's plugin for Workcraft. I will implement a parser for it as part of https://github.com/tuura/scenco/issues/4.