aremazeilles / eurobench_documentation

Description of some Eurobench Benchmarking Software mechanisms
5 stars 3 forks source link

PI output structure #36

Closed aremazeilles closed 4 years ago

aremazeilles commented 4 years ago

Right now the current structure proposed is in simple yaml file. It is described here:

type: 'vector'
value: [0.96867, 1.01667, 0.98843, 0.95168, 0.87936, 0.94576, 0.87802, 0.87571, 0.81802, 0.82336]

We are so far thinking of extending it to contain an additional flag label that could be sued for indicating whether the vector content is strongly related to labels:

type: 'matrix'
label: 1
value: [["label1", "label2", "label3", "label4", "label5", "label6"], [0.96867, 1.01667, 0.98843, 0.95168, 0.87936, 0.94576]]

Using the tag label, we know that the first row contains the lable of each of the provided values in the second column.

That could be useful for output that may vary in size (like if we provide evaluation of joint ranges, that can depend on the number of joints effectively studied).

Would this make sense ?

This is unclear (or maybe not sufficiently detailed) how this would be brought to the PI of the experiment, as a combination of run PI (which we only describe briefly here

aremazeilles commented 4 years ago

ping @alfonsotecnalia

alfonsotecnalia commented 4 years ago

I had forgotten the DB perspective. As this information goes also to the table describing the PI, adding a label field/column for indicating whether the row exists would mean a waste of memory (for scalar/vector otuput types this field would not make sense and would be empty). Maybe it is good what you have proposed of adding a new type. But I would call labelled_matrix instead of labelled_vector