aremazeilles / beat_routine

Performance Indicators developed by BEAT consortium for the Eurobench project
0 stars 2 forks source link

adding reference data, and adjust code #4

Closed aremazeilles closed 4 years ago

aremazeilles commented 4 years ago

Fix #2

aremazeilles commented 4 years ago

ping @juritaborri

aremazeilles commented 4 years ago

Looking at the output you generate. All files should have an extension. I can work on that if you want. But the format of a PI output should be a yaml file, as indicated in the documentation: here and there

In the octave example you can see how we store a vector to a yaml file structure.

Matlab / Octave cells are not output format accepted. In your case it should be a matrix type, and then you should provide the matrix in such format:

type: 'matrix'
value: [["label1", "label2", "label3", "label4", "label5", "label6"], [0.96867, 1.01667, 0.98843, 0.95168, 0.87936, 0.94576]]

If you indicate me (file, line) where you do such storage, I can help you with that.

aremazeilles commented 4 years ago

@alfonsotecnalia here we are in the case where storing labels with the vales would be needed. Should we define a specific output? It sounds like a labelled vector. We could think of something like:

type: 'labelelled_vector'
label: ["label1", "label2", "label3", "label4", "label5", "label6"]
value: [0.96867, 1.01667, 0.98843, 0.95168, 0.87936, 0.94576]

What do you think?

juritaborri commented 4 years ago

Looking at the output you generate. All files should have an extension. I can work on that if you want. But the format of a PI output should be a yaml file, as indicated in the documentation: here and there

In the octave example you can see how we store a vector to a yaml file structure.

Matlab / Octave cells are not output format accepted. In your case it should be a matrix type, and then you should provide the matrix in such format:

type: 'matrix'
value: [["label1", "label2", "label3", "label4", "label5", "label6"], [0.96867, 1.01667, 0.98843, 0.95168, 0.87936, 0.94576]]

If you indicate me (file, line) where you do such storage, I can help you with that.

Ok and sorry for that. The creation of cell is prensent in:

Let me know if you prefer adjusting code or I will performing it

aremazeilles commented 4 years ago

If you could do it, better. If you prefer me to do a first example, and then let you reproduce it, I would need the command to launch, with the input files, and associate it to the expected outcome files.

juritaborri commented 4 years ago

I will do it. Tomorrow I can work on it, today institutional meetings!

aremazeilles commented 4 years ago

Looking at the output PIs. file delta_theta,delta_theta in protocol6 is questionnable. First of all, I would avoid using comma in filename. Then it is particularly huge. 22Mo. 10 time bigger than the input file. Are you sure this corresponds to a PI output, ie a score than can be looked at by a person to compare several experiments?

aremazeilles commented 4 years ago

I will do it. Tomorrow I can work on it, today institutional meetings!

Sure, no hurry

juritaborri commented 4 years ago

Looking at the output PIs. file delta_theta,delta_theta in protocol6 is questionnable. First of all, I would avoid using comma in filename. Then it is particularly huge. 22Mo. 10 time bigger than the input file. Are you sure this corresponds to a PI output, ie a score than can be looked at by a person to compare several experiments?

There was a mistake in the processing. I will update the right file

alfonsotecnalia commented 4 years ago

@alfonsotecnalia here we are in the case where storing labels with the vales would be needed. Should we define a specific output? It sounds like a labelled vector. We could think of something like:

type: 'labelelled_vector'
label: ["label1", "label2", "label3", "label4", "label5", "label6"]
value: [0.96867, 1.01667, 0.98843, 0.95168, 0.87936, 0.94576]

What do you think?

I think that no new type is necessary. For me this is a matrix (or dataframe) and the first row contains labels. But maybe we need to introduce an optional (boolean) field "label" to indicate whether the matrix contains a row with labels. If no "label" field is provided, there is no such a row (all the rows are data). This field will serve to avoid not-allowed operations in intra/inter run aggregations (if it is specified to apply a mean to row 0 and it has the label field to True, then we can detect and inform that this operation is not possible or calculate the mean in the first data row).

aremazeilles commented 4 years ago

OK Alfonso so your suggestion is to stick to the current type - value fields for the output file. And eventually add a tag for label (assuming if not present that no label is associated to it.

My only concern is to know how we should know the appropriate way to present (visually, on the front end) a given output. But maybe we should bring such discussion on other places?

alfonsotecnalia commented 4 years ago

I think so

juritaborri commented 4 years ago

If you could do it, better. If you prefer me to do a first example, and then let you reproduce it, I would need the command to launch, with the input files, and associate it to the expected outcome files.

I have updload all the .yaml file. Please let me know if are ok, specially regarding the one with labels

aremazeilles commented 4 years ago

I would propose to merge this, and then start working on adjusting the program. As far as I can see, you are not yet generating yaml file, which should be the next operations, associated to the main program entry point adjustment.

Do you agree?

juritaborri commented 4 years ago

I would propose to merge this, and then start working on adjusting the program. As far as I can see, you are not yet generating yaml file, which should be the next operations, associated to the main program entry point adjustment.

Do you agree?

I do not know if there have been problems with the upload, but next week I have added the folder output with all the PIs in .yaml format, as well as modified the function by adding the code for saving PIs in .yaml. Can't you find it?

aremazeilles commented 4 years ago

I am in branch input_output, what I see here:

anthony@PRT0962A ~/code/eurobench/beat_routine/output ‹input-output› 
╰─$ tree 
.
├── Protocol1
│   ├── CoV
│   ├── mROM
│   └── NoS
├── Protocol2
│   ├── CoV
│   ├── EA
│   ├── mROM
│   ├── NoS
│   ├── PL
│   ├── PL_AP
│   └── PL_ML
├── Protocol3
├── Protocol4
│   ├── EA
│   ├── PL
│   ├── PL_AP
│   └── PL_ML
├── Protocol5
│   ├── EA_p
│   ├── PL_p
│   └── ROM_p
├── Protocol6
│   ├── delta_theta
│   ├── EA_p
│   ├── OS
│   ├── PL_p
│   ├── ROM_p
│   └── theta_f
└── Protocol7
    ├── EA_p
    ├── G
    ├── phi
    └── PL_p

7 directories, 27 files
aremazeilles commented 4 years ago

Ok, you did it with the master branch apparently. It would have been appropriate to push it the branch input-output. Do you know how you hsould have done it?

aremazeilles commented 4 years ago

Now I am not sure whether it makes sense to merge that branch. Could you check whether the changes in the .m files from that branch are still relevant? To do so, click on Files Changes above

aremazeilles commented 4 years ago

Calm down :)

aremazeilles commented 4 years ago

Can we have a quick phone call?

aremazeilles commented 4 years ago

I would rather have a quick phone call, this is getting messy. By the way you should be able to delete several files in a single commit. You are committing each file deletion (which generates me an email every time...)

aremazeilles commented 4 years ago

As we discussed @juritaborri , I would remove that branch, right ? The input-output information is already in master branch, correct?

juritaborri commented 4 years ago

Yes, right