[wip] test output modification

[djarecka]

• allowing for test output to have "2d structure", i.e. {"key_n": list_n}, one of the key has to be col_names

• checking if results from different tests can be merged: assuming that you can merge tests that either have flat structure or have the same col_names

in addition:

• allowing for an empty list in plots["var_list"] in parameter file, using some default choice in each altair plot

TODO:

• [x] updating simple workflow

• [ ] using a wild card in files names

• [x] proper merging for multiple tests (that have multiple files)

[djarecka]

I haven't fix the wild card issue, but I will merge this PR, want to test other things now. But now we can have tests that take multiple files and can return {key: list} and the results from various tests are properly merged and presented on parallel plot, for my fake-toy example you can see dashboard here.

Unfortunately I still don't know why in case of simple workflow (with all tests returning 0) the plot doesn't look good, see here

[satra]

@djarecka - can you point me to the csv file you are generating for simple workflow? also did you generate the osx version for simple workflow?

[satra]

i'm running it now on osx

[djarecka]

@satra - I'm testing a different expected output in a different PR (yesterday it was failing due to some problems on CircleCI, but I'd think that should pass now): https://github.com/djarecka/regtests/pull/13

[djarecka]

oh, I actually temporary created a PR to my fork, that's why you couldn't see it

[djarecka]

@satra -and the csv: https://200-111057450-gh.circle-artifacts.com/0/home/circleci/regtests/simple_workflow_cwl/output_all.csv

[satra]

i see the same here: http://syntagmatic.github.io/parallel-coordinates/examples/upload.html

so it must be something to do with a single value that is 0.

[djarecka]

yes, I only see this problem when I have zeros only, as you could se for the fake workflow it looks good. I don't know if this would be the same if the single value is 1 (i.e. is it a 0-issue or a single-value-issue).

The example with zeros only is not very exiting, but in our framework we should handle this (hopefully it happens often)

[djarecka]

I actually can easily check if it's ok for a single value that is != 0, and it looks fine when I put ones instead of zeros.