search

assisi / assisipy

Python API for the ASSISI|bf project.
Other
0 stars 3 forks source link

Deployment tool `collect_data.py`: retrieves results to same directory for all casus in a layer #59

Closed rmm-fcul closed 8 years ago

rmm-fcul commented 8 years ago

If any file retrieved has the same name, only one of the casus will have their data correctly retrieved.

I propose to introduce an additional depth of directory in the <proj>_data for each casu. This will be the same as the structure used in the sandbox created by deploy.py.

As far as I know, there are no tools that automatically assume the structure of the retrieved data, so the result should not have knock-on effects within the package.

Currently, we have:

data_dir
│
└───layer1
    │   file111.log <from casu1>
    │   casu2-file112.csv <from casu1>
    │   file111.log <from casu2> <PROBABLY OVERWROTE EXISTING FILE>
    │   casu2-file112.csv <from casu2> <no overwriting because filename included casuname>

I propose this:

data_dir
│
└───layer1
    ├───casu1
    │   │   file111.log
    │   │   file112.csv
    ├───casu2
    │   │   file111.log
    │   │   file112.csv

(see also #27 which is a bigger issue; resolving this may help in a solution for that too)