search

plesager / ece3-postproc

Suite of processing tools for EC-Earth3 output
5 stars 8 forks source link

interceding update of globtable.txt #18

Open plesager opened 6 years ago

plesager commented 6 years ago

I usually submit lots of job in parallel, particularly with hiresclim and ECmean. With the later, I end up with concurrent processes writing to the same files (_globtable.txt, globtablecs.txt, and gregory.txt) at the same time. It just happened while processing only 10 experiments:

for k in {1..5}; do ./ecm.sh k_0${k} 1950 1969; done for k in {1..5}; do ./ecm.sh l_0${k} 1950 1969; done

Then I end up with a globtable.txt like this:

k_05(1950-1969) | 240.1986 | -240.0945 | 0.1040 | 166.1895 | -63.7145 | -19.7021 | -81.... k_04(1950-1969) | 240.3369 | -240.2682 | 0.0687 | 166.2643 | -63.6865 | -19.6702 | -81.... k_01(1950-1969) | 240.2997 | -240.2099 | 0.0898 | 166.2469 | -63.7230 | -19.6938 | -81.... k_02(1950-1969) | 240.2285 | -240.1433 | 0.0852 | 166.1837 | -63.6802 | -19.6335 | -81.... l_01(1950-1969) | 240.4330 | -240.3248 | 0.1082 | 166.3347 | -63.7030 | -19.7245 | -81.... 286.471 | 0.652921 | 0.353735 | 0.265122 | 0.406015 | 2.80747 | -0.0168055 |
k_03(1950-1969) | 240.2991 | -240.2160 | 0.0830 | 166.2451 | -63.7142 | -19.6958 | -81.... l_02(1950-1969) | 240.2646 | -240.1316 | 0.1330 | 166.2470 | -63.7446 | -19.6784 | -81.... l_04(1950-1969) | 240.5139 | -240.3645 | 0.1494 | 166.4253 | -63.7333 | -19.6941 | -81.... l_03(1950-1969) | 240.4386 | -240.2808 | 0.1578 | 166.3905 | -63.7723 | -19.6488 | -81....

The line for experiment l_05 is truncated. Not shown, the line for l_01 ends with some garbage. I was aware of that issue (not the first time) and had put a TODO in the EC-mean.sh code. So far my solution is to repeat the calls to tab2lin.sh on the command line:

for k in {1..5}; do $PIDIR/tab2lin.sh k_0${k} 1950 1969 >> ./globtable.txt; done for k in {1..5}; do $PIDIR/tab2lin.sh l_0${k} 1950 1969 >> ./globtable.txt; done

Then I have a clean global table, and the experiments order is the same as the calls order:

k_01(1950-1969) | 240.2997 | -240.2099 | 0.0898 | 166.2469 | -63.7230 | -19.6938 | -81.... k_02(1950-1969) | 240.2285 | -240.1433 | 0.0852 | 166.1837 | -63.6802 | -19.6335 | -81.... k_03(1950-1969) | 240.2991 | -240.2160 | 0.0830 | 166.2451 | -63.7142 | -19.6958 | -81.... k_04(1950-1969) | 240.3369 | -240.2682 | 0.0687 | 166.2643 | -63.6865 | -19.6702 | -81.... k_05(1950-1969) | 240.1986 | -240.0945 | 0.1040 | 166.1895 | -63.7145 | -19.7021 | -81.... l_01(1950-1969) | 240.4330 | -240.3248 | 0.1082 | 166.3347 | -63.7030 | -19.7245 | -81.... l_02(1950-1969) | 240.2646 | -240.1316 | 0.1330 | 166.2470 | -63.7446 | -19.6784 | -81.... l_03(1950-1969) | 240.4386 | -240.2808 | 0.1578 | 166.3905 | -63.7723 | -19.6488 | -81.... l_04(1950-1969) | 240.5139 | -240.3645 | 0.1494 | 166.4253 | -63.7333 | -19.6941 | -81.... l_05(1950-1969) | 240.3392 | -240.2419 | 0.0974 | 166.2819 | -63.7026 | -19.6153 | -81....

There are at least two ways to address this issue:

  1. using flock (but there may be some issue over NFS file systems, which some of us use)
  2. using a different work flow (essentially separate the parsing done by tab2lin.sh from EC-mean.sh)