search

aodn / content

Tracks AODN Portal content and configuration issues
0 stars 0 forks source link

ACORN - Aggregating SAG wave delayed mode collection #388

Open ggalibert opened 5 years ago

ggalibert commented 5 years ago

The portal fails to aggregate the whole collection of ACORN wave data from SAG because of a change in the grid of the NetCDF files.

Until 2011-04-01 00:30:00 UTC (included) latitude is defined as follow:

 LATITUDE = -34.823425, -34.859473, -34.895522, -34.931571, -34.967619, 
    -35.003668, -35.039716, -35.075765, -35.111813, -35.147862, -35.18391, 
    -35.219959, -35.256007, -35.292056, -35.328104, -35.364153, -35.400201, 
    -35.43625, -35.472298, -35.508347, -35.544395, -35.580444, -35.616492, 
    -35.652541, -35.688589, -35.724638, -35.760686, -35.796735, -35.832783, 
    -35.868832, -35.90488, -35.940929, -35.976977, -36.013026, -36.049074, 
    -36.085123, -36.121171, -36.15722, -36.193268, -36.229317, -36.265365, 
    -36.301414, -36.337462, -36.373511, -36.40956, -36.445608, -36.481657, 
    -36.517705, -36.553754, -36.589802, -36.625851, -36.661899, -36.697948, 
    -36.733996, -36.770045, -36.806093, -36.842142, -36.87819, -36.914239, 
    -36.950287, -36.986336, -37.022384, -37.058433, -37.094481, -37.13053, 
    -37.166578, -37.202627, -37.238675, -37.274724, -37.310772, -37.346821, 
    -37.382869, -37.418918, -37.454966 ;

 LONGITUDE = 132.953957, 132.998409, 133.042861, 133.087313, 133.131765, 
    133.176218, 133.22067, 133.265122, 133.309574, 133.354026, 133.398478, 
    133.44293, 133.487383, 133.531835, 133.576287, 133.620739, 133.665191, 
    133.709643, 133.754096, 133.798548, 133.843, 133.887452, 133.931904, 
    133.976356, 134.020808, 134.065261, 134.109713, 134.154165, 134.198617, 
    134.243069, 134.287521, 134.331974, 134.376426, 134.420878, 134.46533, 
    134.509782, 134.554234, 134.598686, 134.643139, 134.687591, 134.732043, 
    134.776495, 134.820947, 134.865399, 134.909852, 134.954304, 134.998756, 
    135.043208, 135.08766, 135.132112, 135.176564, 135.221017, 135.265469, 
    135.309921, 135.354373, 135.398825, 135.443277, 135.48773, 135.532182, 
    135.576634, 135.621086, 135.665538, 135.70999, 135.754442, 135.798895, 
    135.843347, 135.887799, 135.932251, 135.976703, 136.021155, 136.065608, 
    136.11006, 136.154512, 136.198964, 136.243416, 136.287868, 136.332321, 
    136.376773, 136.421225, 136.465677, 136.510129, 136.554581, 136.599033, 
    136.643486, 136.687938, 136.73239, 136.776842, 136.821294, 136.865746, 
    136.910199, 136.954651, 136.999103, 137.043555, 137.088007, 137.132459, 
    137.176911, 137.221364, 137.265816, 137.310268, 137.35472, 137.399172, 
    137.443624 ;

But from 2011-04-01 01:30:00 UTC (included) it is:

 LATITUDE = -37.4551594, -37.4191082, -37.383057, -37.347004, -37.310955, 
    -37.274899, -37.23885, -37.202801, -37.166748, -37.130699, -37.094646, 
    -37.058594, -37.022545, -36.986492, -36.950443, -36.914391, -36.878338, 
    -36.842285, -36.806236, -36.770187, -36.734135, -36.698086, -36.662029, 
    -36.62598, -36.589931, -36.553879, -36.51783, -36.481773, -36.445724, 
    -36.409676, -36.373623, -36.337574, -36.301521, -36.265469, -36.22942, 
    -36.193367, -36.157318, -36.121265, -36.085213, -36.049164, -36.013111, 
    -35.977062, -35.94101, -35.904961, -35.868908, -35.832855, -35.796806, 
    -35.760754, -35.724705, -35.688652, -35.652599, -35.61655, -35.580498, 
    -35.544449, -35.5084, -35.472343, -35.436295, -35.400242, -35.364193, 
    -35.328144, -35.292088, -35.256039, -35.219986, -35.183937, -35.147888, 
    -35.111835, -35.075783, -35.03973, -35.003681, -34.967632, -34.93158, 
    -34.895527, -34.859474, -34.8234228 ;

 LONGITUDE = 132.953971, 132.998611, 133.043243, 133.087891, 133.132538, 
    133.177155, 133.221802, 133.266449, 133.311081, 133.355728, 133.400375, 
    133.445007, 133.489655, 133.534302, 133.578918, 133.623566, 133.668213, 
    133.712845, 133.757492, 133.802139, 133.846771, 133.891418, 133.936066, 
    133.980698, 134.02533, 134.069977, 134.114624, 134.159256, 134.203903, 
    134.24855, 134.293182, 134.33783, 134.382462, 134.427094, 134.471741, 
    134.516388, 134.56102, 134.605667, 134.650314, 134.694946, 134.739594, 
    134.784225, 134.828857, 134.873505, 134.918152, 134.962784, 135.007431, 
    135.052078, 135.09671, 135.141357, 135.185989, 135.230621, 135.275269, 
    135.319916, 135.364548, 135.409195, 135.453842, 135.498474, 135.543121, 
    135.587753, 135.632401, 135.677032, 135.72168, 135.766327, 135.810959, 
    135.855606, 135.900253, 135.944885, 135.989532, 136.034164, 136.078796, 
    136.123444, 136.168091, 136.212723, 136.25737, 136.302017, 136.346649, 
    136.391281, 136.435928, 136.48056, 136.525208, 136.569855, 136.614487, 
    136.659134, 136.703781, 136.748413, 136.79306, 136.837692, 136.882339, 
    136.926971, 136.971619, 137.016266, 137.060898, 137.105545, 137.150192, 
    137.194824, 137.239456, 137.284103, 137.328735, 137.373383, 137.418023, 
    137.462663 ;

The grid has slightly changed and LATITUDE is ordered increasingly as opposed to decreasingly before.

ggalibert commented 5 years ago

Files covering the period 2011-04-01 01:30 to 2018-09-30 (end) were produced first in 2017-12, while the files covering the period 2009-12-02 (start) to 2011-04-01 00:30 were produced in 2018-02.

The period 2011-04-01 01:30 to 2018-09-30 (end) might need to be re-processed on the same grid as the one used for the more recently produced files.

scosoli commented 5 years ago

will check what has happened with the grid. I have not changed it as far as I am aware but something may have happened when transitioning to a new processing machine. what priority has to you this modifiction? we're all quite busy until December 14

ggalibert commented 5 years ago

@scosoli no worries this can be low priority and will be kept open here until you have time to look at this.

ggalibert commented 5 years ago

@yashahoo please have a look.

yashahoo commented 5 years ago

I believe I started processing these 2017q4, so I am using Simon's more recent grid files. Once we figure out the problem I will re-process the early years with new grid.

lbesnard commented 2 years ago

@yashahoo I'm trying to follow up on this. Has any data been reprocessed on your side to fix this issue?

yashahoo commented 2 years ago

Yes, this remains on the low priority to do list as it is a substantial task and the underlying data are still good (ie there is nothing wrong with the individual files). Myself or @scosoli will complete this as soon as possible.

lbesnard commented 2 years ago

thanks a lot! FYI, I created an "acorn" label to make it easier to find issues related to your facility. Feel free to close the ones which can be: https://github.com/aodn/content/labels/acorn