search

kdrushka / oceanliner_backup

oceanliner: observing system simulation experiments (OSSEs) to subsample high-resolution model output as if by gliders, ships, or other in situ platforms
MIT License
7 stars 4 forks source link

optimize performance #23

Open kdrushka opened 2 years ago

kdrushka commented 2 years ago
kdrushka commented 2 years ago

according to the dask client, "concat" is taking a huge amount of memory. the only concat command is combining ds and dsd (ie, adding the derived data). try just loading ds (skip dsd for now) and see if that speeds things up. image => still slow

kdrushka commented 2 years ago

try xoak, which uses dask already? => xoak doesn't seem to work well. Depenging on which interpolator is used, (a) it crashes (even for only 2 days of data), or (b) it gives a nearest-neighbor answer that isn't accurate.

kdrushka commented 2 years ago

Try just integrating to a less fine grid than 1m - eg 5m or the model vertical grid

kdrushka commented 2 years ago

dask tests: 31 days {'zrange': [-1, -1000.0], 'hspeed': 0.25, 'vspeed': 0.1, 'AT_END': 'reverse', 'PATTERN': 'lawnmower', 'SAMPLING_STRATEGY': 'sim_glider', 'distance_total_km': 664.3325, 'time_total_s': 2657330.0}

tchunk = 12 xchunk = 200 ychunk = 150

186 chunks x 221.48 MiB CPU times: user 46.4 s, sys: 6.22 s, total: 52.7 s Wall time: 7min 16s

kdrushka commented 2 years ago

tchunk = 24 xchunk = 50 ychunk = 50

868 x 38.45 MiB CPU times: user 38.5 s, sys: 4.39 s, total: 42.9 s Wall time: 8min 58s

kdrushka commented 2 years ago

tchunk = 6 xchunk = 200 ychunk = 200 248 x 147.66 MiB Wall time: 3min 58s

kdrushka commented 2 years ago

tchunk = 3 xchunk = 250 ychunk = 250 496 = 92MB Wall time: 4min 48s

kdrushka commented 2 years ago

tchunk = 6 xchunk = 250 ychunk = 250 248 x 184.57MB Wall time: 3min 57s

90 days crashed

kdrushka commented 2 years ago

tchunk = 12 xchunk = 250 ychunk = 250 124 x 369MB Wall time: 3min 54s

kdrushka commented 2 years ago

tchunk = 24 xchunk = 250 ychunk = 250 162 x 738MB Wall time: 4min => crashes for 90 days

kdrushka commented 2 years ago

Try not loading the derived variables - where is where the concat comes from (?)