batch code to combine the intermediate `netcdf` files into one per core variable

[Thomas-Moore-Creative]

need to write batch code to combine the intermediate netcdf files into one per core variable

Originally posted by @Thomas-Moore-Creative in https://github.com/Thomas-Moore-Creative/Climatology-generator-demo/issues/9#issuecomment-2140924664

[Thomas-Moore-Creative]

"unpolished"* final netcdf's here - 2D's done & 3D's running batch

tm4888@gadi-login-03 /g/data/es60/users/thomas_moore/clim_demo_results/daily/bran2020_final_results ls -ltrh
total 6.4G
-rw-r--r-- 1 tm4888 es60 3.3G May 31 10:43 bran2020_mld_ENSO_stats.nc
-rw-r--r-- 1 tm4888 es60 3.2G May 31 10:52 bran2020_eta_t_ENSO_stats.nc

• unpolished means the chunking, format, filenames, variable names, size are final. The metadata polish is NOT.

@matt-csiro - see above.

[Thomas-Moore-Creative]

@matt-csiro the chunk(1,1,300,300) combined files are finished ( caught my bug in the u logic and fixed ).

(base) tm4888@gadi-login-04 /g/data/es60/users/thomas_moore/clim_demo_results/daily/bran2020_final_results ls -ltrh
total 530G
-rw-r--r-- 1 tm4888 es60 3.3G May 31 13:24 bran2020_mld_ENSO_stats_2024.05.31.13.20.45.nc
-rw-r--r-- 1 tm4888 es60 3.2G May 31 13:27 bran2020_eta_t_ENSO_stats_2024.05.31.13.20.45.nc
-rw-r--r-- 1 tm4888 es60 117G May 31 17:24 bran2020_temp_ENSO_stats_2024.05.31.13.20.45.nc
-rw-r--r-- 1 tm4888 es60 103G May 31 20:28 bran2020_salt_ENSO_stats_2024.05.31.13.20.45.nc
-rw-r--r-- 1 tm4888 es60 153G Jun  2 19:52 bran2020_v_ENSO_stats_2024.06.02.16.07.40.nc
-rw-r--r-- 1 tm4888 es60 152G Jun  7 23:25 bran2020_u_ENSO_stats_2024.06.07.19.43.23.nc

I'm still calling these the "final" results but will move on to make the "GIS-ready" versions.