search

CH-Earth / meshflow

Workflow to set up MESH models
GNU General Public License v3.0
1 stars 4 forks source link

Memory issue when running for longer timeframe and finer resolution sub-basin (for forcing file) #18

Open sujata91 opened 5 months ago

sujata91 commented 5 months ago

I encountered a memory issue in generating the forcing file required for MESH using the meshflow workflow. The workflow runs well for smaller timeframe and smaller number of subbasin. But there is issue with the memory with larger forcing dataset. Previously I had ~1600 subbasin and ran from 1980-2018 forcing files and had to provide like 128GB memory specs. With another geofabric (waterloo Merithydro geofabric) there are ~4000 subbasin and same time frame. Providing larger memory makes the job pending in cluster and even if it runs there is a memory issue. May be there could be some alternative solution to this in the workflow?

kasra-keshavarz commented 5 months ago

Unfortunately, I cannot think of a way, but to have MESH read multiple NetCDF files at once. The functionality exists in common hydrological modelling programs such as SUMMA and mizuRoute.