search

IHCantabria / NEOPRENE

Neyman-Scott Process Rainfall Emulator library
GNU General Public License v3.0
13 stars 6 forks source link

STNSRP model taking up huge RAM space and long computational time #21

Closed sabahparvaze closed 1 year ago

sabahparvaze commented 2 years ago

I used the STNSRP model for my basin, having 18 rainfall stations and an area of 2000 sq km. However, the model is taking over 24 hours to run for simulating 100 years' data. It is also occupying a lot of RAM (12 gb). How can I improve the computational time and memory requirement?

JavierDiezSierra commented 2 years ago

@sabahparvaze, are you using the latest release updated during July?

sabahparvaze commented 1 year ago

YESyes

sabahparvaze commented 1 year ago

Yes

JavierDiezSierra commented 1 year ago

The latest release includes some updates to improve the speed of the model. Did this help to solve your problem?

sabahparvaze commented 1 year ago

Thank you for replying. I am working with the latest version available, but the problem is still the same. Also, the spatial model critically underestimates the rainfall amount.

On Wed, 30 Nov 2022 at 13:55, Javier Diez-Sierra @.***> wrote:

The latest release includes some updates which improve the speed of the model. Did this help to solve your problem?

— Reply to this email directly, view it on GitHub https://github.com/IHCantabria/NEOPRENE/issues/21#issuecomment-1331796117, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARVKNRUTYMT2GQ5HRG2YN33WK4FOHANCNFSM6AAAAAAQDX4G4E . You are receiving this because you modified the open/close state.Message ID: @.***>

manueldeljesus commented 1 year ago

May you please share your configuration files to check if the problem may come from there?

sabahparvaze commented 1 year ago

I am sending the files I am using. Please see the attachment.

On Fri, 2 Dec 2022 at 17:18, Manuel @.***> wrote:

May you please share your configuration files to check if the problem may come from there?

— Reply to this email directly, view it on GitHub https://github.com/IHCantabria/NEOPRENE/issues/21#issuecomment-1335125281, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARVKNRT6TL45UB5XFGJX4ILWLHOZTANCNFSM6AAAAAAQDX4G4E . You are receiving this because you modified the open/close state.Message ID: @.***>

manueldeljesus commented 1 year ago

You did not attach any files. You should go to the GitHub page and upload the files there, in the text area. There is some help around the page explaining how to upload files to the interface.

sabahparvaze commented 1 year ago

There was some problem previously. Please see attachment SNSRP.zip

JavierDiezSierra commented 1 year ago

@sabahparvaze, the configuration files available in the repo are defined for the different rainfall regimens available in Spain, where frontal and convective precipitation take place. Rainfall in the India is completely different mainly dominated by the monsoon.

For your study case, the limits of the hyperparameters have to be modified (see attached files SNSRP.zip), specially the parameters cell_duration and number_storm_cells. Note that you should try other different configuration to improve the results which depend on your objectives (e.g. for extreme or water resource analysis).

To reduce the computational time, it is very important reduce the number_storm_cells. Note that simulations time is close related to the number of cell per storm (more cells more time consuming).

In addition, as you will be able to verify with the attached files, the average precipitation obtained for all the simulated series is practically the same as in the observed ones (it is also demonstrate in the exceedance probability Figure uploaded below). However, other configurations (probably reduce the size of the cell radius) should be investigate in order to reproduce the large precipitation gradient existing between the observed series.

Finally, I would like to comment that the optimal set of hyperparameters can be difficult to find it it is your first time time using NEOPRENE. If you are not familiarized with the physical description of the model it can take a time.

Do not hesitate to ask to us any other questions.

Exceedence_probability

sabahparvaze commented 1 year ago

Thank you very much.

On Tue, 6 Dec 2022 at 22:32, Javier Diez-Sierra @.***> wrote:

@sabahparvaze https://github.com/sabahparvaze, the configuration files available in the repo are defined for the different rainfall regimens available in Spain, where frontal and convective precipitation take place. Rainfall in the India is completely different mainly dominated by the monsoon.

For your study case, the limits of the hyperparameters have to be modified (see attached files SNSRP.zip https://github.com/IHCantabria/NEOPRENE/files/10168632/SNSRP.zip), specially the parameters cell_duration and number_storm_cells. Note that you should try other different configuration to improve the results which depend on your objectives (e.g. for extreme or water resource analysis).

To reduce the computational time, it is very important reduce the number_storm_cells. Note that simulations time is close related to the number of cell per storm (more cells more time consuming).

In addition, as you will be able to verify with the attached files, the average precipitation obtained for all the simulated series is practically the same as in the observed ones (it is also demonstrate in the exceedance probability figure). However, other configurations (probably reduce the size of the cell radius) should be investigate in order to reproduce the large precipitation gradient existing between the observed series.

Finally, I would like to comment that the optimal set of hyperparameters can be difficult to find the first time and it can take a time if you are not familiarized with the physical description of the model.

Do not hesitate to ask nay other questions.

— Reply to this email directly, view it on GitHub https://github.com/IHCantabria/NEOPRENE/issues/21#issuecomment-1339684354, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARVKNRT4BMEU3GYCKD55HYLWL5WRBANCNFSM6AAAAAAQDX4G4E . You are receiving this because you were mentioned.Message ID: @.***>