ItziModel / itzi

A dynamic GIS flood model
GNU General Public License v2.0
37 stars 17 forks source link

Itzi-Dev extremely slow and not using full system ressources #108

Open JSUNFGH opened 1 year ago

JSUNFGH commented 1 year ago

Hello, after I didn't get Itzi 20.5 master realease to work (see issue #107), I tried to get Itzi 20.5 dev branch to work (https://github.com/ItziModel/itzi/tree/dev). I downloaded the code and tried to build/install locally via sudo python3 setup.py build and sudo python3 setup.py install. It gave me a few warnings about not having installed Cython and a newer version of aenum which I corrected via sudo pip3 install Cythonand sudo pip install aenum==3.1.11, but in the end it built and installed successfully on Linux Mint 21.2 with Grass Gis 8.2.1 as well as Ubuntu 22.04 LTS and Grass Gis 7.8.7, both with Python 3.10.12. I suppose it's got something to do with the dev branch being used with Cython as compared to the master branch being used with ... (I can't tell)? That supports my suspicion that master branch doesn't build successfully because of incompatibilities in the older code (see issue #107).

The problem occured as I was using Itzi with Grass Gis. The simulation starts without problems. The CPU then only uses around 10-15% on all cores/threads and 38% of RAM on an AMD Ryzen 9 5950X 16-Core Processor with 67 GB of RAM. After a few days of running the simulation, being 1/6th into the simulation time of 16:00 to 18:00, it still says ETA: 68+ days (see screenshot under https://ibb.co/yfvBxJd)! That is too slow and not normal I think. The model area is 22,708,107 cells of 3x3 meters, running over 2 hours simulation time. I used to run simulations like these with the master branch version in under 2 days with a CPU and RAM load of 80-90%. The strange thing is: CPU/RAM load varies with model area. I used a smaller area for further testing (maybe 1/10th) and CPU/RAM load rises to around 40%. Very strange.

So, I'm asking myself, why is it so slow and not using the full potential. Has it something to do with the new Cython code limiting the use of system ressources or is the dev branch very buggy and not fit for operational use? Does Itzi prefer less cores at a higher speed or more power from the graphics card? @lrntct Can you please help me out on this or the other issue (#107)? I need Itzi for an ongoing project. Thank you.

JSUNFGH commented 1 year ago

One more thing I tried to speed up the dev version: I reduced the cores used by the computer in the BIOS settings from 16 cores to only 4 cores. This made Itzi use around 40% of the CPU power, tripling the CPU load compared to the former setting. But it's still painfully slow compared to the master version which I got working on Ubuntu 18.04 LTS (see #107).

neteler commented 5 months ago

Concerning GRASS GIS 8, maybe the g.region hint in #103 matters (raster resolution related).