search

iut-ibk / DynaMind-ToolBox

DynaMind-ToolBox
www.dance4water.org
GNU General Public License v2.0
9 stars 6 forks source link

Loops wrong data after branch #210

Closed christianurich closed 11 years ago

christianurich commented 11 years ago

The first one works screen shot 2013-08-22 at 10 03 46 pm

This one doesn't. It seems that the module is getting the old data again (both modules) screen shot 2013-08-22 at 10 03 28 pm

zacharias2k commented 11 years ago

can't reproduce, this one works fine https://gist.github.com/zacharias2k/6340452 i also checked single-module-successor variants, but it seems to work fine.

christianurich commented 11 years ago

I get wrong results when I add export raster data or another module to this (https://gist.github.com/christianurich/c57884bbaa3fe06d8df8) simulation screen shot 2013-08-26 at 10 12 02 pm

Conn Max should drop see here the right log

INFO     Mon Aug 26 22:07:11 2013| Conn Max  -1.6
INFO     Mon Aug 26 22:07:11 2013| Conn Min  -40
INFO     Mon Aug 26 22:07:12 2013| Successful  19 / 960
INFO     Mon Aug 26 22:07:12 2013| module ' GenerateSewerNetwork ' executed successfully (took  1669 ms)
INFO     Mon Aug 26 22:07:12 2013| running group ' Generate '
INFO     Mon Aug 26 22:07:12 2013| condition fulfilled for group ' Generate '
INFO     Mon Aug 26 22:07:12 2013| running module ' GenerateSewerNetwork '
INFO     Mon Aug 26 22:07:12 2013| 2
INFO     Mon Aug 26 22:07:12 2013| Conn Max  -0.8
INFO     Mon Aug 26 22:07:12 2013| Conn Min  -20
INFO     Mon Aug 26 22:07:13 2013| Successful  6 / 960
INFO     Mon Aug 26 22:07:13 2013| module ' GenerateSewerNetwork ' executed successfully (took  700 ms)
INFO     Mon Aug 26 22:07:13 2013| running group ' Generate '
INFO     Mon Aug 26 22:07:13 2013| condition fulfilled for group ' Generate '
INFO     Mon Aug 26 22:07:13 2013| running module ' GenerateSewerNetwork '
INFO     Mon Aug 26 22:07:13 2013| 3
INFO     Mon Aug 26 22:07:13 2013| Conn Max  -0.4
INFO     Mon Aug 26 22:07:13 2013| Conn Min  -10
INFO     Mon Aug 26 22:07:14 2013| Successful  7 / 960
INFO     Mon Aug 26 22:07:14 2013| module ' GenerateSewerNetwork ' executed successfully (took  559 ms)
INFO     Mon Aug 26 22:07:14 2013| running group ' Generate '
INFO     Mon Aug 26 22:07:14 2013| condition fulfilled for group ' Generate '
INFO     Mon Aug 26 22:07:14 2013| running module ' GenerateSewerNetwork '
INFO     Mon Aug 26 22:07:14 2013| 4
INFO     Mon Aug 26 22:07:14 2013| Conn Max  -0.2
INFO     Mon Aug 26 22:07:14 2013| Conn Min  -5
INFO     Mon Aug 26 22:07:14 2013| Successful  8 / 960
INFO     Mon Aug 26 22:07:14 2013| module ' GenerateSewerNetwork ' executed successfully (took  697 ms)
INFO     Mon Aug 26 22:07:14 2013| running group ' Generate '
INFO     Mon Aug 26 22:07:14 2013| condition fulfilled for group ' Generate '
INFO     Mon Aug 26 22:07:14 2013| running module ' GenerateSewerNetwork '
INFO     Mon Aug 26 22:07:14 2013| 5
INFO     Mon Aug 26 22:07:14 2013| Conn Max  -0.1

here the wrong log (the input data seem to be the same after 3 steps (Conn Max -0.1 when right here Conn Max -0.4) )

INFO         Mon Aug 26 22:10:23 2013| 1
INFO     Mon Aug 26 22:10:23 2013| Conn Max  -1.6
INFO     Mon Aug 26 22:10:23 2013| Conn Min  -40
INFO     Mon Aug 26 22:10:24 2013| Successful  14 / 960
INFO     Mon Aug 26 22:10:24 2013| module ' GenerateSewerNetwork ' executed successfully (took  1287 ms)
INFO     Mon Aug 26 22:10:24 2013| running module ' <class: ExportRasterData> '
INFO     Mon Aug 26 22:10:24 2013| module ' <class: ExportRasterData> ' executed successfully (took  18 ms)
INFO     Mon Aug 26 22:10:24 2013| running group ' Generate '
INFO     Mon Aug 26 22:10:24 2013| condition fulfilled for group ' Generate '
INFO     Mon Aug 26 22:10:24 2013| running module ' GenerateSewerNetwork '
INFO     Mon Aug 26 22:10:24 2013| 2
INFO     Mon Aug 26 22:10:24 2013| Conn Max  -0.8
INFO     Mon Aug 26 22:10:24 2013| Conn Min  -20
INFO     Mon Aug 26 22:10:25 2013| Successful  9 / 960
INFO     Mon Aug 26 22:10:25 2013| module ' GenerateSewerNetwork ' executed successfully (took  933 ms)
INFO     Mon Aug 26 22:10:25 2013| running module ' <class: ExportRasterData> '
INFO     Mon Aug 26 22:10:25 2013| module ' <class: ExportRasterData> ' executed successfully (took  17 ms)
INFO     Mon Aug 26 22:10:25 2013| running group ' Generate '
INFO     Mon Aug 26 22:10:25 2013| condition fulfilled for group ' Generate '
INFO     Mon Aug 26 22:10:25 2013| running module ' GenerateSewerNetwork '
INFO     Mon Aug 26 22:10:25 2013| 3
INFO     Mon Aug 26 22:10:25 2013| Conn Max  -0.4
INFO     Mon Aug 26 22:10:25 2013| Conn Min  -10
INFO     Mon Aug 26 22:10:26 2013| Successful  6 / 960
INFO     Mon Aug 26 22:10:26 2013| module ' GenerateSewerNetwork ' executed successfully (took  610 ms)
INFO     Mon Aug 26 22:10:26 2013| running module ' <class: ExportRasterData> '
INFO     Mon Aug 26 22:10:26 2013| module ' <class: ExportRasterData> ' executed successfully (took  18 ms)
INFO     Mon Aug 26 22:10:26 2013| running group ' Generate '
INFO     Mon Aug 26 22:10:26 2013| condition fulfilled for group ' Generate '
INFO     Mon Aug 26 22:10:26 2013| running module ' GenerateSewerNetwork '
INFO     Mon Aug 26 22:10:26 2013| 4
INFO     Mon Aug 26 22:10:26 2013| Conn Max  -0.4
INFO     Mon Aug 26 22:10:26 2013| Conn Min  -10
INFO     Mon Aug 26 22:10:27 2013| Successful  13 / 960
INFO     Mon Aug 26 22:10:27 2013| module ' GenerateSewerNetwork ' executed successfully (took  1074 ms)
INFO     Mon Aug 26 22:10:27 2013| running module ' <class: ExportRasterData> '
INFO     Mon Aug 26 22:10:27 2013| module ' <class: ExportRasterData> ' executed successfully (took  18 ms)
INFO     Mon Aug 26 22:10:27 2013| running group ' Generate '
INFO     Mon Aug 26 22:10:27 2013| condition fulfilled for group ' Generate '
INFO     Mon Aug 26 22:10:27 2013| running module ' GenerateSewerNetwork '
INFO     Mon Aug 26 22:10:27 2013| 5
INFO     Mon Aug 26 22:10:27 2013| Conn Max  -0.4
INFO     Mon Aug 26 22:10:27 2013| Conn Min  -10
INFO     Mon Aug 26 22:10:29 2013| Successful  16 / 960

log right

https://gist.github.com/christianurich/bf68b68c8f1544a00400

log wrong

https://gist.github.com/christianurich/132469c6c3824e638c5a

zacharias2k commented 11 years ago

"seam to be" - well lets improve that, to isolate the error. The most important thing: is the system the same? To clearify that, you may want to print out the system-address in every loop. I created a nice defien (see dmsimulationwriter.h), which converts a pointer to a human-readable int e.g. for output:

#define ADDRESS_TO_INT(x) *(int*)(&x)

in my case, the pointer is different every iteration. Just add the define and put something like

DM::Logger(Debug) << "sys address = " << ADDRESS_TO_INT(sys);
christianurich commented 11 years ago

The first 4 or 5 are fine then Con_in stays the same but it should alternate like in the 2nd example

INFO     Mon Aug 26 23:24:55 2013| 6
INFO     Mon Aug 26 23:24:55 2013| Conn Max  -0.1
INFO     Mon Aug 26 23:24:55 2013| Conn Min  -5.24786
INFO     Mon Aug 26 23:24:55 2013| Con_in    1286898672
INFO     Mon Aug 26 23:24:55 2013| Con_out    1595956912
INFO     Mon Aug 26 23:24:56 2013| Successful  11 / 960
INFO     Mon Aug 26 23:24:56 2013| module ' GenerateSewerNetwork ' executed successfully (took  991 ms)
INFO     Mon Aug 26 23:24:56 2013| running module ' <class: ExportRasterData> '
INFO     Mon Aug 26 23:24:56 2013| module ' <class: ExportRasterData> ' executed successfully (took  18 ms)
INFO     Mon Aug 26 23:24:56 2013| running group ' Generate '
INFO     Mon Aug 26 23:24:56 2013| condition fulfilled for group ' Generate '
INFO     Mon Aug 26 23:24:56 2013| running module ' GenerateSewerNetwork '
INFO     Mon Aug 26 23:24:56 2013| 7
INFO     Mon Aug 26 23:24:56 2013| Conn Max  -0.1
INFO     Mon Aug 26 23:24:56 2013| Conn Min  -5.24786
INFO     Mon Aug 26 23:24:56 2013| Con_in    1286898672
INFO     Mon Aug 26 23:24:56 2013| Con_out    1310870448
INFO     Mon Aug 26 23:24:57 2013| Successful  6 / 960
INFO     Mon Aug 26 23:24:57 2013| module ' GenerateSewerNetwork ' executed successfully (took  602 ms)
INFO     Mon Aug 26 23:24:57 2013| running module ' <class: ExportRasterData> '
INFO     Mon Aug 26 23:24:57 2013| module ' <class: ExportRasterData> ' executed successfully (took  17 ms)
INFO     Mon Aug 26 23:24:57 2013| running group ' Generate '
INFO     Mon Aug 26 23:24:57 2013| condition fulfilled for group ' Generate '
INFO     Mon Aug 26 23:24:57 2013| running module ' GenerateSewerNetwork '
INFO     Mon Aug 26 23:24:57 2013| 8
INFO     Mon Aug 26 23:24:57 2013| Conn Max  -0.1
INFO     Mon Aug 26 23:24:57 2013| Conn Min  -5.24786
INFO     Mon Aug 26 23:24:57 2013| Con_in    1286898672
INFO     Mon Aug 26 23:24:57 2013| Con_out    1318191216
INFO     Mon Aug 26 23:24:58 2013| Successful  6 / 960
INFO     Mon Aug 26 23:24:58 2013| module ' GenerateSewerNetwork ' executed successfully (took  584 ms)
INFO     Mon Aug 26 23:24:58 2013| running module ' <class: ExportRasterData> '
INFO     Mon Aug 26 23:24:58 2013| module ' <class: ExportRasterData> ' executed successfully (took  17 ms)
INFO     Mon Aug 26 23:24:58 2013| running group ' Generate '
INFO     Mon Aug 26 23:24:58 2013| condition fulfilled for group ' Generate '
INFO     Mon Aug 26 23:24:58 2013| running module ' GenerateSewerNetwork '
INFO     Mon Aug 26 23:24:58 2013| 9
INFO     Mon Aug 26 23:24:58 2013| Conn Max  -0.1
INFO     Mon Aug 26 23:24:58 2013| Conn Min  -5.24786
INFO     Mon Aug 26 23:24:58 2013| Con_in    1286898672
INFO     Mon Aug 26 23:24:58 2013| Con_out    1325555888
INFO     Mon Aug 26 23:24:58 2013| Successful  12 / 960
INFO     Mon Aug 26 23:24:58 2013| module ' GenerateSewerNetwork ' executed successfully (took  776 ms)
INFO     Mon Aug 26 23:24:58 2013| running module ' <class: ExportRasterData> '
INFO     Mon Aug 26 23:24:58 2013| module ' <class: ExportRasterData> ' executed successfully (took  18 ms)
INFO     Mon Aug 26 23:24:58 2013| running group ' Generate '
INFO     Mon Aug 26 23:24:58 2013| condition fulfilled for group ' Generate '
INFO     Mon Aug 26 23:24:58 2013| running module ' GenerateSewerNetwork '
INFO     Mon Aug 26 23:24:58 2013| 10
INFO     Mon Aug 26 23:24:58 2013| Conn Max  -0.1
INFO     Mon Aug 26 23:24:58 2013| Conn Min  -5.24786
INFO     Mon Aug 26 23:24:58 2013| Con_in    1286898672
INFO     Mon Aug 26 23:24:58 2013| Con_out    1332927408
INFO     Mon Aug 26 23:24:59 2013| Successful  8 / 960
INFO     Mon Aug 26 23:24:59 2013| module ' GenerateSewerNetwork ' executed successfully (took  630 ms)
INFO     Mon Aug 26 23:24:59 2013| running module ' <class: ExportRasterData> '
INFO     Mon Aug 26 23:24:59 2013| module ' <class: ExportRasterData> ' executed successfully (took  18 ms)
INFO     Mon Aug 26 23:24:59 2013| running group ' Generate '
INFO     Mon Aug 26 23:24:59 2013| condition fulfilled for group ' Generate '
INFO     Mon Aug 26 23:24:59 2013| running module ' GenerateSewerNetwork '

INFO     Mon Aug 26 23:26:51 2013| 10
INFO     Mon Aug 26 23:26:51 2013| Conn Max  -0.000243508
INFO     Mon Aug 26 23:26:51 2013| Conn Min  -8.02349
INFO     Mon Aug 26 23:26:51 2013| Con_in    994408112
INFO     Mon Aug 26 23:26:51 2013| Con_out    1002138672
INFO     Mon Aug 26 23:26:56 2013| Successful  152 / 960
INFO     Mon Aug 26 23:26:56 2013| module ' GenerateSewerNetwork ' executed successfully (took  5132 ms)
INFO     Mon Aug 26 23:26:56 2013| running module ' AddDataToNewView '
INFO     Mon Aug 26 23:26:56 2013| module ' AddDataToNewView ' executed successfully (took  0 ms)
INFO     Mon Aug 26 23:26:56 2013| running group ' Generate '
INFO     Mon Aug 26 23:26:56 2013| condition fulfilled for group ' Generate '
INFO     Mon Aug 26 23:26:56 2013| running module ' GenerateSewerNetwork '
INFO     Mon Aug 26 23:26:56 2013| 11
INFO     Mon Aug 26 23:26:56 2013| Conn Max  -0.00086981
INFO     Mon Aug 26 23:26:56 2013| Conn Min  -6.31673
INFO     Mon Aug 26 23:26:56 2013| Con_in    1002138672
INFO     Mon Aug 26 23:26:56 2013| Con_out    1009322960
INFO     Mon Aug 26 23:27:00 2013| Successful  125 / 960
INFO     Mon Aug 26 23:27:00 2013| module ' GenerateSewerNetwork ' executed successfully (took  4386 ms)
INFO     Mon Aug 26 23:27:00 2013| running module ' Parcels '
INFO     Mon Aug 26 23:27:00 2013| module ' Parcels ' executed successfully (took  52 ms)
INFO     Mon Aug 26 23:27:00 2013| running group ' Generate '
INFO     Mon Aug 26 23:27:00 2013| condition fulfilled for group ' Generate '
INFO     Mon Aug 26 23:27:00 2013| running module ' GenerateSewerNetwork '
INFO     Mon Aug 26 23:27:00 2013| 12
INFO     Mon Aug 26 23:27:00 2013| Conn Max  -0.00163628
INFO     Mon Aug 26 23:27:00 2013| Conn Min  -5.49925
INFO     Mon Aug 26 23:27:00 2013| Con_in    1009322960
INFO     Mon Aug 26 23:27:00 2013| Con_out    1019983600
INFO     Mon Aug 26 23:27:08 2013| Successful  332 / 960
zacharias2k commented 11 years ago

strange, groups with more than 2 inports should create a successor. i'm on it.

zacharias2k commented 11 years ago

also works perfectly: image may be related to rasterdata; how about adding a non-raster-data affacting module?

zacharias2k commented 11 years ago

I was able to reproduce the bug, made a little flowchart. I hope i'll find a solution tomorrow. Btw this only affects RasterData. image

zacharias2k commented 11 years ago

Got it: view access type "WRITE" will always create a new rasterset. So the view gets a new rasterset every iteration. getRasterData gives you the first rasterset of a map-type, which orders according to uuid -> chaotic selection of rasterset. Should we limit the number of rastersets per view to one?

christianurich commented 11 years ago

I don't have a use case at the moment where I would see more than one raster data set in a view. So for me this is fine @abroxos?

abroxos commented 11 years ago

I think we should wait until we have a real use case. I am fine as well.

Am 27.08.2013 um 02:28 schrieb Christian Urich notifications@github.com:

I don't have a use case at the moment where I would see more than one raster data set in a view. So for me this is fine @abroxos?

— Reply to this email directly or view it on GitHub.

zacharias2k commented 11 years ago

Ok i will wait. Btw a 'cleaner' solution would be to remove the 'old' rasterdata from the view (in the module).

christianurich commented 11 years ago

Is there a fix for now to guarantee that the simulation is producing the right data? I also need to debug some stuff and without being able to export raster data in between that is really hard.

zacharias2k commented 11 years ago

I will provide a solution ASAP.

zacharias2k commented 11 years ago

I spent some time understanding the bug, as it was really nasty and hard to debug The issue was related to a recent backward-compatibility fix in combination with the above mentioned view issue and the group-reset, which we discussed some time ago. The fix is a two-liner, probably also fixing a leak issue in future: win-win. https://github.com/iut-ibk/DynaMind/commit/12aa37ab1c8cb7e2142f4a621c57cf4273a37422 please try it out, especially with generate sewer networks and cellularautomata.

christianurich commented 11 years ago

seems to work. That reminds of how is the progress of the unit test framework for simulation so you can also test simulations?

christianurich commented 11 years ago

The bugfix causes a problem with an existing simulation when running.

AppendAttributes doesn't find a raster data set anymore. Worked fine before the bug fix. Simulation file is in Dropbox. DyanMind-Testdata/Models/bugraster

couldn't reproduce it with a smaller example. (don't know what makes the problem)

INFO     Wed Sep 4 15:45:36 2013| module ' <class: AddDataToNewView> ' executed successfully (took  38 ms)
INFO     Wed Sep 4 15:45:36 2013| running module ' <class: AppendAttributes> '
ERROR    Wed Sep 4 15:45:36 2013| getRasterData: rasterdata in view ' UrbanArea ' not found
ERROR    Wed Sep 4 15:45:36 2013| Raster Data  UrbanArea  not found
INFO     Wed Sep 4 15:45:36 2013| module ' <class: AppendAttributes> ' executed successfully (took  0 ms)
INFO     Wed Sep 4 15:45:36 2013| running module ' Impervious '
INFO     Wed Sep 4 15:45:37 2013| module ' Impervious ' executed successfully (took  155 ms)
INFO     Wed Sep 4 15:45:37 2013| running module ' Population '
zacharias2k commented 11 years ago 
INFO     Mi 4. Sep 12:11:54 2013| module ' <class: AddDataToNewView> ' executed successfully (took  1920 ms)
INFO     Mi 4. Sep 12:11:54 2013| running module ' <class: AppendAttributes> '
DEBUG    Mi 4. Sep 12:11:54 2013| VIEWNAME:  CATCHMENT
INFO     Mi 4. Sep 12:12:02 2013| module ' <class: AppendAttributes> ' executed successfully (took  8250 ms)
INFO     Mi 4. Sep 12:12:02 2013| running module ' Impervious '
DEBUG    Mi 4. Sep 12:12:02 2013| VIEWNAME:  CATCHMENT
INFO     Mi 4. Sep 12:12:31 2013| module ' Impervious ' executed successfully (took  29197 ms)
INFO     Mi 4. Sep 12:12:31 2013| running module ' Population '

guess it works for me?

christianurich commented 11 years ago

have you tried it a couple of times and in linux?

zacharias2k commented 11 years ago
christianurich commented 11 years ago

just tried again, debug, release, when I run it 3 times it fails 1 or 2 times.

christianurich commented 11 years ago

clean build fails as well and also the older version so it is not related to the last bug fix

zacharias2k commented 11 years ago

please make a new issue with the exact properties of the error: random, initial, after 2nd run etc