UMEP-dev / UMEP

Urban Multi-scale Environmental Predictor
https://umep-docs.readthedocs.io/
61 stars 15 forks source link

Urban Wind Field: URock v2023a - The linestring must be closed. #622

Open nicolacolaninno opened 4 months ago

nicolacolaninno commented 4 months ago

QGIS version: 3.36.2-Maidenhead QGIS code revision: 6d250527 Qt version: 5.15.13 Python version: 3.12.3 GDAL version: 3.8.5 GEOS version: 3.12.1-CAPI-1.18.1 PROJ version: Rel. 9.4.0, March 1st, 2024 PDAL version: 2.6.3 (git-version: 816ce1) Algorithm started at: 2024-06-02T16:06:59 Algorithm 'Urban Wind Field: URock v2023a' starting… Input parameters: { 'ATTENUATION_FIELD' : '', 'BUILDINGS' : 'C:/Users/nicol/OneDrive - Politecnico di Milano/Documenti/MultiCAST/_DATA/Barcelona/Paper_update_22122023/3_BCN/New_Inputs_UMEP/WInd_Test/Buildings_urock_CLIP1.shp', 'HEIGHT_FIELD_BUILD' : 'ROOF_HEIGH', 'HORIZONTAL_RESOLUTION' : 2, 'INPUT_PROFILE_FILE' : '', 'INPUT_PROFILE_TYPE' : 0, 'INPUT_WIND_DIRECTION' : 45, 'INPUT_WIND_HEIGHT' : 10, 'INPUT_WIND_SPEED' : 2, 'LOAD_OUTPUT' : True, 'OUTPUT_FILENAME' : 'urock_output', 'RASTER_OUTPUT' : None, 'SAVE_NETCDF' : True, 'SAVE_RASTER' : True, 'SAVE_VECTOR' : True, 'UROCK_OUTPUT' : 'C:\Users\nicol\OneDrive - Politecnico di Milano\Documenti\MultiCAST\_DATA\Barcelona\Paper_update_22122023\3_BCN\New_Inputs_UMEP\WInd_Test\WInd_urock_out_res_2m', 'VEGETATION' : 'C:\Users\nicol\OneDrive - Politecnico di Milano\Documenti\MultiCAST\_DATA\Barcelona\Paper_update_22122023\3_BCN\New_Inputs_UMEP\WInd_Test\Vegetation_urock_CLIP1_multipart.shp', 'VEGETATION_CROWN_BASE_HEIGHT' : '', 'VEGETATION_CROWN_TOP_HEIGHT' : 'VEG_HEIGHT', 'VERTICAL_RESOLUTION' : 2, 'WIND_HEIGHT' : '1.5' }

Writing settings for this model run to specified output folder (Filename: RunInfoURock_YYYY_DOY_HHMM.txt) Initiating algorithm Creates an H2GIS Instance and load data Creates the stacked blocks used as obstacles Rotates obstacles to the right direction and calculates geometry properties Creates the 2D Röckle zones Traceback (most recent call last): File "JdbcPreparedStatement.java", line 237, in org.h2.jdbc.JdbcPreparedStatement.execute java.lang.java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: The linestring must be closed.

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "JdbcPreparedStatement.java", line 237, in org.h2.jdbc.JdbcPreparedStatement.execute Exception: Java Exception

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "C:\OSGeo4W\apps\Python312\Lib\site-packages\jaydebeapi__init__.py", line 534, in execute is_rs = self._prep.execute() ^^^^^^^^^^^^^^^^^^^^ org.h2.jdbc.org.h2.jdbc.JdbcSQLNonTransientException: org.h2.jdbc.JdbcSQLNonTransientException: Exception calling user-defined function: "makePolygon(LINESTRING (429572.34 4584029.23, 429572.27 4584029.09, 429572.2 4584028.95, 429572.13 4584028.81, 429572.06 4584028.67, 429571.99 4584028.53, 429571.92 4584028.4, ... , 429572.2 4584037.92, 429572.27 4584037.99, 429572.34 4584038.06, 429488.53 4584052.44)): The linestring must be closed."; SQL statement:

CREATE TABLE CAVITY_ZONE_POLYGONS_20240602160725 AS SELECT ST_MAKEVALID(ST_MAKEPOLYGON(ST_MAKELINE(ST_ACCUM(ST_PRECISIONREDUCER(THE_GEOM,2))))) AS THE_GEOM, DOWNWIND_FACADE_ID FROM CAVITY_ZONE_POINTS_20240602160725 GROUP BY DOWNWIND_FACADE_ID; [90105-200]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users/nicol/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\processor\urock_processing_algorithm.py", line 447, in processAlgorithm MainCalculation.main(javaEnvironmentPath = javaEnvVar, File "C:\Users/nicol/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\functions\URock\MainCalculation.py", line 290, in main Zones.cavityAndWakeZones(cursor = cursor, File "C:\Users/nicol/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\functions\URock\Zones.py", line 250, in cavityAndWakeZones cursor.execute(";".join([ File "C:\OSGeo4W\apps\Python312\Lib\site-packages\jaydebeapi__init.py", line 536, in execute _handle_sql_exception() File "C:\OSGeo4W\apps\Python312\Lib\site-packages\jaydebeapi__init__.py", line 165, in _handle_sql_exception_jpype reraise(exc_type, exc_info[1], exc_info[2]) File "C:\OSGeo4W\apps\Python312\Lib\site-packages\jaydebeapi__init__.py", line 57, in reraise raise value.with_traceback(tb) File "C:\OSGeo4W\apps\Python312\Lib\site-packages\jaydebeapi\init__.py", line 534, in execute is_rs = self._prep.execute() ^^^^^^^^^^^^^^^^^^^^ jaydebeapi.DatabaseError: org.h2.jdbc.JdbcSQLNonTransientException: Exception calling user-defined function: "makePolygon(LINESTRING (429572.34 4584029.23, 429572.27 4584029.09, 429572.2 4584028.95, 429572.13 4584028.81, 429572.06 4584028.67, 429571.99 4584028.53, 429571.92 4584028.4, 429571.85 4584028.26, ... ,429572.2 4584037.92, 429572.27 4584037.99, 429572.34 4584038.06, 429488.53 4584052.44)): The linestring must be closed."; SQL statement:

CREATE TABLE CAVITY_ZONE_POLYGONS_20240602160725 AS SELECT ST_MAKEVALID(ST_MAKEPOLYGON(ST_MAKELINE(ST_ACCUM(ST_PRECISIONREDUCER(THE_GEOM,2))))) AS THE_GEOM, DOWNWIND_FACADE_ID FROM CAVITY_ZONE_POINTS_20240602160725 GROUP BY DOWNWIND_FACADE_ID; [90105-200]

Execution failed after 28.90 seconds

Loading resulting layers Algorithm 'Urban Wind Field: URock v2023a' finished

j3r3m1 commented 4 months ago

Hi, can you share your building dataset ?

nicolacolaninno commented 4 months ago

Hi, thanks for replying. Sure, here is the dataset I used. Wind_Test_uRock.zip

Based on these files, I first used the pre-processing, and the uRock prepare, and I got these files. uRock_Vegetation_Buildings.zip

Using the latter files I tried to simulate wind, but I got that error.

Thanks in advance!

j3r3m1 commented 4 months ago

Thanks for your prompt reply. I look it up

nicolacolaninno commented 4 months ago

Thanks to you for your time!

j3r3m1 commented 4 months ago

I had just a check to your file, I strongly recommend you to use a real vector footprint as input for your building. Your derive from raster pixels and is definitely not appropriate for URock. It will create many artefact zones that might affect the performance and accuracy of the model. As building footprint (in the URock preprocessing), you can for example easily download the ones from OSM using GeoClimate I have attached the configuration file used for GeoClimate and the data once executed:

nicolacolaninno commented 4 months ago

Hey, Jeremy, thanks a lot for your help. It is not clear to me if I should use the OSM buildings in the prepare or the processing. When I use OSM buildings in prepare I get the following error:

Loading resulting layers Algorithm 'Urban Wind Field: URock Prepare' finished QGIS version: 3.34.6-Prizren QGIS code revision: 623828f5 Qt version: 5.15.13 Python version: 3.12.3 GDAL version: 3.8.5 GEOS version: 3.12.1-CAPI-1.18.1 PROJ version: Rel. 9.4.0, March 1st, 2024 PDAL version: 2.6.3 (git-version: 816ce1) Algorithm started at: 2024-06-03T10:51:19 Algorithm 'Urban Wind Field: URock Prepare' starting… Input parameters: { 'BUILDINGS_WITH_HEIGHT' : 'C:/Users/nicol/OneDrive - Politecnico di Milano/Documenti/MultiCAST/_DATA/Barcelona/Paper_update_22122023/3_BCN/New_Inputs_UMEP/WInd_Test/uRock_Buildings_2.shp', 'HEIGHT_VEG_FIELD' : '', 'INPUT_BUILD_DEM' : 'C:/Users/nicol/OneDrive - Politecnico di Milano/Documenti/MultiCAST/_DATA/Barcelona/Paper_update_22122023/3_BCN/New_Inputs_UMEP/WInd_Test/Gracia_Ground_CLIP.tif', 'INPUT_BUILD_DSM' : 'C:/Users/nicol/OneDrive - Politecnico di Milano/Documenti/MultiCAST/_DATA/Barcelona/Paper_update_22122023/3_BCN/New_Inputs_UMEP/WInd_Test/Gracia_Buildings_CLIP.tif', 'INPUT_BUILD_FOOTPRINT' : 'C:/Users/nicol/OneDrive - Politecnico di Milano/Documenti/MultiCAST/_DATA/Barcelona/Paper_update_22122023/3_BCN/New_Inputs_UMEP/WInd_Test/osm_41.402157_2.152937_41.406701_2.159071/building.fgb|layername=unknown|geometrytype=Polygon|uniqueGeometryType=yes', 'INPUT_VEG_CDSM' : 'C:/Users/nicol/OneDrive - Politecnico di Milano/Documenti/MultiCAST/_DATA/Barcelona/Paper_update_22122023/3_BCN/New_Inputs_UMEP/WInd_Test/Gracia_Trees_CLIP.tif', 'INPUT_VEG_POINTS' : None, 'OUTPUT_BUILD_HEIGHT_FIELD' : 'ROOF_HEIGHT', 'OUTPUT_VEG_HEIGHT_FIELD' : 'VEG_HEIGHT', 'RADIUS_VEG_FIELD' : '', 'VEGETATION_ASPECT' : '0.75', 'VEGETATION_WITH_HEIGHT' : 'C:/Users/nicol/OneDrive - Politecnico di Milano/Documenti/MultiCAST/_DATA/Barcelona/Paper_update_22122023/3_BCN/New_Inputs_UMEP/WInd_Test/uRock_Vegetation_2.shp' }

Output format is invalid Execution failed after 0.03 seconds

Loading resulting layers Algorithm 'Urban Wind Field: URock Prepare' finished

If I use the OSM building in the processing for wind simulation, I get the following error:

QGIS version: 3.34.6-Prizren QGIS code revision: 623828f5 Qt version: 5.15.13 Python version: 3.12.3 GDAL version: 3.8.5 GEOS version: 3.12.1-CAPI-1.18.1 PROJ version: Rel. 9.4.0, March 1st, 2024 PDAL version: 2.6.3 (git-version: 816ce1) Algorithm started at: 2024-06-03T10:53:46 Algorithm 'Urban Wind Field: URock v2023a' starting… Input parameters: { 'ATTENUATION_FIELD' : '', 'BUILDINGS' : 'C:/Users/nicol/OneDrive - Politecnico di Milano/Documenti/MultiCAST/_DATA/Barcelona/Paper_update_22122023/3_BCN/New_Inputs_UMEP/WInd_Test/osm_41.402157_2.152937_41.406701_2.159071/building.fgb|layername=unknown|geometrytype=Polygon|uniqueGeometryType=yes', 'HEIGHT_FIELD_BUILD' : 'HEIGHT_ROOF', 'HORIZONTAL_RESOLUTION' : 2, 'INPUT_PROFILE_FILE' : '', 'INPUT_PROFILE_TYPE' : 0, 'INPUT_WIND_DIRECTION' : 45, 'INPUT_WIND_HEIGHT' : 10, 'INPUT_WIND_SPEED' : 2, 'LOAD_OUTPUT' : True, 'OUTPUT_FILENAME' : 'urock_output', 'RASTER_OUTPUT' : None, 'SAVE_NETCDF' : True, 'SAVE_RASTER' : True, 'SAVE_VECTOR' : True, 'UROCK_OUTPUT' : 'C:\Users\nicol\OneDrive - Politecnico di Milano\Documenti\MultiCAST\_DATA\Barcelona\Paper_update_22122023\3_BCN\New_Inputs_UMEP\WInd_Test\WInd_urock_out_res_2m', 'VEGETATION' : 'C:/Users/nicol/OneDrive - Politecnico di Milano/Documenti/MultiCAST/_DATA/Barcelona/Paper_update_22122023/3_BCN/New_Inputs_UMEP/WInd_Test/uRock_Vegetation.shp', 'VEGETATION_CROWN_BASE_HEIGHT' : '', 'VEGETATION_CROWN_TOP_HEIGHT' : 'VEG_HEIGHT', 'VERTICAL_RESOLUTION' : 2, 'WIND_HEIGHT' : '1.5' }

Coordinate system of input building layer and vegetation layer differ! Writing settings for this model run to specified output folder (Filename: RunInfoURock_YYYY_DOY_HHMM.txt) Initiating algorithm Creates an H2GIS Instance and load data Traceback (most recent call last): File "JdbcPreparedStatement.java", line 237, in org.h2.jdbc.JdbcPreparedStatement.execute Exception: Java Exception

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "C:\OSGeo4W\apps\Python312\Lib\site-packages\jaydebeapi__init__.py", line 534, in execute is_rs = self._prep.execute() ^^^^^^^^^^^^^^^^^^^^ org.h2.jdbc.org.h2.jdbc.JdbcSQLSyntaxErrorException: org.h2.jdbc.JdbcSQLSyntaxErrorException: Function "NONE" not found; SQL statement:

CALL None('C:\Users\nicol\OneDrive - Politecnico di Milano\Documenti\MultiCAST_DATA\Barcelona\Paper_update_22122023\3_BCN\New_Inputs_UMEP\WInd_Test\osm_41.402157_2.152937_41.406701_2.159071\building.fgb','TEMPO'); CREATE TABLE build_pre_srid_20240603105346 AS SELECT * FROM TEMPO; [90022-200]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users/nicol/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\processor\urock_processing_algorithm.py", line 447, in processAlgorithm MainCalculation.main(javaEnvironmentPath = javaEnvVar, File "C:\Users/nicol/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\functions\URock\MainCalculation.py", line 135, in main loadData.loadData(fromCad = False, File "C:\Users/nicol/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\functions\URock\loadData.py", line 120, in loadData loadFile(cursor = cursor, File "C:\Users/nicol/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\functions\URock\loadData.py", line 318, in loadFile cursor.execute(""" File "C:\OSGeo4W\apps\Python312\Lib\site-packages\jaydebeapi__init.py", line 536, in execute _handle_sql_exception() File "C:\OSGeo4W\apps\Python312\Lib\site-packages\jaydebeapi__init__.py", line 165, in _handle_sql_exception_jpype reraise(exc_type, exc_info[1], exc_info[2]) File "C:\OSGeo4W\apps\Python312\Lib\site-packages\jaydebeapi__init__.py", line 57, in reraise raise value.with_traceback(tb) File "C:\OSGeo4W\apps\Python312\Lib\site-packages\jaydebeapi\init__.py", line 534, in execute is_rs = self._prep.execute() ^^^^^^^^^^^^^^^^^^^^ jaydebeapi.DatabaseError: org.h2.jdbc.JdbcSQLSyntaxErrorException: Function "NONE" not found; SQL statement:

CALL None('C:\Users\nicol\OneDrive - Politecnico di Milano\Documenti\MultiCAST_DATA\Barcelona\Paper_update_22122023\3_BCN\New_Inputs_UMEP\WInd_Test\osm_41.402157_2.152937_41.406701_2.159071\building.fgb','TEMPO'); CREATE TABLE build_pre_srid_20240603105346 AS SELECT * FROM TEMPO; [90022-200]

Execution failed after 0.33 seconds

Loading resulting layers Algorithm 'Urban Wind Field: URock v2023a' finished

nicolacolaninno commented 4 months ago

There was an issue with the projection. I have fixed it. But still not able to compute wind. I have run the uRock prepare, using the OSM buildings, and it worked out. Then I used the new buildings with the trees. But again. Here I add the shapefiles I used. uRock_Buildings_Vegetation_2.zip

QGIS version: 3.34.6-Prizren QGIS code revision: 623828f5 Qt version: 5.15.13 Python version: 3.12.3 GDAL version: 3.8.5 GEOS version: 3.12.1-CAPI-1.18.1 PROJ version: Rel. 9.4.0, March 1st, 2024 PDAL version: 2.6.3 (git-version: 816ce1) Algorithm started at: 2024-06-03T11:13:35 Algorithm 'Urban Wind Field: URock v2023a' starting… Input parameters: { 'ATTENUATION_FIELD' : '', 'BUILDINGS' : 'C:/Users/nicol/OneDrive - Politecnico di Milano/Documenti/MultiCAST/_DATA/Barcelona/Paper_update_22122023/3_BCN/New_Inputs_UMEP/WInd_Test/uRock_Buildings_2.shp', 'HEIGHT_FIELD_BUILD' : 'ROOF_HEIGH', 'HORIZONTAL_RESOLUTION' : 2, 'INPUT_PROFILE_FILE' : '', 'INPUT_PROFILE_TYPE' : 0, 'INPUT_WIND_DIRECTION' : 45, 'INPUT_WIND_HEIGHT' : 10, 'INPUT_WIND_SPEED' : 2, 'LOAD_OUTPUT' : True, 'OUTPUT_FILENAME' : 'urock_output', 'RASTER_OUTPUT' : None, 'SAVE_NETCDF' : True, 'SAVE_RASTER' : True, 'SAVE_VECTOR' : True, 'UROCK_OUTPUT' : 'C:\Users\nicol\OneDrive - Politecnico di Milano\Documenti\MultiCAST\_DATA\Barcelona\Paper_update_22122023\3_BCN\New_Inputs_UMEP\WInd_Test\WInd_urock_out_res_2m', 'VEGETATION' : 'C:/Users/nicol/OneDrive - Politecnico di Milano/Documenti/MultiCAST/_DATA/Barcelona/Paper_update_22122023/3_BCN/New_Inputs_UMEP/WInd_Test/uRock_Vegetation_2.shp', 'VEGETATION_CROWN_BASE_HEIGHT' : '', 'VEGETATION_CROWN_TOP_HEIGHT' : 'VEG_HEIGHT', 'VERTICAL_RESOLUTION' : 2, 'WIND_HEIGHT' : '1.5' }

Writing settings for this model run to specified output folder (Filename: RunInfoURock_YYYY_DOY_HHMM.txt) Initiating algorithm Creates an H2GIS Instance and load data Creates the stacked blocks used as obstacles Rotates obstacles to the right direction and calculates geometry properties Creates the 2D Röckle zones Traceback (most recent call last): File "JdbcPreparedStatement.java", line 237, in org.h2.jdbc.JdbcPreparedStatement.execute org.locationtech.jts.geom.org.locationtech.jts.geom.TopologyException: org.locationtech.jts.geom.TopologyException: found non-noded intersection between LINESTRING ( 429546.4386454006 4583859.548103601, 429546.4853386385 4583859.501410363 ) and LINESTRING ( 429546.48533863755 4583859.501410364, 429547.14575218176 4583858.84099682 ) [ (429546.4853386384, 4583859.501410363, NaN) ]

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "JdbcPreparedStatement.java", line 237, in org.h2.jdbc.JdbcPreparedStatement.execute Exception: Java Exception

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "C:\OSGeo4W\apps\Python312\Lib\site-packages\jaydebeapi__init__.py", line 534, in execute is_rs = self._prep.execute() ^^^^^^^^^^^^^^^^^^^^ org.h2.jdbc.org.h2.jdbc.JdbcSQLNonTransientException: org.h2.jdbc.JdbcSQLNonTransientException: Exception calling user-defined function: "union(MULTIPOLYGON (((430024.1088303046 4583808.302416177, 430023.97 4583808.163585872, 430023.97 4583809.109244838, 430024.17481328064 4583808.904431558, 430024.16 4583808.78, , 429755.74225263204 4583988.241537776, 429755.03514585085 4583988.948644558, 429755.74225263204 4583989.655751338)), ((429750.79250516417 4583988.948644558, 429751.49961194536 4583988.241537777, 429750.79250516417 4583987.534430996, 429750.085398383 4583988.241537777, 429750.79250516417 4583988.948644558)), ((429750.085398383 4583988.241537777, 429749.3782916018 4583987.534430996, 429748.6711848206 4583988.241537777, 429749.3782916018 4583988.948644558, 429750.085398383 4583988.241537777)), ((429748.6711848206 4583988.241537777, ...., ...., ...., 429820.79607650125 4583827.021191666)), ((429821.50318328245 4583829.1425120095, 429820.79607650125 4583828.435405228, 429820.08896972006 4583829.1425120095, 429820.79607650125 4583829.849618791, 429821.50318328245 4583830.556725572, 429822.21029006364 4583829.849618791, 429821.50318328245 4583829.1425120095)))): found non-noded intersection between LINESTRING ( 429546.4386454006 4583859.548103601, 429546.4853386385 4583859.501410363 ) and LINESTRING ( 429546.48533863755 4583859.501410364, 429547.14575218176 4583858.84099682 ) [ (429546.4853386384, 4583859.501410363, NaN) ]"; SQL statement: (SELECT ST_UNION(ST_ACCUM(THE_GEOM)) AS THE_GEOM, MIN(MIN_HEIGHT) AS MIN_HEIGHT, MIN(MAX_HEIGHT) AS MAX_HEIGHT, MIN(ATTENUATIO) AS ATTENUATIO, ID_VEG FROM temporary_built_vegetation_20240603111600 GROUP BY ID_VEG) [90105-200]

Execution failed after 1524.38 seconds (25 minutes 24 seconds)

Loading resulting layers Algorithm 'Urban Wind Field: URock v2023a' finished

j3r3m1 commented 4 months ago

OK there

CALL None('C:\Users\nicol\OneDrive - Politecnico di Milano\Documenti\MultiCAST_DATA\Barcelona\Paper_update_22122023\3_BCN\New_Inputs_UMEP\WInd_Test\osm_41.402157_2.152937_41.406701_2.159071\building.fgb','TEMPO'); CREATE TABLE build_pre_srid_20240603105346 AS SELECT * FROM TEMPO; [90022-200]

There was actually a problem with the reading functions for .fgb formats. It is now solved in the newest version (no need for your data since you have resaved it to .shp which made it functional).

Several comments. Do you use the last UMEP version (the one downloaded as a zip) ? If not first have a try to the newest one downloading the repo as zip and then importing it into the extension window in QGIS.

If this does not work, for now the calculation you are making are too long (making it also difficult to debug). You should unfortunately reduce the size of the area computed. I recommend:

nicolacolaninno commented 4 months ago

Hi Jeremy, many thanks for this priceless support! Actually, I am still having trouble. I'll try your suggestions and run it again. Hopefully, it will work. I'll keep you posted if you don't mind.

j3r3m1 commented 4 months ago

So if I understand well, you still have the same problem using the last "snapshot" version (the zip) ?

And you will soon have a try with the configuration I propose (splitting the area into several and increasing the vegetation raster resolution). If so, great, let us know

nicolacolaninno commented 4 months ago

I haven't tried the zip version. Do you mind providing the link to the repository of the zip version? So, I'll try all your suggestions, including using the zip first, and then confining the area to be smaller and the tree canopy resolution.

j3r3m1 commented 4 months ago

Here is the version: https://github.com/UMEP-dev/UMEP-processing/archive/refs/heads/main.zip (with this link you always have the development version).

I definitely recommend reducing the size of the zone and the tree canopy resolution anyway.

nicolacolaninno commented 4 months ago

Hi Jeremy, So, I have followed your suggestions, but no way. I cannot move on. I am using the zip from the link you provided. Here attached the inputs I used, a screenshot of the setting, and the error. I would really appreciate any help.

uRock_Build_Veg_CLIP_300m.zip

uRock

QGIS version: 3.34.6-Prizren QGIS code revision: 623828f5 Qt version: 5.15.13 Python version: 3.12.3 GDAL version: 3.8.5 GEOS version: 3.12.1-CAPI-1.18.1 PROJ version: Rel. 9.4.0, March 1st, 2024 PDAL version: 2.6.3 (git-version: 816ce1) Algorithm started at: 2024-06-04T16:25:15 Algorithm 'Urban Wind Field: URock v2023a' starting… Input parameters: { 'ATTENUATION_FIELD' : '', 'BUILDINGS' : 'C:/Users/nicol/OneDrive - Politecnico di Milano/Documenti/MultiCAST/_DATA/Barcelona/Paper_update_22122023/3_BCN/New_Inputs_UMEP/WInd_Test/uRock_Buildings_CLIP_300m.shp', 'HEIGHT_FIELD_BUILD' : 'ROOF_HEIGH', 'HORIZONTAL_RESOLUTION' : 2, 'INPUT_PROFILE_FILE' : '', 'INPUT_PROFILE_TYPE' : 0, 'INPUT_WIND_DIRECTION' : 45, 'INPUT_WIND_HEIGHT' : 10, 'INPUT_WIND_SPEED' : 2, 'LOAD_OUTPUT' : True, 'OUTPUT_FILENAME' : 'urock_output', 'RASTER_OUTPUT' : None, 'SAVE_NETCDF' : True, 'SAVE_RASTER' : True, 'SAVE_VECTOR' : True, 'UROCK_OUTPUT' : 'C:\Users\nicol\OneDrive - Politecnico di Milano\Documenti\MultiCAST\_DATA\Barcelona\Paper_update_22122023\3_BCN\New_Inputs_UMEP\WInd_Test\WInd_urock_out_res_2m', 'VEGETATION' : 'C:/Users/nicol/OneDrive - Politecnico di Milano/Documenti/MultiCAST/_DATA/Barcelona/Paper_update_22122023/3_BCN/New_Inputs_UMEP/WInd_Test/uRock_Vegetation_CLIP_300m.shp', 'VEGETATION_CROWN_BASE_HEIGHT' : '', 'VEGETATION_CROWN_TOP_HEIGHT' : 'VEG_HEIGHT', 'VERTICAL_RESOLUTION' : 2, 'WIND_HEIGHT' : '1.5' }

Writing settings for this model run to specified output folder (Filename: RunInfoURock_YYYY_DOY_HHMM.txt) Initiating algorithm Creates an H2GIS Instance and load data Traceback (most recent call last): File "JdbcPreparedStatement.java", line 265, in org.h2.jdbc.JdbcPreparedStatement.execute java.lang.java.lang.UnsupportedClassVersionError: java.lang.UnsupportedClassVersionError: org/h2gis/functions/factory/H2GISFunctions has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "JdbcPreparedStatement.java", line 265, in org.h2.jdbc.JdbcPreparedStatement.execute Exception: Java Exception

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "C:\OSGeo4W\apps\Python312\Lib\site-packages\jaydebeapi__init__.py", line 534, in execute is_rs = self._prep.execute() ^^^^^^^^^^^^^^^^^^^^ org.h2.jdbc.org.h2.jdbc.JdbcSQLNonTransientException: org.h2.jdbc.JdbcSQLNonTransientException: General error: "org.h2gis.functions.factory.H2GISFunctions"; SQL statement: CREATE ALIAS IF NOT EXISTS H2GIS_SPATIAL FOR "org.h2gis.functions.factory.H2GISFunctions.load" [50000-224]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users/nicol/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\UMEP-processing-main\processor\urock_processing_algorithm.py", line 447, in processAlgorithm MainCalculation.main(javaEnvironmentPath = javaEnvVar, File "C:\Users/nicol/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\UMEP-processing-main\functions\URock\MainCalculation.py", line 134, in main H2gisConnection.startH2gisInstance(dbDirectory = dBDir, File "C:\Users/nicol/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\UMEP-processing-main\functions\URock\H2gisConnection.py", line 155, in startH2gisInstance cur.execute("CREATE ALIAS IF NOT EXISTS H2GIS_SPATIAL FOR \"org.h2gis.functions.factory.H2GISFunctions.load\";") File "C:\OSGeo4W\apps\Python312\Lib\site-packages\jaydebeapi__init.py", line 536, in execute _handle_sql_exception() File "C:\OSGeo4W\apps\Python312\Lib\site-packages\jaydebeapi__init__.py", line 165, in _handle_sql_exception_jpype reraise(exc_type, exc_info[1], exc_info[2]) File "C:\OSGeo4W\apps\Python312\Lib\site-packages\jaydebeapi__init__.py", line 57, in reraise raise value.with_traceback(tb) File "C:\OSGeo4W\apps\Python312\Lib\site-packages\jaydebeapi\init__.py", line 534, in execute is_rs = self._prep.execute() ^^^^^^^^^^^^^^^^^^^^ jaydebeapi.DatabaseError: org.h2.jdbc.JdbcSQLNonTransientException: General error: "org.h2gis.functions.factory.H2GISFunctions"; SQL statement: CREATE ALIAS IF NOT EXISTS H2GIS_SPATIAL FOR "org.h2gis.functions.factory.H2GISFunctions.load" [50000-224]

Execution failed after 0.04 seconds

Loading resulting layers Algorithm 'Urban Wind Field: URock v2023a' finished

j3r3m1 commented 4 months ago

Do you know what is your java version ? We have updated some of the libraries since last time, maybe your java version is too old ?

nicolacolaninno commented 4 months ago

Hey Jeremy, Thanks a lot for your time and support. However, I am not able to make this work. I cannot figure out what is wrong. I have also uninstalled Java and re-installed the latest version available. But, no way. Any idea? Can I ask you if you tried running the uRock with the data I provided to you and if it works for you? I am getting crazy with this.

nicolacolaninno commented 4 months ago

QGIS version: 3.36.2-Maidenhead QGIS code revision: 6d250527 Qt version: 5.15.13 Python version: 3.12.3 GDAL version: 3.8.5 GEOS version: 3.12.1-CAPI-1.18.1 PROJ version: Rel. 9.4.0, March 1st, 2024 PDAL version: 2.6.3 (git-version: 816ce1) Algorithm started at: 2024-06-05T17:32:04 Algorithm 'Urban Wind Field: URock v2023a' starting… Input parameters: { 'ATTENUATION_FIELD' : '', 'BUILDINGS' : 'C:/Users/nicol/OneDrive - Politecnico di Milano/Documenti/MultiCAST/_DATA/Barcelona/Paper_update_22122023/3_BCN/New_Inputs_UMEP/WInd_Test/uRock_Buildings_CLIP_300m.shp', 'HEIGHT_FIELD_BUILD' : 'ROOF_HEIGH', 'HORIZONTAL_RESOLUTION' : 2, 'INPUT_PROFILE_FILE' : '', 'INPUT_PROFILE_TYPE' : 0, 'INPUT_WIND_DIRECTION' : 45, 'INPUT_WIND_HEIGHT' : 10, 'INPUT_WIND_SPEED' : 2, 'LOAD_OUTPUT' : True, 'OUTPUT_FILENAME' : 'urock_output', 'RASTER_OUTPUT' : None, 'SAVE_NETCDF' : True, 'SAVE_RASTER' : True, 'SAVE_VECTOR' : True, 'UROCK_OUTPUT' : 'C:\Users\nicol\OneDrive - Politecnico di Milano\Documenti\MultiCAST\_DATA\Barcelona\Paper_update_22122023\3_BCN\New_Inputs_UMEP\WInd_Test\WInd_urock_out_res_2m', 'VEGETATION' : 'C:/Users/nicol/OneDrive - Politecnico di Milano/Documenti/MultiCAST/_DATA/Barcelona/Paper_update_22122023/3_BCN/New_Inputs_UMEP/WInd_Test/uRock_Vegetation_CLIP_300m.shp', 'VEGETATION_CROWN_BASE_HEIGHT' : '', 'VEGETATION_CROWN_TOP_HEIGHT' : 'VEG_HEIGHT', 'VERTICAL_RESOLUTION' : 2, 'WIND_HEIGHT' : '1.5' }

Writing settings for this model run to specified output folder (Filename: RunInfoURock_YYYY_DOY_HHMM.txt) Initiating algorithm Creates an H2GIS Instance and load data Traceback (most recent call last): File "JdbcPreparedStatement.java", line 265, in org.h2.jdbc.JdbcPreparedStatement.execute java.lang.java.lang.UnsupportedClassVersionError: java.lang.UnsupportedClassVersionError: org/h2gis/functions/factory/H2GISFunctions has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "JdbcPreparedStatement.java", line 265, in org.h2.jdbc.JdbcPreparedStatement.execute Exception: Java Exception

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "C:\OSGeo4W\apps\Python312\Lib\site-packages\jaydebeapi__init__.py", line 534, in execute is_rs = self._prep.execute() ^^^^^^^^^^^^^^^^^^^^ org.h2.jdbc.org.h2.jdbc.JdbcSQLNonTransientException: org.h2.jdbc.JdbcSQLNonTransientException: General error: "org.h2gis.functions.factory.H2GISFunctions"; SQL statement: CREATE ALIAS IF NOT EXISTS H2GIS_SPATIAL FOR "org.h2gis.functions.factory.H2GISFunctions.load" [50000-224]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users/nicol/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\UMEP-processing-main\processor\urock_processing_algorithm.py", line 447, in processAlgorithm MainCalculation.main(javaEnvironmentPath = javaEnvVar, File "C:\Users/nicol/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\UMEP-processing-main\functions\URock\MainCalculation.py", line 134, in main H2gisConnection.startH2gisInstance(dbDirectory = dBDir, File "C:\Users/nicol/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\UMEP-processing-main\functions\URock\H2gisConnection.py", line 155, in startH2gisInstance cur.execute("CREATE ALIAS IF NOT EXISTS H2GIS_SPATIAL FOR \"org.h2gis.functions.factory.H2GISFunctions.load\";") File "C:\OSGeo4W\apps\Python312\Lib\site-packages\jaydebeapi__init.py", line 536, in execute _handle_sql_exception() File "C:\OSGeo4W\apps\Python312\Lib\site-packages\jaydebeapi__init__.py", line 165, in _handle_sql_exception_jpype reraise(exc_type, exc_info[1], exc_info[2]) File "C:\OSGeo4W\apps\Python312\Lib\site-packages\jaydebeapi__init__.py", line 57, in reraise raise value.with_traceback(tb) File "C:\OSGeo4W\apps\Python312\Lib\site-packages\jaydebeapi\init__.py", line 534, in execute is_rs = self._prep.execute() ^^^^^^^^^^^^^^^^^^^^ jaydebeapi.DatabaseError: org.h2.jdbc.JdbcSQLNonTransientException: General error: "org.h2gis.functions.factory.H2GISFunctions"; SQL statement: CREATE ALIAS IF NOT EXISTS H2GIS_SPATIAL FOR "org.h2gis.functions.factory.H2GISFunctions.load" [50000-224]

Execution failed after 8.62 seconds

Loading resulting layers Algorithm 'Urban Wind Field: URock v2023a' finished

j3r3m1 commented 4 months ago

What is the version of Java you are using ? If you do not know you can open a terminal and type "java --version". Unfortunately I had a try with the last dataset you sent to me and it is taking ages. And I have no time right now to investigate the reason. Maybe next week but not sure.

nicolacolaninno commented 4 months ago

Hi Jeremy, No worries, I do not want to mess up your schedule. This is the Java version

C:\Users\nicol>java -version java version "1.8.0_411" Java(TM) SE Runtime Environment (build 1.8.0_411-b09) Java HotSpot(TM) 64-Bit Server VM (build 25.411-b09, mixed mode)

nicolacolaninno commented 4 months ago

You may happen to have a Jupiter Notebook to run the uRock outside of QGIS?

j3r3m1 commented 4 months ago

You are using a version of Java which is quite old (version 8). Or at least you are using an old version besides a more recent one. You should at least use the version 11. Could you try uninstalling the 1.8 version ?

nicolacolaninno commented 4 months ago

If I go to the Java webpage this is the latest released version apparently. They say Release date: April 16, 2024.

https://www.java.com/en/download/

image

j3r3m1 commented 4 months ago

Please prefer to use a higher version (I confess not that simple to get...): https://www.oracle.com/fr/java/technologies/javase/jdk11-archive-downloads.html

nicolacolaninno commented 3 months ago

Hi Jeremy, Thanks a lot for your help. I have tried all your suggestions. Now also have the proper Java version. Indeed, with this Java, no error messages arise. However, the process gets stuck, running hours without giving any results. Which is very weird as the area is now quite small. I really cannot figure out what is wrong. Do you have any further suggestions? Do you happen to have a testing area that works with you that I can try on my machine? The inputs I mean.

j3r3m1 commented 3 months ago

Can you have a try with the official UMEP test data link ? You first have to use the URock preprocessor using DEM, DSM and CDSM (the last for the vegetation - optional) before using the URock processor.

nicolacolaninno commented 3 months ago

Thanks, Jeremy! I'll give it a try, and let you know.

nicolacolaninno commented 3 months ago

Hi Jeremy, I have tried with your data and it actually works. Then, I have tried with mine again. I figured out what the issue is. The tool works properly. The issue is only related to the complexity of the geometries of the polygons and the extent of the area under investigation. The input I was using for buildings was very complex, full of holes, and, per each bolock footpring I have multiple vuildings inside with different heights. So, the data was very complex. I have simplified it and it works. Many thanks for your time and your kindness. It was very heplful. Just a question. Do you think in the future will be possible to speed up the process and allow bigger areas to be processed. Currently, If I try an extent of 1.200 meters side, it is not possible to get results. The beauty of this tool in GIS is the (theoretically) capacity of handling wider areas with respect to other tools such as EnviMET, for instance.

j3r3m1 commented 3 months ago

OK, nice to read that you could manage to run your area. Using a 1'200 m wide area is currently not working because it takes too much memory (you have to consider both horizontal and vertical dimensions, the number of voxel resulting will then be too high...). In this case the area should be split. In the future we could think about using nested grids to run large areas more easily. About the URock quickness improvement, a clear improvement for long simulations can already been performed modifying the way URock is iterating over time steps. The first part of the code (initialization) could be performed for a limited number of directions (let's say every x degrees) and then the calculation would be much quicker since only the mass balance will be solved for each time step... But for both these work I currently have no time for that. Anyone interested can have a try (but better have a discussion before of course).

nicolacolaninno commented 3 months ago

This sounds great, Jeremy! Many thanks for you priceless support, time and consideration. I would like to keep posted. As I am not that much able to work with coding, I think what you mention is out of my skills. However, if I can find other colleqgues interested, I will definitely let you know, no worries.