UMEP-dev / UMEP-processing

7 stars 9 forks source link

UMEP Processing / URock v2023a - crashes with Java Exception No transformation found from epsg:31255 to epsg:4326 #75

Closed johanneshorak closed 1 month ago

johanneshorak commented 2 months ago

Hi all,

I've been attempting to test URock a bit, it might be perfectly useful for our applications. However, I keep running into issues - not sure if I'm at fault or something else is going on, I was unable to pin down the issue. See the protocol below! If I attempt to reproject the layer to epsg 4326 the crash occurs earlier already during the "rotating obstacles to the right direction" step. Namely a SQL statement is made that include two columns "None". The protocoll for the epsg 31255 shape file attempt is found below - any help appreciated, Thanks for all your work and maintanance efforts!

QGIS-Version: 3.36.1-Maidenhead
QGIS-Codeversion: 3e589453
Qt-Version: 5.15.3
Python-Version: 3.9.18
GDAL-Version: 3.8.4
GEOS-Version: 3.12.1-CAPI-1.18.1
PROJ-Version: Rel. 9.3.1, December 1st, 2023
PDAL-Version: 2.6.0 (git-version: 3fced5)
Algorithmus gestartet um: 2024-04-22T21:37:11
Algorithmus Urban Wind Field: URock v2023a startet…
Eingabeparameter:
{ 'ATTENUATION_FIELD' : '', 'BUILDINGS' : 'path2buildingshapefile/buildings epsg 31255.shp', 'HEIGHT_FIELD_BUILD' : 'HOEH', 'HORIZONTAL_RESOLUTION' : 2, 'INPUT_PROFILE_FILE' : '', 'INPUT_PROFILE_TYPE' : 0, 'INPUT_WIND_DIRECTION' : 45, 'INPUT_WIND_HEIGHT' : 10, 'INPUT_WIND_SPEED' : 2, 'LOAD_OUTPUT' : True, 'OUTPUT_FILENAME' : 'urock_output', 'RASTER_OUTPUT' : None, 'SAVE_NETCDF' : True, 'SAVE_RASTER' : True, 'SAVE_VECTOR' : True, 'UROCK_OUTPUT' : 'TEMPORARY_OUTPUT', 'VEGETATION' : None, 'VEGETATION_CROWN_BASE_HEIGHT' : '', 'VEGETATION_CROWN_TOP_HEIGHT' : '', 'VERTICAL_RESOLUTION' : 2, 'WIND_HEIGHT' : '1.5' }

Writing settings for this model run to specified output folder (Filename: RunInfoURock_YYYY_DOY_HHMM.txt)
Initiating algorithm
Creates an H2GIS Instance and load data
Creates the stacked blocks used as obstacles
Rotates obstacles to the right direction and calculates geometry properties
Creates the 2D Röckle zones
Creates the 2D grid
Initializes the 3D grid within Röckle zones
Deals with building zones superimposition
Deals with vegetation zones superimposition
Initialize the 3D wind in the grid
Rasterize the data
Apply the wind solver equations
Iteration 1 (max 500) - eps = 0.466106 >= 0.0001

Iteration 51 (max 500) - eps = 0.009922 >= 0.0001

Iteration 101 (max 500) - eps = 0.001968 >= 0.0001

Iteration 151 (max 500) - eps = 0.000407 >= 0.0001

Traceback (most recent call last):
File "JdbcPreparedStatement.java", line 230, in org.h2.jdbc.JdbcPreparedStatement.execute
java.lang.java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: No transformation found from epsg:31255 to epsg:4326

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "JdbcPreparedStatement.java", line 230, in org.h2.jdbc.JdbcPreparedStatement.execute
Exception: Java Exception

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "C:\...\AppData\Roaming\Python\Python39\site-packages\jaydebeapi\__init__.py", line 534, in execute
is_rs = self._prep.execute()
org.h2.jdbc.org.h2.jdbc.JdbcSQLNonTransientException: org.h2.jdbc.JdbcSQLNonTransientException: Fehler beim Aufruf eine benutzerdefinierten Funktion: "ST_Transform(conn722: url=jdbc:default:connection user=SA, POINT (73246.46941414499 355711.07123156055), 4326): No transformation found from epsg:31255 to epsg:4326"
Exception calling user-defined function: "ST_Transform(conn722: url=jdbc:default:connection user=SA, POINT (73246.46941414499 355711.07123156055), 4326): No transformation found from epsg:31255 to epsg:4326"; SQL statement:

SELECT ST_X(THE_GEOM) AS LON, ST_Y(THE_GEOM) AS LAT FROM 
(SELECT ST_TRANSFORM(ST_SETSRID(THE_GEOM,31255), 4326) AS THE_GEOM FROM GRID_ROTATED)
[90105-200]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users/.../AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\processor\urock_processing_algorithm.py", line 447, in processAlgorithm
MainCalculation.main(javaEnvironmentPath = javaEnvVar,
File "C:\Users/.../AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\functions\URock\MainCalculation.py", line 776, in main
saveData.saveBasicOutputs(cursor = cursor , z_out = z_out,
File "C:\Users/.../AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\functions\URock\saveData.py", line 42, in saveBasicOutputs
cursor.execute("""
File "C:\Users\...\AppData\Roaming\Python\Python39\site-packages\jaydebeapi\__init__.py", line 536, in execute
_handle_sql_exception()
File "C:\Users\...\AppData\Roaming\Python\Python39\site-packages\jaydebeapi\__init__.py", line 165, in _handle_sql_exception_jpype
reraise(exc_type, exc_info[1], exc_info[2])
File "C:\Users\...\AppData\Roaming\Python\Python39\site-packages\jaydebeapi\__init__.py", line 57, in reraise
raise value.with_traceback(tb)
File "C:\Users\...\AppData\Roaming\Python\Python39\site-packages\jaydebeapi\__init__.py", line 534, in execute
is_rs = self._prep.execute()
jaydebeapi.DatabaseError: org.h2.jdbc.JdbcSQLNonTransientException: Fehler beim Aufruf eine benutzerdefinierten Funktion: "ST_Transform(conn722: url=jdbc:default:connection user=SA, POINT (73246.46941414499 355711.07123156055), 4326): No transformation found from epsg:31255 to epsg:4326"
Exception calling user-defined function: "ST_Transform(conn722: url=jdbc:default:connection user=SA, POINT (73246.46941414499 355711.07123156055), 4326): No transformation found from epsg:31255 to epsg:4326"; SQL statement:

SELECT ST_X(THE_GEOM) AS LON, ST_Y(THE_GEOM) AS LAT FROM 
(SELECT ST_TRANSFORM(ST_SETSRID(THE_GEOM,31255), 4326) AS THE_GEOM FROM GRID_ROTATED)
[90105-200]

Execution failed after 18.09 Sekunden

Lade Ergebnis Layer
Algorithmus 'Urban Wind Field: URock v2023a' beendet
biglimp commented 2 months ago

It looks like a CRS transformation between your local CRS in Austria and WGS84 is what is causing the issue. You can try to first transform your data to something more common such as UTM and hen try again. Otherwise, @j3r3m1 will assist you when he is available.

johanneshorak commented 2 months ago

Thanks for the advice, I transformed to epsg 25833 / UTM zone 33N. Unfortunately I run into a different error now, even though everything until the final transform seems to work as intended:

QGIS-Version: 3.36.1-Maidenhead
QGIS-Codeversion: 3e589453
Qt-Version: 5.15.3
Python-Version: 3.9.18
GDAL-Version: 3.8.4
GEOS-Version: 3.12.1-CAPI-1.18.1
PROJ-Version: Rel. 9.3.1, December 1st, 2023
PDAL-Version: 2.6.0 (git-version: 3fced5)
Algorithmus gestartet um: 2024-04-23T20:21:14
Algorithmus Urban Wind Field: URock v2023a startet…
Eingabeparameter:
{ 'ATTENUATION_FIELD' : '', 'BUILDINGS' : 'C:\\Users\\...\\OneDrive\\Desktop\\windspeed_tests\\buildings epsg 25833 UTM zone 33N.shp|layername=buildings epsg 25833 UTM zone 33N', 'HEIGHT_FIELD_BUILD' : 'HOEH', 'HORIZONTAL_RESOLUTION' : 2, 'INPUT_PROFILE_FILE' : '', 'INPUT_PROFILE_TYPE' : 1, 'INPUT_WIND_DIRECTION' : 45, 'INPUT_WIND_HEIGHT' : 10, 'INPUT_WIND_SPEED' : 2, 'LOAD_OUTPUT' : True, 'OUTPUT_FILENAME' : 'urock_output', 'RASTER_OUTPUT' : None, 'SAVE_NETCDF' : True, 'SAVE_RASTER' : True, 'SAVE_VECTOR' : True, 'UROCK_OUTPUT' : 'C:\\Users\\...\\OneDrive\\Desktop\\windspeed_tests', 'VEGETATION' : None, 'VEGETATION_CROWN_BASE_HEIGHT' : '', 'VEGETATION_CROWN_TOP_HEIGHT' : '', 'VERTICAL_RESOLUTION' : 2, 'WIND_HEIGHT' : '1.5' }

Writing settings for this model run to specified output folder (Filename: RunInfoURock_YYYY_DOY_HHMM.txt)
Initiating algorithm
Creates an H2GIS Instance and load data
Creates the stacked blocks used as obstacles
Rotates obstacles to the right direction and calculates geometry properties
Creates the 2D Röckle zones
Creates the 2D grid
Initializes the 3D grid within Röckle zones
Deals with building zones superimposition
Deals with vegetation zones superimposition
Initialize the 3D wind in the grid
Rasterize the data
Apply the wind solver equations
Iteration 1 (max 500) - eps = 0.50175 >= 0.0001

Iteration 51 (max 500) - eps = 0.009421 >= 0.0001

Iteration 101 (max 500) - eps = 0.001858 >= 0.0001

Iteration 151 (max 500) - eps = 0.000389 >= 0.0001

Traceback (most recent call last):
File "JdbcPreparedStatement.java", line 230, in org.h2.jdbc.JdbcPreparedStatement.execute
org.cts.crs.org.cts.crs.CRSException: org.cts.crs.CRSException: Registry 'epsg' contains no parameter for [epsg:5556] 

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "JdbcPreparedStatement.java", line 230, in org.h2.jdbc.JdbcPreparedStatement.execute
java.sql.java.sql.SQLException: java.sql.SQLException: Cannot create the CRS

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "JdbcPreparedStatement.java", line 230, in org.h2.jdbc.JdbcPreparedStatement.execute
Exception: Java Exception

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "C:\Users\...\AppData\Roaming\Python\Python39\site-packages\jaydebeapi\__init__.py", line 534, in execute
is_rs = self._prep.execute()
org.h2.jdbc.org.h2.jdbc.JdbcSQLException: org.h2.jdbc.JdbcSQLException: Cannot create the CRS; SQL statement:

SELECT ST_X(THE_GEOM) AS LON, ST_Y(THE_GEOM) AS LAT FROM 
(SELECT ST_TRANSFORM(ST_SETSRID(THE_GEOM,5556), 4326) AS THE_GEOM FROM GRID_ROTATED)
[0-200]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users/.../AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\processor\urock_processing_algorithm.py", line 447, in processAlgorithm
MainCalculation.main(javaEnvironmentPath = javaEnvVar,
File "C:\Users/.../AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\functions\URock\MainCalculation.py", line 776, in main
saveData.saveBasicOutputs(cursor = cursor , z_out = z_out,
File "C:\Users/.../AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\functions\URock\saveData.py", line 42, in saveBasicOutputs
cursor.execute("""
File "C:\Users\...\AppData\Roaming\Python\Python39\site-packages\jaydebeapi\__init__.py", line 536, in execute
_handle_sql_exception()
File "C:\Users\...\AppData\Roaming\Python\Python39\site-packages\jaydebeapi\__init__.py", line 165, in _handle_sql_exception_jpype
reraise(exc_type, exc_info[1], exc_info[2])
File "C:\Users\...\AppData\Roaming\Python\Python39\site-packages\jaydebeapi\__init__.py", line 57, in reraise
raise value.with_traceback(tb)
File "C:\Users\...\AppData\Roaming\Python\Python39\site-packages\jaydebeapi\__init__.py", line 534, in execute
is_rs = self._prep.execute()
jaydebeapi.DatabaseError: org.h2.jdbc.JdbcSQLException: Cannot create the CRS; SQL statement:

SELECT ST_X(THE_GEOM) AS LON, ST_Y(THE_GEOM) AS LAT FROM 
(SELECT ST_TRANSFORM(ST_SETSRID(THE_GEOM,5556), 4326) AS THE_GEOM FROM GRID_ROTATED)
[0-200]

Execution failed after 25.76 Sekunden

Lade Ergebnis Layer
Algorithmus 'Urban Wind Field: URock v2023a' beendet
biglimp commented 2 months ago

Hopefully @j3r3m1 can help you here.

johanneshorak commented 2 months ago

Appreciate it, thank you.

Meanwhile I managed to install QGIS via conda on ubuntu, unfortunately, I keep running into the exact same errors. I'll attempt to find another dataset that I can use for testing, maybe there's an issue there..

edit: Different dataset yields the very same results.

j3r3m1 commented 2 months ago

Hi @johanneshorak Thank you for reporting. Can you share your data with us to simplify the debug ?

johanneshorak commented 1 month ago

I can't share the exact dataset, but this causes the same issue, see attached! Height field is HOEHE_OBJE

Meanwhile I attempted to create the shapes from scratch as well (e.g. starting with a epsg 4326 project and creating a new shape layer containing polygons), this causes the following error:

` QGIS-Version: 3.36.1-Maidenhead QGIS-Codeversion: 3e589453 Qt-Version: 5.15.3 Python-Version: 3.9.18 GDAL-Version: 3.8.4 GEOS-Version: 3.12.1-CAPI-1.18.1 PROJ-Version: Rel. 9.3.1, December 1st, 2023 PDAL-Version: 2.6.0 (git-version: 3fced5) Algorithmus gestartet um: 2024-05-05T10:42:52 Algorithmus Urban Wind Field: URock v2023a startet… Eingabeparameter: { 'ATTENUATION_FIELD' : '', 'BUILDINGS' : 'C:/Users/.../OneDrive/Desktop/windspeed_tests/buildings shape file/buildings epsg 4326 manual.shp', 'HEIGHT_FIELD_BUILD' : 'hoehe', 'HORIZONTAL_RESOLUTION' : 2, 'INPUT_PROFILE_FILE' : '', 'INPUT_PROFILE_TYPE' : 0, 'INPUT_WIND_DIRECTION' : 45, 'INPUT_WIND_HEIGHT' : 10, 'INPUT_WIND_SPEED' : 2, 'LOAD_OUTPUT' : True, 'OUTPUT_FILENAME' : 'urock_output', 'RASTER_OUTPUT' : None, 'SAVE_NETCDF' : True, 'SAVE_RASTER' : True, 'SAVE_VECTOR' : True, 'UROCK_OUTPUT' : 'C:\Users\...\OneDrive\Desktop\windspeed_tests', 'VEGETATION' : None, 'VEGETATION_CROWN_BASE_HEIGHT' : '', 'VEGETATION_CROWN_TOP_HEIGHT' : '', 'VERTICAL_RESOLUTION' : 2, 'WIND_HEIGHT' : '1.5' }

Writing settings for this model run to specified output folder (Filename: RunInfoURock_YYYY_DOY_HHMM.txt) Initiating algorithm Creates an H2GIS Instance and load data Creates the stacked blocks used as obstacles Rotates obstacles to the right direction and calculates geometry properties Traceback (most recent call last): File "JdbcPreparedStatement.java", line 237, in org.h2.jdbc.JdbcPreparedStatement.execute Exception: Java Exception

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "C:\Users...\AppData\Roaming\Python\Python39\site-packages\jaydebeapi__init__.py", line 534, in execute is_rs = self._prep.execute() org.h2.jdbc.org.h2.jdbc.JdbcSQLSyntaxErrorException: org.h2.jdbc.JdbcSQLSyntaxErrorException: Feld "NONE" nicht gefunden Column "NONE" not found; SQL statement:

CREATE TABLE stacked_block_table_ROTATED AS SELECT ST_MAKEVALID(ST_ROTATE(THE_GEOM, 0.7853981633974483, None, None)) AS THE_GEOM, ID_STACKED_BLOCK,ID_BLOCK,HEIGHT_ROO FROM stacked_block_table; [42122-200]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users/.../AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\processor\urock_processing_algorithm.py", line 447, in processAlgorithm MainCalculation.main(javaEnvironmentPath = javaEnvVar, File "C:\Users/.../AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\functions\URock\MainCalculation.py", line 187, in main Obstacles.windRotation(cursor = cursor, File "C:\Users/.../AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\functions\URock\Obstacles.py", line 81, in windRotation cursor.execute(";".join(sqlRotateQueries)) File "C:\Users...\AppData\Roaming\Python\Python39\site-packages\jaydebeapi__init.py", line 536, in execute _handle_sql_exception() File "C:\Users...\AppData\Roaming\Python\Python39\site-packages\jaydebeapi__init__.py", line 165, in _handle_sql_exception_jpype reraise(exc_type, exc_info[1], exc_info[2]) File "C:\Users...\AppData\Roaming\Python\Python39\site-packages\jaydebeapi__init__.py", line 57, in reraise raise value.with_traceback(tb) File "C:\Users...\AppData\Roaming\Python\Python39\site-packages\jaydebeapi\init__.py", line 534, in execute is_rs = self._prep.execute() jaydebeapi.DatabaseError: org.h2.jdbc.JdbcSQLSyntaxErrorException: Feld "NONE" nicht gefunden Column "NONE" not found; SQL statement:

CREATE TABLE stacked_block_table_ROTATED AS SELECT ST_MAKEVALID(ST_ROTATE(THE_GEOM, 0.7853981633974483, None, None)) AS THE_GEOM, ID_STACKED_BLOCK,ID_BLOCK,HEIGHT_ROO FROM stacked_block_table; [42122-200]

Execution failed after 1.95 Sekunden

Lade Ergebnis Layer Algorithmus 'Urban Wind Field: URock v2023a' beendet `

buildings DLM 8000.zip

johanneshorak commented 1 month ago

Quick update: I got it to run through! Creating the polygons in EPSG 3587 seems to work. I haven't had time to look into whether the other building shapes work if I just reproject them, i'll get back to you once I tested it.

j3r3m1 commented 1 month ago

Thanks for the update. Indeed, working with 4326 is not possible since it is in lat/lon, you should first reproject in a local metric coordinate system. Let us know if you want us to have a look to the dataset that creates the error (you can send me a subset of your data directly via direct email)

johanneshorak commented 1 month ago

Hi j3r3m1 - thank you - see above, I've attached a dataset in that post (https://github.com/UMEP-dev/UMEP-processing/files/15212243/buildings.DLM.8000.zip)

j3r3m1 commented 1 month ago

It should now works with the last version (the one that you can dowload here: https://github.com/UMEP-dev/UMEP-processing/archive/refs/heads/main.zip). Can you try and report (close if it works) ?

johanneshorak commented 1 month ago

@j3r3m1 can confirm, urock now works with a dataset in epsg:31255 projection. Thank you very much!!

j3r3m1 commented 1 month ago

Thanks to @ebocher and the @orbisgis team !

ebocher commented 1 month ago

Hey ! UMEP friends, you are welcome. Have fun with H2GIS !