Urock: intermittant NumberFormatException

bweeding commented 10 months ago

When running URock in a loop, I occasionally get a NumberFormatException error, causing the code to fail. When I rerun the individual combination of speed and direction, the error doesn't occur.

The loop I use is below. I have included some example errors in a txt file. NumberFormatException errors.txt

df = pd.concat([pd.read_csv(i, sep=' ') for i in glob.glob("/mnt/bweeding_workspace/phd_jupyter/paper3_workflows/p3_metfiles/*.txt", recursive=True)])

data = {'wdir': 5 * round(df['wdir']/5).values,
        'U': 0.5 * round(df['U']/0.5).values}

wind_frame = pd.DataFrame(data)

cur_df = np.array_split(wind_frame.drop_duplicates(),6)[0].reset_index()

current_folder = '/mnt/bweeding_workspace/phd_jupyter/paper3_workflows/p3_digital_models/'

for idx in range(0,len(cur_df)):

        try:

            print(str(idx)+ ' of '+str(len(cur_df)))

            processing.run("umep:Urban Wind Field: URock", 
            {'BUILDINGS':current_folder+'build_height_gen15.shp',
            'HEIGHT_FIELD_BUILD':'ROOF_HEIGH',
            'VEGETATION':current_folder+'veg_height_gen15.shp',
            'VEGETATION_CROWN_TOP_HEIGHT':'VEG_HEIGHT',
            'VEGETATION_CROWN_BASE_HEIGHT':'',
            'ATTENUATION_FIELD':'',
            'INPUT_PROFILE_FILE':'',
            'INPUT_PROFILE_TYPE':0,
            'INPUT_WIND_HEIGHT':10,
            'INPUT_WIND_SPEED':str(cur_df['U'][idx]),
            'INPUT_WIND_DIRECTION':str(cur_df['wdir'][idx]),
            'RASTER_OUTPUT':current_folder+'mp_dsm_50mb_modified_final.tif',
            'HORIZONTAL_RESOLUTION':2,
            'VERTICAL_RESOLUTION':2,
            'WIND_HEIGHT':'1.5',
            'UROCK_OUTPUT':'/scratch/bweeding/p3_outputs/p3_urock_outputs_2m/CNRM-CERFACS-CNRM-CM5'+'_uwind_'+str(cur_df['U'][idx])+'_wdir'+str(cur_df['wdir'][idx])+'/',
            'OUTPUT_FILENAME':'CNRM-CERFACS-CNRM-CM5_uwind_'+str(cur_df['U'][idx])+'_wdir'+str(cur_df['wdir'][idx]),
            'SAVE_RASTER':True,
            'SAVE_VECTOR':False,
            'SAVE_NETCDF':False,
            'LOAD_OUTPUT':False})

            rast = rx.open_rasterio('/scratch/bweeding/p3_outputs/p3_urock_outputs_2m/CNRM-CERFACS-CNRM-CM5_uwind_'+str(cur_df['U'][idx])+'_wdir'+str(cur_df['wdir'][idx])+'/z1_5/CNRM-CERFACS-CNRM-CM5_uwind_'+str(cur_df['U'][idx])+'_wdir'+str(cur_df['wdir'][idx])+'WS.Gtiff')

            print('rast opened')

            rast = rast.reindex(y=list(reversed(rast.y)))

            rast['x'] = rast.x-0.5

            rast['y'] = rast.y-0.5

            rast = rast[0,:,:]

            rast.transpose().to_zarr('/scratch/bweeding/p3_outputs/p3_urock_outputs_2m/uwind_'+str(cur_df['U'][idx])+'_wdir'+str(cur_df['wdir'][idx])+'.zarr')                        

            rast.close()

            os.system('rm -r /scratch/bweeding/p3_outputs/p3_urock_outputs_2m/CNRM-CERFACS-CNRM-CM5'+'_uwind_'+str(cur_df['U'][idx])+'_wdir'+str(cur_df['wdir'][idx]))

        except:

            print('Failed')
            print('INPUT_WIND_SPEED:'+str(cur_df['U'][idx]))
            print('INPUT_WIND_DIRECTION:'+str(cur_df['wdir'][idx]))

j3r3m1 commented 10 months ago

What do you mean by occasionally ? For every wind direction ? once every 10 wind direction ?

Did you get this problem with any kind of spatial data or a specific one ?

If you have a simple unit test case that I can debug I would opt for it.

bweeding commented 10 months ago

Apologies for the lack of detail, here's an example:

urock_vectors_generalised_1.5m.zip

Wind speed: 8.04m/s at 10m Wind direction: 296.37 Horizontal res: 1m Vertical res: 2m

1st attempt fails reporting the error below, 2nd attempt works fine. Sometimes I can run 20+ URock models in a row without it happening, other times it will happen 3 times out of 10.

Connected!

Spatial functions added!

exited H2GIS Load input data Load table 'build_pre_srid_20240119130022' Load table 'veg_pre_srid_20240119130022' Creates blocks and stacked blocks Rotates geometries from 296.37 degrees Identify block base height and block cavity base Calculates obstacle properties Calculates zone properties Initializes upwind facades Update upwind facades base height Initializes downwind facades Calculates study area properties Rotates geometries from -296.37 degrees Rotates geometries from -296.37 degrees Rotates geometries from -296.37 degrees Creates displacement zones Rotates geometries from -296.37 degrees Rotates geometries from -296.37 degrees Creates cavity and wake zones debug1 debug2 debug3 debug4 debug5 Rotates geometries from -296.37 degrees Rotates geometries from -296.37 degrees Creates street canyon zones Rotates geometries from -296.37 degrees Creates rooftop zones (perpendicular and corner) Rotates geometries from -296.37 degrees Rotates geometries from -296.37 degrees Creates built-up and open vegetation zones Rotates geometries from -296.37 degrees Rotates geometries from -296.37 degrees Identify the buildings concerned by the impacted zone chosen by the user Creates the grid of points Affects each grid point to a building Rockle zone and calculates needed variables for 3D wind speed Affects each grid point to a vegetation Rockle zone and calculates needed variables for 3D wind speed Remove some of the Röckle zone points Creates backward zones Calculates the 3D wind speed factor value for each point of each BUILDING zone Rotates geometries from -296.37 degrees Rotates geometries from -296.37 degrees Rotates geometries from -296.37 degrees Rotates geometries from -296.37 degrees Rotates geometries from -296.37 degrees Rotates geometries from -296.37 degrees Rotates geometries from -296.37 degrees Rotates geometries from -296.37 degrees Rotates geometries from -296.37 degrees Calculates the 3D wind speed factor value for each point of each VEGETATION zone Rotates geometries from -296.37 degrees Deals with superimposition (keeps only 1 value per 3D point) Deals with superimposition (keeps only 1 value per 3D point) Identify upstreamer points in TEMPO_WEIGHTING table Identify upstreamer points in TEMPO_PRIORITIES table Identify upstreamer points in TEMPO_PRIORITIES_WEIGHT table Deals with superimposition (keeps only 1 value per 3D point) Identify upstreamer points in TEMPO_WEIGHTING table Identify upstreamer points in TEMPO_PRIORITIES table Identify upstreamer points in TEMPO_PRIORITIES_WEIGHT table Rotates geometries from -296.37 degrees Identify grid points intersecting buildings Set the initial 3D wind speed field Time spent for wind speed initialization: 454.79771852493286 s Shape: (558, 835, 40) - Nb cells: 18637200 Start to apply the wind solver Time spent by the wind speed solver: 580.928831577301 s Rotates geometries from -296.37 degrees

Logged critical: Traceback (most recent call last): File "JdbcPreparedStatement.java", line 237, in org.h2.jdbc.JdbcPreparedStatement.execute java.lang.java.lang.NumberFormatException: java.lang.NumberFormatException: For input string: ".061885983537887324"

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "DbException.java", line 435, in org.h2.message.DbException.getJdbcSQLException org.h2.jdbc.org.h2.jdbc.JdbcSQLDataException: org.h2.jdbc.JdbcSQLDataException: Data conversion error converting ".061885983537887324" [22018-200]

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "JdbcPreparedStatement.java", line 237, in org.h2.jdbc.JdbcPreparedStatement.execute org.h2.message.org.h2.message.DbException: org.h2.message.DbException: Data conversion error converting ".061885983537887324" [22018-200]

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "JdbcPreparedStatement.java", line 237, in org.h2.jdbc.JdbcPreparedStatement.execute Exception: Java Exception

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/init.py", line 534, in execute is_rs = self._prep.execute() org.h2.jdbc.org.h2.jdbc.JdbcSQLDataException: org.h2.jdbc.JdbcSQLDataException: Data conversion error converting "'.061885983537887324' (TEMPO_HORIZ: ""ID_POINT"" INTEGER)"; SQL statement:

        CREATE TABLE TEMPO_HORIZ(ID_POINT INTEGER, HWS DOUBLE, HWD DOUBLE, VWS DOUBLE, WS DOUBLE)
            AS SELECT ID_POINT, HWS, HWD, VWS, WS FROM CSVREAD('/tmp/tempo_horiz_wind.csv'); [22018-200]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/share/qgis/python/plugins/processing_umep/processor/urock_processing_algorithm.py", line 448, in processAlgorithm MainCalculation.main(javaEnvironmentPath = javaEnvVar, File "/usr/share/qgis/python/plugins/processing_umep/functions/URock/MainCalculation.py", line 826, in main saveData.saveBasicOutputs(cursor = cursor , z_out = z_out, File "/usr/share/qgis/python/plugins/processing_umep/functions/URock/saveData.py", line 105, in saveBasicOutputs cursor.execute( File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/init.py", line 536, in execute _handle_sql_exception() File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/init.py", line 165, in _handle_sql_exception_jpype reraise(exc_type, exc_info[1], exc_info[2]) File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/init.py", line 57, in reraise raise value.with_traceback(tb) File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/init.py", line 534, in execute is_rs = self._prep.execute() jaydebeapi.DatabaseError: org.h2.jdbc.JdbcSQLDataException: Data conversion error converting "'.061885983537887324' (TEMPO_HORIZ: ""ID_POINT"" INTEGER)"; SQL statement:

        CREATE TABLE TEMPO_HORIZ(ID_POINT INTEGER, HWS DOUBLE, HWD DOUBLE, VWS DOUBLE, WS DOUBLE)
            AS SELECT ID_POINT, HWS, HWD, VWS, WS FROM CSVREAD('/tmp/tempo_horiz_wind.csv'); [22018-200]

j3r3m1 commented 10 months ago

Here is a piece of code using your data but that does not reproduce the error you are logging. Can you test it to see if you get an error or change it in order to get an error ?


import os
from processing_umep.functions.URock.GlobalVariables import *

########################################
# Input definition #####################
########################################
plugin_directory_calc = "/home/decide/Code/Python/processing_umep/processor"
outputDirectory = "/tmp"
inputbuild = "/home/decide/Data/TestsUMEP/Ben/issue63/build_height_gen15.shp"
vegetationFilePath = "/home/decide/Data/TestsUMEP/Ben/issue63/veg_height_gen15.shp"
srid = 28355
z_ref = 10
v_ref = 2
windDirection = 200
prefix = ''
meshSize = 3
dz = 3
alongWindZoneExtend = ALONG_WIND_ZONE_EXTEND
crossWindZoneExtend = CROSS_WIND_ZONE_EXTEND
verticalExtend = VERTICAL_EXTEND
cadTriangles = ""
cadTreesIntersection = ""
tempoDirectory = TEMPO_DIRECTORY
onlyInitialization = ONLY_INITIALIZATION
maxIterations = MAX_ITERATIONS
thresholdIterations = THRESHOLD_ITERATIONS
idFieldBuild = None,
buildingHeightField = 'ROOF_HEIGH'
vegetationBaseHeight = ''
vegetationTopHeight = "VEG_HEIGHT"
idVegetation = None,
vegetationAttenuationFactor = ""
saveRockleZones = SAVE_ROCKLE_ZONES
outputRaster = None
feedback = None
saveRaster = True
saveVector = True
saveNetcdf = True
z_out = "1"
profileType = "power"
verticalProfileFile = None  

########################################
# Call QGIS in Python ##################
########################################
from qgis.core import QgsApplication

# # Starts the qgis application without the graphical user interface
gui_flag = False
app = QgsApplication([], gui_flag)
app.initQgis()

import processing

from processing.core.Processing import Processing
Processing.initialize()

from processing_umep.processing_umep_provider import ProcessingUMEPProvider
umep_provider = ProcessingUMEPProvider()
QgsApplication.processingRegistry().addProvider(umep_provider)

########################################
# Start calculations ####################
########################################

for wd in [296.37, 256.32, 12.32,56.32, 12.68, 13.65, 245.36, 353.12, 8.05, 78.65, 77.32, 12.63]:
        processing.run("umep:Urban Wind Field: URock",
                       {'BUILDINGS':inputbuild,
                        'HEIGHT_FIELD_BUILD':buildingHeightField,
                        'VEGETATION':vegetationFilePath,
                        'VEGETATION_CROWN_TOP_HEIGHT':vegetationTopHeight,
                        'VEGETATION_CROWN_BASE_HEIGHT':vegetationBaseHeight,
                        'ATTENUATION_FIELD':vegetationAttenuationFactor,
                        'INPUT_PROFILE_FILE':verticalProfileFile,
                        'INPUT_PROFILE_TYPE':prof_type,
                        'INPUT_WIND_HEIGHT':z_ref,
                        'INPUT_WIND_SPEED':v_ref,
                        'INPUT_WIND_DIRECTION':wd,
                        'RASTER_OUTPUT':outputRaster,
                        'HORIZONTAL_RESOLUTION':meshSize,
                        'VERTICAL_RESOLUTION':dz,
                        'WIND_HEIGHT':z_out,
                        'UROCK_OUTPUT':outputDirectory,
                        'OUTPUT_FILENAME':'urock_output',
                        'SAVE_RASTER':saveRaster,
                        'SAVE_VECTOR':saveVector,
                        'SAVE_NETCDF':saveNetcdf,
                        'LOAD_OUTPUT':False})

j3r3m1 commented 9 months ago

@bweeding any outputs from what I have proposed ?

bweeding commented 9 months ago

Apologies Jeremy, I'll try running it tomorrow!

On Thu, 22 Feb 2024, 8:25 pm jeremy-b, @.***> wrote:

@bweeding https://github.com/bweeding any outputs from what I have proposed ?

— Reply to this email directly, view it on GitHub https://github.com/UMEP-dev/UMEP-processing/issues/63#issuecomment-1959030990, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANYGKRHWXOEEON2FACSVH73YU4FJ3AVCNFSM6AAAAABBXXK65SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJZGAZTAOJZGA . You are receiving this because you were mentioned.Message ID: @.***>

bweeding commented 9 months ago

This time I received a different intermittent error. The first time I ran the code I received the error below for a wind direction of 78.65, but when I restarted my kernel and ran it again, there was no problem...

I also had to make a couple of modifications to your code to make it run on my system:

########################################
# Input definition #####################
########################################
# plugin_directory_calc = "/home/decide/Code/Python/processing_umep/processor"
# outputDirectory = "/tmp"
current_folder = '/mnt/bweeding_workspace/phd_jupyter/paper3_workflows/p3_digital_models/'

inputbuild = current_folder+'build_height_gen15.shp'
vegetationFilePath =current_folder+'veg_height_gen15.shp'
srid = 28355
z_ref = 10
v_ref = 2
windDirection = 200
prefix = ''
meshSize = 3
dz = 3
# alongWindZoneExtend = ALONG_WIND_ZONE_EXTEND
# crossWindZoneExtend = CROSS_WIND_ZONE_EXTEND
# verticalExtend = VERTICAL_EXTEND
cadTriangles = ""
cadTreesIntersection = ""
# tempoDirectory = TEMPO_DIRECTORY
# onlyInitialization = ONLY_INITIALIZATION
# maxIterations = MAX_ITERATIONS
# thresholdIterations = THRESHOLD_ITERATIONS
idFieldBuild = None,
buildingHeightField = 'ROOF_HEIGH'
vegetationBaseHeight = ''
vegetationTopHeight = "VEG_HEIGHT"
idVegetation = None,
vegetationAttenuationFactor = ""
# saveRockleZones = SAVE_ROCKLE_ZONES
outputRaster = None
feedback = None
saveRaster = True
saveVector = True
saveNetcdf = True
z_out = "1"
prof_type = "power"
verticalProfileFile = None

for wd in [296.37, 256.32, 12.32,56.32, 12.68, 13.65, 245.36, 353.12, 8.05, 78.65, 77.32, 12.63]:
        processing.run("umep:Urban Wind Field: URock",
                       {'BUILDINGS':inputbuild,
                        'HEIGHT_FIELD_BUILD':buildingHeightField,
                        'VEGETATION':vegetationFilePath,
                        'VEGETATION_CROWN_TOP_HEIGHT':vegetationTopHeight,
                        'VEGETATION_CROWN_BASE_HEIGHT':vegetationBaseHeight,
                        'ATTENUATION_FIELD':vegetationAttenuationFactor,
                        'INPUT_PROFILE_FILE':verticalProfileFile,
                        'INPUT_PROFILE_TYPE':0,
                        'INPUT_WIND_HEIGHT':z_ref,
                        'INPUT_WIND_SPEED':v_ref,
                        'INPUT_WIND_DIRECTION':wd,
                        'RASTER_OUTPUT':outputRaster,
                        'HORIZONTAL_RESOLUTION':meshSize,
                        'VERTICAL_RESOLUTION':dz,
                        'WIND_HEIGHT':z_out,
                        'UROCK_OUTPUT':'/scratch/bweeding/urock_error_tests/',
                        'OUTPUT_FILENAME':'urock_output',
                        'SAVE_RASTER':saveRaster,
                        'SAVE_VECTOR':saveVector,
                        'SAVE_NETCDF':saveNetcdf,
                        'LOAD_OUTPUT':False})

Error:

Logged critical: Traceback (most recent call last):
  File "JdbcPreparedStatement.java", line 237, in org.h2.jdbc.JdbcPreparedStatement.execute
java.lang.java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Operation does not support GeometryCollection arguments

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "JdbcPreparedStatement.java", line 237, in org.h2.jdbc.JdbcPreparedStatement.execute
Exception: Java Exception

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/__init__.py", line 534, in execute
    is_rs = self._prep.execute()
org.h2.jdbc.org.h2.jdbc.JdbcSQLNonTransientException: org.h2.jdbc.JdbcSQLNonTransientException: Exception calling user-defined function: "validGeom(POLYGON ((527331 5252398, 527332 5252398, 527333 5252398, 527333 5252399, 527334 5252399, 527335.0000000009 5252398.999999999, 527335 5252400, 527333.9999999991 5252400.000000001, 527334 5252401, 527334 5252402.000000001, 527335.0000000009 5252402, 527335 5252403, 527336 5252403, 527336 5252403.999999999, 527336 5252405, 527336 5252406, 527334.9999999991 5252405.999999999, 527336 5252408, 527337.0000000009 5252408, 527338 5252408, 527338 5252407.000000001, 527338.9999999991 5252407.000000001, 527339 5252408, 527338.9999999991 5252409, 527339.9999999991 5252409, 527342 5252403, 527340.0000000009 5252402, 527340 5252403, 527339 5252403.000000001, 527339 5252402.000000001, 527339 5252400.999999999, 527337.9999999991 5252400.000000001, 527337 5252400, 527337.0000000009 5252399, 527336 5252396, 527336.1632530484 5252396.489759147, 527331.3598316666 5252394.999999999, 527333 5252395, 527331 5252394.999999999, 527331 5252398), (527334 5252397, 527333 5252397, 527333 5252396, 527334.0000000009 5252396.000000001, 527334 5252397), (527334 5252397, 527334.9999999991 5252396.999999999, 527334.9999999991 5252398, 527334 5252398.000000001, 527334 5252397))): Operation does not support GeometryCollection arguments"; SQL statement:

        DROP TABLE IF EXISTS OPEN_VEGETATION_ZONES_ROTATED; [90105-200]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/qgis/python/plugins/processing_umep/processor/urock_processing_algorithm.py", line 448, in processAlgorithm
    MainCalculation.main(javaEnvironmentPath = javaEnvVar,
  File "/usr/share/qgis/python/plugins/processing_umep/functions/URock/MainCalculation.py", line 378, in main
    saveData.saveTable(cursor = cursor                      , tableName = vegetationOpenZoneTable,
  File "/usr/share/qgis/python/plugins/processing_umep/functions/URock/saveData.py", line 332, in saveTable
    tableName = windRotation(cursor = cursor,
  File "/usr/share/qgis/python/plugins/processing_umep/functions/URock/Obstacles.py", line 81, in windRotation
    cursor.execute(";".join(sqlRotateQueries))
  File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/__init__.py", line 536, in execute
    _handle_sql_exception()
  File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/__init__.py", line 165, in _handle_sql_exception_jpype
    reraise(exc_type, exc_info[1], exc_info[2])
  File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/__init__.py", line 57, in reraise
    raise value.with_traceback(tb)
  File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/__init__.py", line 534, in execute
    is_rs = self._prep.execute()
jaydebeapi.DatabaseError: org.h2.jdbc.JdbcSQLNonTransientException: Exception calling user-defined function: "validGeom(POLYGON ((527331 5252398, 527332 5252398, 527333 5252398, 527333 5252399, 527334 5252399, 527335.0000000009 5252398.999999999, 527335 5252400, 527333.9999999991 5252400.000000001, 527334 5252401, 527334 5252402.000000001, 527335.0000000009 5252402, 527335 5252403, 527336 5252403, 527336 5252403.999999999, 527336 5252405, 527336 5252406, 527334.9999999991 5252405.999999999, 527336 5252408, 527337.0000000009 5252408, 527338 5252408, 527338 5252407.000000001, 527338.9999999991 5252407.000000001, 527339 5252408, 527338.9999999991 5252409, 527339.9999999991 5252409, 527342 5252403, 527340.0000000009 5252402, 527340 5252403, 527339 5252403.000000001, 527339 5252402.000000001, 527339 5252400.999999999, 527337.9999999991 5252400.000000001, 527337 5252400, 527337.0000000009 5252399, 527336 5252396, 527336.1632530484 5252396.489759147, 527331.3598316666 5252394.999999999, 527333 5252395, 527331 5252394.999999999, 527331 5252398), (527334 5252397, 527333 5252397, 527333 5252396, 527334.0000000009 5252396.000000001, 527334 5252397), (527334 5252397, 527334.9999999991 5252396.999999999, 527334.9999999991 5252398, 527334 5252398.000000001, 527334 5252397))): Operation does not support GeometryCollection arguments"; SQL statement:

        DROP TABLE IF EXISTS OPEN_VEGETATION_ZONES_ROTATED; [90105-200]

---------------------------------------------------------------------------
QgsProcessingException                    Traceback (most recent call last)
Cell In[10], line 2
      1 for wd in [296.37, 256.32, 12.32,56.32, 12.68, 13.65, 245.36, 353.12, 8.05, 78.65, 77.32, 12.63]:
----> 2         processing.run("umep:Urban Wind Field: URock",
      3                        {'BUILDINGS':inputbuild,
      4                         'HEIGHT_FIELD_BUILD':buildingHeightField,
      5                         'VEGETATION':vegetationFilePath,
      6                         'VEGETATION_CROWN_TOP_HEIGHT':vegetationTopHeight,
      7                         'VEGETATION_CROWN_BASE_HEIGHT':vegetationBaseHeight,
      8                         'ATTENUATION_FIELD':vegetationAttenuationFactor,
      9                         'INPUT_PROFILE_FILE':verticalProfileFile,
     10                         'INPUT_PROFILE_TYPE':0,
     11                         'INPUT_WIND_HEIGHT':z_ref,
     12                         'INPUT_WIND_SPEED':v_ref,
     13                         'INPUT_WIND_DIRECTION':wd,
     14                         'RASTER_OUTPUT':outputRaster,
     15                         'HORIZONTAL_RESOLUTION':meshSize,
     16                         'VERTICAL_RESOLUTION':dz,
     17                         'WIND_HEIGHT':z_out,
     18                         'UROCK_OUTPUT':'/scratch/bweeding/urock_error_tests/',
     19                         'OUTPUT_FILENAME':'urock_output',
     20                         'SAVE_RASTER':saveRaster,
     21                         'SAVE_VECTOR':saveVector,
     22                         'SAVE_NETCDF':saveNetcdf,
     23                         'LOAD_OUTPUT':False})

File /opt/anaconda3/envs/qgis_bw/share/qgis/python/plugins/processing/tools/general.py:106, in run(algOrName, parameters, onFinish, feedback, context, is_child_algorithm)
     91 """
     92 Executes given algorithm and returns its outputs as dictionary object.
     93 
   (...)
    103 :rtype: Union[dict, None]
    104 """
    105 if onFinish or not is_child_algorithm:
--> 106     return Processing.runAlgorithm(algOrName, parameters, onFinish, feedback, context)
    107 else:
    108     # for child algorithms, we disable to default post-processing step where layer ownership
    109     # is transferred from the context to the caller. In this case, we NEED the ownership to remain
    110     # with the context, so that further steps in the algorithm have guaranteed access to the layer.
    111     def post_process(_alg, _context, _feedback):

File /opt/anaconda3/envs/qgis_bw/share/qgis/python/plugins/processing/core/Processing.py:176, in Processing.runAlgorithm(algOrName, parameters, onFinish, feedback, context)
    172 if not alg.validateInputCrs(parameters, context):
    173     feedback.pushInfo(
    174         Processing.tr('Warning: Not all input layers use the same CRS.\nThis can cause unexpected results.'))
--> 176 ret, results = execute(alg, parameters, context, feedback, catch_exceptions=False)
    177 if ret:
    178     feedback.pushInfo(
    179         Processing.tr('Results: {}').format(results))

File /opt/anaconda3/envs/qgis_bw/share/qgis/python/plugins/processing/gui/AlgorithmExecutor.py:70, in execute(alg, parameters, context, feedback, catch_exceptions)
     68         return False, {}
     69 else:
---> 70     results, ok = alg.run(parameters, context, feedback, {}, False)
     71     return ok, results

QgsProcessingException: Traceback (most recent call last):
  File "JdbcPreparedStatement.java", line 237, in org.h2.jdbc.JdbcPreparedStatement.execute
java.lang.java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Operation does not support GeometryCollection arguments

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "JdbcPreparedStatement.java", line 237, in org.h2.jdbc.JdbcPreparedStatement.execute
Exception: Java Exception

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/__init__.py", line 534, in execute
    is_rs = self._prep.execute()
org.h2.jdbc.org.h2.jdbc.JdbcSQLNonTransientException: org.h2.jdbc.JdbcSQLNonTransientException: Exception calling user-defined function: "validGeom(POLYGON ((527331 5252398, 527332 5252398, 527333 5252398, 527333 5252399, 527334 5252399, 527335.0000000009 5252398.999999999, 527335 5252400, 527333.9999999991 5252400.000000001, 527334 5252401, 527334 5252402.000000001, 527335.0000000009 5252402, 527335 5252403, 527336 5252403, 527336 5252403.999999999, 527336 5252405, 527336 5252406, 527334.9999999991 5252405.999999999, 527336 5252408, 527337.0000000009 5252408, 527338 5252408, 527338 5252407.000000001, 527338.9999999991 5252407.000000001, 527339 5252408, 527338.9999999991 5252409, 527339.9999999991 5252409, 527342 5252403, 527340.0000000009 5252402, 527340 5252403, 527339 5252403.000000001, 527339 5252402.000000001, 527339 5252400.999999999, 527337.9999999991 5252400.000000001, 527337 5252400, 527337.0000000009 5252399, 527336 5252396, 527336.1632530484 5252396.489759147, 527331.3598316666 5252394.999999999, 527333 5252395, 527331 5252394.999999999, 527331 5252398), (527334 5252397, 527333 5252397, 527333 5252396, 527334.0000000009 5252396.000000001, 527334 5252397), (527334 5252397, 527334.9999999991 5252396.999999999, 527334.9999999991 5252398, 527334 5252398.000000001, 527334 5252397))): Operation does not support GeometryCollection arguments"; SQL statement:

        DROP TABLE IF EXISTS OPEN_VEGETATION_ZONES_ROTATED; [90105-200]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/qgis/python/plugins/processing_umep/processor/urock_processing_algorithm.py", line 448, in processAlgorithm
    MainCalculation.main(javaEnvironmentPath = javaEnvVar,
  File "/usr/share/qgis/python/plugins/processing_umep/functions/URock/MainCalculation.py", line 378, in main
    saveData.saveTable(cursor = cursor                      , tableName = vegetationOpenZoneTable,
  File "/usr/share/qgis/python/plugins/processing_umep/functions/URock/saveData.py", line 332, in saveTable
    tableName = windRotation(cursor = cursor,
  File "/usr/share/qgis/python/plugins/processing_umep/functions/URock/Obstacles.py", line 81, in windRotation
    cursor.execute(";".join(sqlRotateQueries))
  File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/__init__.py", line 536, in execute
    _handle_sql_exception()
  File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/__init__.py", line 165, in _handle_sql_exception_jpype
    reraise(exc_type, exc_info[1], exc_info[2])
  File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/__init__.py", line 57, in reraise
    raise value.with_traceback(tb)
  File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/__init__.py", line 534, in execute
    is_rs = self._prep.execute()
jaydebeapi.DatabaseError: org.h2.jdbc.JdbcSQLNonTransientException: Exception calling user-defined function: "validGeom(POLYGON ((527331 5252398, 527332 5252398, 527333 5252398, 527333 5252399, 527334 5252399, 527335.0000000009 5252398.999999999, 527335 5252400, 527333.9999999991 5252400.000000001, 527334 5252401, 527334 5252402.000000001, 527335.0000000009 5252402, 527335 5252403, 527336 5252403, 527336 5252403.999999999, 527336 5252405, 527336 5252406, 527334.9999999991 5252405.999999999, 527336 5252408, 527337.0000000009 5252408, 527338 5252408, 527338 5252407.000000001, 527338.9999999991 5252407.000000001, 527339 5252408, 527338.9999999991 5252409, 527339.9999999991 5252409, 527342 5252403, 527340.0000000009 5252402, 527340 5252403, 527339 5252403.000000001, 527339 5252402.000000001, 527339 5252400.999999999, 527337.9999999991 5252400.000000001, 527337 5252400, 527337.0000000009 5252399, 527336 5252396, 527336.1632530484 5252396.489759147, 527331.3598316666 5252394.999999999, 527333 5252395, 527331 5252394.999999999, 527331 5252398), (527334 5252397, 527333 5252397, 527333 5252396, 527334.0000000009 5252396.000000001, 527334 5252397), (527334 5252397, 527334.9999999991 5252396.999999999, 527334.9999999991 5252398, 527334 5252398.000000001, 527334 5252397))): Operation does not support GeometryCollection arguments"; SQL statement:

        DROP TABLE IF EXISTS OPEN_VEGETATION_ZONES_ROTATED; [90105-200]

j3r3m1 commented 9 months ago

Thanks for reporting. It should now be OK using the last snapshot version (2.0.19) which has been pushed right now. Please let me know

bweeding commented 9 months ago

Ok great! I'll update and do some testing.

bweeding commented 9 months ago

Have updated and run some trials. The update seems to have removed the errors, but when I try and run it at 1m horizontal resolution, it hangs at "Identify upstreamer points in TEMPO_PRIORITIES_WEIGHT table". I've included the code below:


for wd in np.arange(270,360,0.5):
    processing.run("umep:Urban Wind Field: URock",
                   {'BUILDINGS':inputbuild,
                    'HEIGHT_FIELD_BUILD':buildingHeightField,
                    'VEGETATION':vegetationFilePath,
                    'VEGETATION_CROWN_TOP_HEIGHT':vegetationTopHeight,
                    'VEGETATION_CROWN_BASE_HEIGHT':vegetationBaseHeight,
                    'ATTENUATION_FIELD':vegetationAttenuationFactor,
                    'INPUT_PROFILE_FILE':verticalProfileFile,
                    'INPUT_PROFILE_TYPE':0,
                    'INPUT_WIND_HEIGHT':10,
                    'INPUT_WIND_SPEED':v_ref,
                    'INPUT_WIND_DIRECTION':str(wd),
                    'RASTER_OUTPUT':outputRaster,
                    'HORIZONTAL_RESOLUTION':1,
                    'VERTICAL_RESOLUTION':2,
                    'WIND_HEIGHT':2,
                    'UROCK_OUTPUT':'/scratch/bweeding/urock_error_tests/',
                    'OUTPUT_FILENAME':'urock_output',
                    'SAVE_RASTER':saveRaster,
                    'SAVE_VECTOR':saveVector,
                    'SAVE_NETCDF':saveNetcdf,
                    'LOAD_OUTPUT':False})

    cur_rx = rx.open_rasterio('/scratch/bweeding/urock_error_tests/z2_0/urock_outputWS.Gtiff')

    cur_rx.plot()

    plt.show()

    cur_rx.close()

Output:

Connecting to database
    ->/tmp/myDbH2
/usr/share/qgis/python/plugins/processing_umep/functions/URock/h2gis-standalone/h2gis-dist-2.2.1.jar

/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/__init__.py:200: DeprecationWarning: jpype._core.isThreadAttachedToJVM is deprecated, use java.lang.Thread.isAttached instead
  if not jpype.isThreadAttachedToJVM():

Connected!

SLF4J(W): No SLF4J providers were found.
SLF4J(W): Defaulting to no-operation (NOP) logger implementation
SLF4J(W): See https://www.slf4j.org/codes.html#noProviders for further details.

Spatial functions added!

Load input data
Load table 'build_pre_srid_20240227163818'
Load table 'veg_pre_srid_20240227163818'
Creates blocks and stacked blocks
Rotates geometries from 270.0 degrees
Identify block base height and block cavity base
Calculates obstacle properties
Calculates zone properties
Initializes upwind facades
Update upwind facades base height
Initializes downwind facades
Calculates study area properties
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Creates displacement zones
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Creates cavity and wake zones
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Creates street canyon zones
Rotates geometries from -270.0 degrees
Creates rooftop zones (perpendicular and corner)
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Creates built-up and open vegetation zones
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Creates the grid of points
Affects each grid point to a building Rockle zone and calculates needed 
          variables for 3D wind speed
Affects each grid point to a vegetation Rockle zone and calculates 
          needed variables for 3D wind speed
Remove some of the Röckle zone points
Creates backward zones
Calculates the 3D wind speed factor value for each point of each BUILDING zone
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Calculates the 3D wind speed factor value for each point of each VEGETATION zone
Rotates geometries from -270.0 degrees
Deals with superimposition (keeps only 1 value per 3D point)
Deals with superimposition (keeps only 1 value per 3D point)
Identify upstreamer points in TEMPO_WEIGHTING table
Identify upstreamer points in TEMPO_PRIORITIES table
Identify upstreamer points in TEMPO_PRIORITIES_WEIGHT table
Deals with superimposition (keeps only 1 value per 3D point)
Identify upstreamer points in TEMPO_WEIGHTING table
Identify upstreamer points in TEMPO_PRIORITIES table
Identify upstreamer points in TEMPO_PRIORITIES_WEIGHT table

bweeding commented 8 months ago

Interestingly the update doesn't hang when I run it in QGIS on my desktop (as opposed to a linux server in the example above)!

bweeding commented 8 months ago

I've found the code that causes my linux machine to hang, in InitWindField.py. The calls to DataUtil.createIndex all go through successfully, but the code then appears to get stuck.

    # Join the downstream weigthted points to the non downstream weighted ones
    cursor.execute("""
          {12};
          {13};
          {10};
          {11};
          DROP TABLE IF EXISTS {9};
          CREATE TABLE {9}
              AS SELECT   a.{2}, a.{3}, a.{4}, a.{5}, a.{6}, a.{7}, a.{8}
              FROM     {0} AS a LEFT JOIN {1} AS b
                       ON a.{2} = b.{2} AND a.{3} = b.{3}
              WHERE    b.{2} IS NULL
              UNION ALL
              SELECT    c.{2}, c.{3}, c.{4}, c.{5}, c.{6}, c.{7}, c.{8}
              FROM     {1} AS c
          """.format( tempoPrioritiesWeightedAllPlusBack    , tempoUpstreamAndDownstream,
                      ID_POINT                              , ID_POINT_Z,
                      HEIGHT_FIELD                          , U,
                      V                                     , W,
                      REF_HEIGHT_FIELD                      , initializedWindFactorTable,
                      DataUtil.createIndex(tableName=tempoUpstreamAndDownstream, 
                                            fieldName=ID_POINT,
                                            isSpatial=False),
                      DataUtil.createIndex(tableName=tempoUpstreamAndDownstream, 
                                            fieldName=ID_POINT_Z,
                                            isSpatial=False),
                      DataUtil.createIndex(tableName=tempoPrioritiesWeightedAllPlusBack, 
                                            fieldName=ID_POINT,
                                            isSpatial=False),
                      DataUtil.createIndex(tableName=tempoPrioritiesWeightedAllPlusBack, 
                                            fieldName=ID_POINT_Z,
                                            isSpatial=False)))

j3r3m1 commented 8 months ago

OK, quite difficult to debug when it becomes machine related. I can have a try to split this SQL query into 2 in a specific branch on my repo and then you can have a try if it solves the problem on your server. If yes we will merge to the current master.

bweeding commented 8 months ago

Ah that sounds like a good idea!

I'm very much out of my depth when it comes to the SQL/Java stuff. I the machine issue might be to do with where the temporary files were being stored on the linux machine and the transfer speeds to that drive, but that isn't the case. Have you tried running the call at 1m resolution on your machine/s?

j3r3m1 commented 8 months ago

I have not try running the code at 1 m resolution, I will try now.

Here you can also try the new version of the code on your server to see if it solves the issue: https://github.com/j3r3m1/UMEP-processing/tree/modifBenServer

bweeding commented 8 months ago

Ok great, I'll try running it!

I also noticed today when running at 2m resolution, direction 269.44 and a wind speed of 9.4m/s at 10m that it took a very long time, due to modelling going almost 7km up!

Time spent for wind speed initialization: 1873.6515927314758 s
Shape: (265, 397, 3376) - Nb cells: 355172080
10
Start to apply the wind solver
Time spent by the wind speed solver: 10153.252950429916 s

bweeding commented 8 months ago

Hi Jeremy, I've tried the new code and it's still getting stuck in the same spot, at # Join the downstream weigthted points to the non downstream weighted ones

I set the createIndex function in dataUtil.py to print the table names, and it gets to:

TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305201659
TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305201659
TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305201659
TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305201659

j3r3m1 commented 8 months ago

I also noticed today when running at 2m resolution, direction 269.44 and a wind speed of 9.4m/s at 10m that it took a very long time, due to modelling going almost 7km up!

Good catch. It might be related to issue (there is trouble with the rooftop corner scheme) https://github.com/UMEP-dev/UMEP-processing/issues/41

Is it still for the same area ? If so I should investigate this case and solve that.

bweeding commented 8 months ago

Yep I'm just using the same area.

j3r3m1 commented 8 months ago

I have the same problem than you for the endless SQL query. How long did it take on your regular labtop and what are the characteristics of your regular labtop ?

It seems that the number of cells for this area and this resolution is quite high (> 20 M grid cells) which might also explain the time spent by the SQL query (but not an endless time as it seems to be however...).

@ebocher any idea how to investigate what's wrong with the query (except that the two tables are huge) ?

Here is the SQL query:

  CREATE  INDEX IF NOT EXISTS id_ID_POINT_TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305165447 ON TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305165447(ID_POINT);;
  CREATE  INDEX IF NOT EXISTS id_ID_Z_TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305165447 ON TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305165447(ID_Z);;
  CREATE  INDEX IF NOT EXISTS id_ID_POINT_TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447 ON

 TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447(ID_POINT);;
  CREATE  INDEX IF NOT EXISTS id_ID_Z_TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447 ON TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447(ID_Z);;
  DROP TABLE IF EXISTS INITIALIZED_WIND_FACTOR_FIELD;
  CREATE TABLE INITIALIZED_WIND_FACTOR_FIELD
      AS SELECT   a.ID_POINT, a.ID_Z, a.HEIGHT_ROO, a.U, a.V, a.W, a.REF_HEIGHT
      FROM     TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305165447 AS a LEFT JOIN TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447 AS b
               ON a.ID_POINT = b.ID_POINT AND a.ID_Z = b.ID_Z
      WHERE    b.ID_POINT IS NULL
      UNION ALL
      SELECT    c.ID_POINT, c.ID_Z, c.HEIGHT_ROO, c.U, c.V, c.W, c.REF_HEIGHT
      FROM     TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447 AS c

Here are the corresponding tables, saved as CSV (do not contain a geometry field):

files_URock.zip

ebocher commented 8 months ago

Please give me the H2 database version you are using .

ebocher commented 8 months ago

and give the result of

 EXPLAIN SELECT   a.ID_POINT, a.ID_Z, a.HEIGHT_ROO, a.U, a.V, a.W, a.REF_HEIGHT
      FROM     TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305165447 AS a LEFT JOIN TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447 AS b
               ON a.ID_POINT = b.ID_POINT AND a.ID_Z = b.ID_Z
      WHERE    b.ID_POINT IS NULL
      UNION ALL
      SELECT    c.ID_POINT, c.ID_Z, c.HEIGHT_ROO, c.U, c.V, c.W, c.REF_HEIGHT
      FROM     TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447 AS c

j3r3m1 commented 8 months ago

The DB is definitely too big (1.9 Go) to be shared simply.

The result of the query is:

(SELECT
    "A"."ID_POINT",
    "A"."ID_Z",
    "A"."HEIGHT_ROO",
    "A"."U",
    "A"."V",
    "A"."W",
    "A"."REF_HEIGHT"
FROM "PUBLIC"."TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305165447" "A"
    /* PUBLIC.TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305165447.tableScan */
LEFT OUTER JOIN "PUBLIC"."TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447" "B"
    /* PUBLIC.ID_ID_Z_TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447: ID_Z = A.ID_Z */
    ON ("A"."ID_POINT" = "B"."ID_POINT")
    AND ("A"."ID_Z" = "B"."ID_Z")
WHERE "B"."ID_POINT" IS NULL)
UNION ALL
(SELECT
    "C"."ID_POINT",
    "C"."ID_Z",
    "C"."HEIGHT_ROO",
    "C"."U",
    "C"."V",
    "C"."W",
    "C"."REF_HEIGHT"
FROM "PUBLIC"."TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447" "C"
    /* PUBLIC.TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447.tableScan */)

ebocher commented 8 months ago

It seems that the index of the table TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305165447 is not used. Because H2 runs a tableScan. This bug is present in the latest release of H2. We must investigate the current snapshot.

j3r3m1 commented 8 months ago

OK I did not know about this. Good that it is a known bug.

j3r3m1 commented 8 months ago

Yep I'm just using the same area.

OK I will try to solve the issue when I will have more time.

ebocher commented 8 months ago

A workaround to fix this issue is to create a multicolumn index

 CREATE  INDEX IF NOT EXISTS id_ID_POINT_TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305165447 ON TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305165447(ID_POINT, ID_Z);
  CREATE  INDEX IF NOT EXISTS id_ID_POINT_TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447 ON
 TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447(ID_POINT, ID_Z);

j3r3m1 commented 8 months ago

Should now be solved thanks to the last commit (https://github.com/UMEP-dev/UMEP-processing/commit/fa8a4ecf2bf4d64c79f78f961daee086642b98c7). Have a try to the last version and report (close if solved)

Concerning the very high domain it is definitely related to the rooftop zone so it will be fixed in the other issue.

bweeding commented 8 months ago

Seems to be working now. However, things are running much slower than previously and generating very large databases. Partially to do with the high domains I think.

j3r3m1 commented 8 months ago

Seems to be working now. However, things are running much slower than previously and generating very large databases. Partially to do with the high domains I think.

Do you observe this phenomenon on cases you had previously run before ? Because it is supposed to be faster...

bweeding commented 8 months ago

I'll do some testing, it seems to vary a lot.

On Thu, 7 Mar 2024, 11:44 pm jeremy-b, @.***> wrote:

Seems to be working now. However, things are running much slower than previously and generating very large databases. Partially to do with the high domains I think.

Do you observe this phenomenon on cases you had previously run before ? Because it is supposed to be faster...

— Reply to this email directly, view it on GitHub https://github.com/UMEP-dev/UMEP-processing/issues/63#issuecomment-1983432833, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANYGKRG3QTY3HRIKHROKNRTYXBOLPAVCNFSM6AAAAABBXXK65SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBTGQZTEOBTGM . You are receiving this because you modified the open/close state.Message ID: @.***>

UMEP-dev / UMEP-processing

Urock: intermittant NumberFormatException #63