Length mismatch: Expected axis has 38 elements, new values have 24 elements

[marciaamatias]

I'm running the SUEWS model within UMEP and I get this error when runngi the SUEWS simple

Length mismatch: Expected axis has 38 elements, new values have 24 elements

What length?!! Can someone help?

I checked the Supy.log and it doesn't present any information on the error

"2024-04-28 18:47:38,762 - SuPy - INFO - All cache cleared. 2024-04-28 18:47:39,549 - SuPy - INFO - All cache cleared."

[biglimp]

Hi @marciaamatias,

• Can you zip your data and attach it here for us to have a look at?
• please also clarify the versions of UMEP and QGIS you are using.

Thanks!

[sunt05]

Hello @biglimp I assume the SUEWS simple plugin would only use the default dataset shipped in UMEP, right? if so, I guest this issue is related to some upstream packages used in pre-processing the default dataset.

[biglimp]

Hello @biglimp I assume the SUEWS simple plugin would only use the default dataset shipped in UMEP, right? if so, I guest this issue is related to some upstream packages used in pre-processing the default dataset.

We will have to see when the user submit the data.

[marciaamatias]

I'm sorry I am taking so long to get you the files

But first I need to complete the problems I'm having with the Tree morphology

[marciaamatias]

So this is the input data that I have. I think that is all you need. If there is anything missing I'll send it.

files_M_1.zip

I've run the model again after I've fixed the tree morphology problem with @biglimp's help but the same error appeared.

And this is the version of QGIS and umep I'm using

UMEP: 4.0.4 UMEP for processing: 2.0.13 QGIS:

[sunt05]

Hello @biglimp, it seems the issue could be on the upstream side since the error message doesn't appear to be directly generated by SuPy itself.

---

Also, the data provided by UMEP users isn’t immediately usable for testing on the SuPy side. We need to determine whether the problem originates from SuPy or elsewhere. (This might be more of a UMEP development issue that should be branched out separately.)

[marciaamatias]

Is there anything missing I should have provided for testing on the SuPy side?

[sunt05]

No, not at the moment - Fredrik and I need to figure sth else first and update this further.

[biglimp]

Your attached data is not complete. If you want run use your own data in SUEWSSimple, you need isotrophic files from buildings, vegetation and landcover using DSM, DEM, CDSM and LandCover rasters. Please provide your isotrphic file for buildings.

[marciaamatias]

Sorry, here it is

buildings_IMPPoint_isotropic.txt

[biglimp]

Your data is inconsistent. In your land cover file, building fraction is 0.233 whereas in your building file you have a building fraction of 0.338.

Furthermore, when I look at your geodata, you lack a DEM and your DSM has tall vegetation included. Is should consist of only buildings and ground. Have a close look at the tutorial data for our London test case. There you can see how the data should be. https://umep-docs.readthedocs.io/projects/tutorial/en/latest/Tutorials/IntroductionToSuews.html#introductiontosuews https://umep-docs.readthedocs.io/projects/tutorial/en/latest/Tutorials/SuewsAdvanced.html#suewsadvanced

[marciaamatias]

So, I now have the correct DSM (only buildings)

However, the same error popped up:

So is the error because of the DSM or is it another thing that is making the length mistake appear?

[biglimp]

Not sure yet where your error come from but you still have large differences between your land cover building fraction and Building morphology fraction. When I test with your data I get this message since the data is inconsistent:

I also see that you are using the met_data from London (if the attached files are correct. That is not possible since that dataset is only valid for that location. Please clean your data and resubmit for us to test. As it is now I cannot debug your problem.

[biglimp]

@sunt05 , I was able to debug and it seem to be something wrong with the input metfile (apart from the issues with the input surface data). Do you automatically add a spinup year? I attached the data used. Can you check?

RunControl.zip

[marciaamatias]

Met_data.txt

This is the met data I'm using for the model. I've created this based on the data EPW data for Lisbon. That is the file I've sent you.

[marciaamatias]

About the difference between the DSM and the LCF

I don't understand why this is happening since the LCF for the buildings is exactly the same that is accounted in the DSM

Please see images: 1) Buildings over the DSM

2) only the DSM

3) Building grid

[biglimp]

What grid are you using as landcover?

[marciaamatias]

A raster with the classification that UMEP requires, from 1 to 5

land_cover_umep_telheiras.zip

[biglimp]

The extent of the land cover is smaller that the other data. If you use a 200 meter radius a large part of your analysis will be outside your land cover grid.

[marciaamatias]

I've moved the point and changed the search area to 130 meters, so now everything is within the area and the values do change a little bit.

However, the morphometric values for the trees I can't calculate because of this error, that I think is the same I've had before

This is the data I'm using. I'm only using the CDSM filesM.zip

And I've checked the box for the DSM (only 3D objects)

ERROR: Algoritmo 'Urban Morphology: Morphometric Calculator (Point)' a iniciar… Parâmetros de entrada: { 'FILE_PREFIX' : 'arvores', 'INPUT_DEM' : None, 'INPUT_DISTANCE' : 130, 'INPUT_DSM' : None, 'INPUT_DSMBUILD' : 'E:/Doutoramento/TESE DE DOUTORAMENTO/ARTIGOS/Artigo 2/QGIS/CDSM_telheiras.tif', 'INPUT_INTERVAL' : 5, 'INPUT_POINT' : None, 'INPUT_POINTLAYER' : 'E:/Doutoramento/TESE DE DOUTORAMENTO/ARTIGOS/Artigo 2/QGIS/ponto_interesse.shp', 'OUTPUT_DIR' : 'E:\Doutoramento\TESE DE DOUTORAMENTO\ARTIGOS\Artigo 2\QGIS', 'OUTPUT_POINT' : 'TEMPORARY_OUTPUT', 'OUTPUT_POLYGON' : 'TEMPORARY_OUTPUT', 'ROUGH' : 0, 'SAVE_POINT' : False, 'USE_DSM_BUILD' : True }

Point location obtained from point in vector layer x = 484855.7461486188 y = 4290302.393890262 Traceback (most recent call last): File "C:\Users/marci/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\preprocessor\imagemorphparmspoint_algorithm.py", line 247, in processAlgorithm immorphresult = morph.imagemorphparam_v2(dsm, dem, scale, 1, degree, feedback, 1) File "C:\Users/marci/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\util\imageMorphometricParms_v2.py", line 100, in imagemorphparam_v2 lineMid = a[0:imidy,imid] IndexError: only integers, slices (:), ellipsis (...), numpy.newaxis (None) and integer or boolean arrays are valid indices

Execution failed after 0.09 segundos

[biglimp]

I cannot replicate your error. Used your data and settings.

Try start a new project in QGIS and make sure that you use same CRS (your shapefile point has no registered CRS).

[marciaamatias]

New blank project

All layers with the same CRS - even the point layer but it failed

[biglimp]

Sorry, cannot help since i cannot replicate your error.

[marciaamatias]

can you send me the output file you got on the morphometric calculator?

so I can see if with that I can run the model?

[biglimp]

erer_IMPPoint_anisotropic.txt erer_IMPPoint_isotropic.txt

[marciaamatias]

Thank you.

It seems the problem remains in the difference between Land Cover and the DSM, manly on the vegetation difference

My question is: beeing a possitive differece means there is more vegetation in the DSM than in the Land Cover?

I've updated the Land Cover, to account for all the trees the morphometric calculator is assuming but the difference remains

[biglimp]

You have 12.2% evergreen in your landcover and 0.4% tree cover from the morphology file, i.e. no trees in your CDSM.

[marciaamatias]

Can you help me with this error the plugin is crashing on?

What GridLayoutMETDATA_NEW.nml file is UMEP referring to?

This is what is on the Supy.log

2024-05-13 15:13:48,999 - SuPy - INFO - All cache cleared. 2024-05-13 15:13:49,621 - SuPy - ERROR - C:\Users\marci\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\UMEP\suewsmodel\Input\GridLayoutMETDATA_NEW.nml does not exists! Traceback (most recent call last): File "C:\Users\marci\AppData\Roaming\Python\Python39\site-packages\supy_load.py", line 278, in load_SUEWS_nml df_nml = pd.DataFrame(parser.read(p_nml)) File "C:\Users\marci\AppData\Roaming\Python\Python39\site-packages\f90nml\parser.py", line 279, in read nml_file = open(nml_fname, 'r') if nml_is_path else nml_fname FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\marci\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\UMEP\suewsmodel\Input\GridLayoutMETDATA_NEW.nml'

[marciaamatias]

And what about this error (again) on the Land Cover Grid? I'm using new shapes, on a blank project so I make sure is not that that is making the error. I've seen that all layers have the same CRS, same size, extent and resolution. I was able to use the same grid on the Morphometric calculator and it added to the grid attribute table the values. But with the Land cover is not working.

[biglimp]

This error indicates that you have i discrapency in CRS between your land cover and vector data OR that your land cover consist of NoData values. The land cover dataset should only consist of values between 1 and 7.

[marciaamatias]

I didn't alter anything but I was able to get the values.

However: the original error for this issue is popping up again. I've checked the Supy log:

2024-05-14 11:27:11,260 - SuPy - INFO - All cache cleared. 2024-05-14 11:27:12,839 - SuPy - INFO - All cache cleared.

[biglimp]

Your new error is different. Now 25 vs 24 mismatch. This indicated that it is something wrong with the number of columns in your met forcing data.

[marciaamatias]

What abou this error?

What file is the Supy log referring?

2024-05-14 13:36:56,878 - SuPy - ERROR - C:\Users\marci\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\UMEP\suewsmodel\Input\GridLayoutMETEO1.nml does not exists! Traceback (most recent call last): File "C:\Users\marci\AppData\Roaming\Python\Python39\site-packages\supy_load.py", line 278, in load_SUEWS_nml df_nml = pd.DataFrame(parser.read(p_nml)) File "C:\Users\marci\AppData\Roaming\Python\Python39\site-packages\f90nml\parser.py", line 279, in read nml_file = open(nml_fname, 'r') if nml_is_path else nml_fname FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\marci\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\UMEP\suewsmodel\Input\GridLayoutMETEO1.nml'

[biglimp]

I need more info. When does this error occur?

[marciaamatias]

When running the SUEWS simple

Instead of the length mismatch this is what happens

[biglimp]

This is a new error? Please be more detailed in your error description and submit all data used. Otherwise we cannot help you.

[marciaamatias]

I fixed the length mismatch error. Now, this is the error appearing

The data I'm using is the same I've been using before. Nothing changed, except the met file that I've changed with your suggestions.

**The error now is: Length of new names must be 1, got 2

Check also: C:\Users\marci\Documents\SuPy.log

Please report any errors to https://github.com/UMEP-dev/UMEP/issues**

ON the supy log: 2024-05-14 13:36:56,878 - SuPy - ERROR - C:\Users\marci\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\UMEP\suewsmodel\Input\GridLayoutMETEO1.nml does not exists! Traceback (most recent call last): File "C:\Users\marci\AppData\Roaming\Python\Python39\site-packages\supy_load.py", line 278, in load_SUEWS_nml df_nml = pd.DataFrame(parser.read(p_nml)) File "C:\Users\marci\AppData\Roaming\Python\Python39\site-packages\f90nml\parser.py", line 279, in read nml_file = open(nml_fname, 'r') if nml_is_path else nml_fname FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\marci\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\UMEP\suewsmodel\Input\GridLayoutMETEO1.nml'

I understand the plugin is trying to find the GridLayoutMETEO1.nml.

So my questions are: 1) what file is the plugin referring to?? I've searched in the tutorials, in the forum and had no results. 2) In what step is this made? 3) and why is the plugin using this folder as input? Should we change at any point? This information is not given at any part of the tutorials also.

The tutorials are a great help but they lack several intermediate steps so "normal" people that are using the plugin with the QGIS software are able to use it.

This is the data I'm entering in the SUEWS simple. new_files.zip

[biglimp]

I think a know what the issue might be. Try again but dont change file prefix, i.e. keep 'kc'. Let me know what the result will be.

The GridlayoutFile is for a newer version of the SUEWSmodel. See possibly SUEWS-manual for more info. This file is not used in SUEWSSimple but it has to exist.

This folder includes many files required to run the model.

[marciaamatias]

Ok, so I've tried again, replacing the file code to Kc and this is what the model said.

On the Supy log: 2024-05-14 16:24:46,178 - SuPy - INFO - All cache cleared. 2024-05-14 16:24:46,973 - SuPy - INFO - All cache cleared. 2024-05-14 16:24:47,842 - SuPy - INFO - All cache cleared. 2024-05-14 16:24:48,549 - SuPy - INFO - SuPy is validating df_forcing... 2024-05-14 16:24:48,553 - SuPy - ERROR - Issues found in df_forcing: u should be between [0.0001, 60] but 2 outliers are found at: [47, 95] 2024-05-14 16:24:48,553 - SuPy - CRITICAL - df_forcing is NOT valid to drive SuPy!

On the main window

[biglimp]

I have no idea what this error means but it might be related to errors in your met forcing data. @sunt05 , do you have an idea about this error?

[sunt05]

2024-05-14 16:24:48,549 - SuPy - INFO - SuPy is validating df_forcing...

2024-05-14 16:24:48,553 - SuPy - ERROR - Issues found in df_forcing:

Hi @marciaamatias

It appears your forcing file contains invalid wind speed values—please refer to the log information above to debug your file. I suspect there are some zero wind speeds in your forcing file.

[marciaamatias]

@sunt05 thank you

I did have 0 values for wind speed. I replaced them with 0.0001. However, is not running. I've checked and all values are within the range from 0.0001 to 60.

This is what the supy log says

2024-05-15 08:51:03,453 - SuPy - INFO - All cache cleared.
2024-05-15 08:51:04,287 - SuPy - INFO - All cache cleared.
2024-05-15 08:51:05,113 - SuPy - INFO - All cache cleared.
2024-05-15 08:51:05,675 - SuPy - INFO - SuPy is validating `df_forcing`...
2024-05-15 08:51:05,691 - SuPy - ERROR - Issues found in `df_forcing`:
`u` should be between [0.0001, 60] but 2 outliers are found at:
 [47, 95]
2024-05-15 08:51:05,691 - SuPy - CRITICAL - `df_forcing` is NOT valid to drive SuPy!

this is the txt file I'm using METEO2.txt

[sunt05]

Please make the u values even larger to see if this error persists @marciaamatias

[marciaamatias]

Ok, I replaced the values for 0.1.

I tried to run and QGIS just shuts down completely. Is this normal?

Not just the UMEP plugin but QGIS programme. completely

[sunt05]

Sorry to hear that – it seems beyond our control.

However, I checked the forcing file. After replacing the invalid u values, it's OK with SuPy now.

[marciaamatias]

Ok. I just wanted report these two things:

1) the model shuts down in several computers; so the problem is not just in my PC. Even in a Computer, the same happens

2) back to the original problem I want to report that, using the test data the software is reporting this same error

[biglimp]

Ok. I just wanted report these two things:

1. the model shuts down in several computers; so the problem is not just in my PC. Even in a Computer, the same happens

Ok. I dont know if we can assist in this issue any more (see point 2)

2. back to the original problem I want to report that, using the test data the software is reporting this same error

I cannot replicate your error. I runs fine on my computer. What version of QGIS are your using? At the moment we are having issues with the latest version that we are trying to solve(3.36.2).