search

UMEP-dev / UMEP

Urban Multi-scale Environmental Predictor
https://umep-docs.readthedocs.io/
59 stars 15 forks source link

Shadows not generated for canopy height of 1 m (Tree Height 3 m, Trunk space 2 m) #306

Closed rarygit closed 2 years ago

rarygit commented 3 years ago

QGIS version: 3.16.4-Hannover UMEP plugin v3.17.2 UMEP Processing Plugin v1.3

Having used both SOLWEIG (UMEP) and Shadow Generator (UMEP Processing): shadows are not properly generated for trees whose canopy height is 1 m. This impacts the Tmrt values for trees whose canopy height is only 1 m.

Attached is zip file containing the log file for UMEP Processing / Shadow Generator, the cdsm, tdsm and shadow raster files, outline shapefile and various screenshots. Areas of concern are outlined in red. Within this red outline you can see shadows for pixels whose canopy heights were changed from 3 m to 4 m using the Serval plugin.

If the canopy height is changed to 2 m by increasing the tree height to 4 m, the shadow generation works OK.

The first objective is to change canopy heights and transmissivity to match pergola and tree canopies recorded in urban centres. For example, most pergola canopy heights for shading are usually <= 1 m. Alternatively trees pruned into an umbrella shape with high LAI, low transmissivity, 1 m canopy height and 2 mH trunk space. In these situations the 2 m canopy height does not accurately represent the actual canopy height.

Thanks for any suggestions.

Shadow_Generator_20210716.zip

shadows_only

shadows_ _cdsm_tree locations

Crown_Height

biglimp commented 3 years ago

I assume when you write canopy height you mean crown height? This seems to be a rounding issue somewhere in the code. I will check when I find the time.

biglimp commented 3 years ago

Can you please also attach your DSM

rarygit commented 3 years ago

Thanks for replying!

Yes, crown height

I temporarily circumvented the problem by reducing the trunk height for the "pergola" structures to 1 m H. Attached are both the cdsm and tdsm files.

You can use the Serval plugin to change those "rectangular pergola" trunk heights to 2 m, thereby reducing the pergola crown height to 1 m.

RASTER_Trees.zip

biglimp commented 3 years ago

Nevertheless, your first example should be possible.

Could you also send me your DSM so that I can try.

rarygit commented 3 years ago

Sure, here are all the relevant files attached. I tested them just now with UMEP Daily Shadow Pattern

You can see the differences in shadows, e.g. 14:00 between the two sets of DSM files.

The DSM "fail" folder contains the 2 m trunk height The other folder has trunk height at 1 mH and works OK. But the trunk height is impractical ...

I have also included other files, if it helps to make sense with another simulation. One of the pergolas sits on top of a building terrace. The rest sit on the ground.

The simulation where the error occurred is back in time; the specific earlier tdsm file is now one of several, consigned to a 'fail' folder.
However the pair of DSM files, tsdm.tif and cdsm.tif, repeat the error.

I would be very interested in the steps you take to debug this. Earlier I had a look at the python scripts, but ran out of ideas how to analyse the problem.

Thanks!

Test_CrownHeight.zip

biglimp commented 3 years ago

Just to let you know that I have not forgot about this.

Now I think I know what is happening. Now I just need to try to find a solution.

It has actually nothing to do with the fact that the crown height is 1 meter or less. It has to do with that during one iteration of the shadowcasting algorithm, both the cdsm and the tdsm becomes negative AND that it is not the first iteration cos for that I have a solution. See e.g. Lindberg and Grimmond for details on how the algorithm is working. Also, I am testing this with the function shadowingfunction_20 found in Utilities/shadowingfunctions.py. Below you see an example from a cutout using your data. As I change the sun altitude (50,60,70), the shadows from the pergola appears and disappears (I use your DEM as DSM so that no building shadows are generated:

50deg 60deg 70deg

A 50 degrees, tdsm and cdsm becomes negative in different iterations At 60, negative at the same time At 70, tdsm goes negative at first iteration (special case fixed in line174)

I will think of a solution...

rarygit commented 3 years ago

Thanks for replying! I was away for the week and returned yesterday.

I will follow along with your description and the function shadowingfunction_20 found in Utilities/shadowingfunctions.py

biglimp commented 3 years ago

Well, I might have solved this. This algorithm was written long time ago and it took some time to get into the code again. I would be very grateful if you could test the code before I release it. I have pushed UMEP to a branch called PergolaShadows where the new shadow algorithms are included.

rarygit commented 3 years ago

Thanks for working on this!

It failed the first crown height test (i.e. height = 1 m) for the pergolas, as attached. I think there are shading artifacts associated where trunk height = 0.5

CrownHeight_Test1.zip

Test1_Result_1200PM_cdsm_Fail Test1_Result_1200PM_tdsm_Fail

biglimp commented 3 years ago

I dont see where it has failed. The pergolas at the top has shadows?

rarygit commented 3 years ago

The tdsm and cdsm shapefiles (in red outline) show the location of the rectangular pergolas. The pergolas are located at bottom RHS. Are are looking at the rectangular building shadows at the top?

I can send the building shapefile as well, if you need it? I thought the cdsm and tdsm outlines would be easier to detect where the shadows have artifacts.

rarygit commented 3 years ago

There are also some small trees and small pergolas elsewhere, that have a crown height = 1 m

The comparatively larger pergolas are easier to detect. But they are not located at the top of these images.

biglimp commented 3 years ago

There is a building there according to the DSM you send before.

image

rarygit commented 3 years ago

Yes, so in the images I sent there shouldn't be a 1 m wide strip of shadows missing or cut-out from the shading.
I will show the place on your image.

With Buildings_Pergolas_Shown

rarygit commented 3 years ago

And here is a comparison with the crown height = 2 m (it is a slightly modified pergola outline).

Even though the pergola outlines are different, I am seeing a strip cut away from the shading. Where crown height = 2 m, I am seeing a relatively solid black shading where the trunk height = 0.5 m

Not_This This_(1200PM_crown height 2 m)

biglimp commented 3 years ago

Ok, Now I see. I will investigate.

rarygit commented 3 years ago

As above, which is the bottom of the original image I posted, slightly to the right. Can you load the tdsm and cdsm shapefiles and check the shadows?

Three quasi blue arrows for the location:

three blue arrows

biglimp commented 3 years ago

One question before I move on. Do you tick in "facade shadow"?

rarygit commented 3 years ago

Ok, I generated these shadows using SOLWEIG and not the Daily Shadow Pattern. But your question made me check again ;- )

biglimp commented 3 years ago

Just push a new version to the PergolaShadows branch. Please try it out. I have to think if this is the way to go. What this version will do is, that in some cases you will see one pixel longer shadows from some vegetation units. This is a philosophical issue. Before I always produced shadows that were rounded downwards i.e., shorter shadows by one pixel. Now it is the other way around...

biglimp commented 3 years ago

Hold on. There is something strange with the facade version (solweig). I will look into it. In the meantime, you can test the DailyShadowPattern without facade shadows

rarygit commented 3 years ago

Your new version pushed today I call test2 - and it seems to be working much better than yesterdays test1 version. What do you think?

I have attached the Daily Shadow Pattern results (no facade shadows) as zip files.

DailyShadowPattern_test1.zip DailyShadowPattern_test2.zip

Test2 (today's push version) screenshot:

DailyShadowPattern_test2

Test1 (yesterday's push version) screenshot:

DailyShadowPattern_test1

biglimp commented 3 years ago

Ok, now it should be fixed so that facade shadows works also.

rarygit commented 3 years ago

Great, thanks for that!

biglimp commented 3 years ago

Yes, now it looks better.

biglimp commented 3 years ago

Please try some different cases and see if you find some strange behaviour

rarygit commented 3 years ago

I will -- now I take a break and test it later when I come back.

rarygit commented 2 years ago

Testing code: https://github.com/UMEP-dev/UMEP/tree/PergolaShadows

There seems to be slightly strange behaviour between paired time intervals, where the tree shadows change but not the pergola shadows, which remain fixed.

In particular 12:00 - 12:30 and 10:00 - 10:30

At some time periods, the pergola shadows have individual pixels which remain unshaded. These might be/not artifacts.

I have attached a subset of the shadow tiff output from Solweig (Date: 2015, DOY 218, 09:00 - 13:30. T = 10 min intervals) Along with some screenshots showing where the pergola shadows don't change.

Subset_2015_DOY_218.zip

biglimp commented 2 years ago

As this is a raster-based method and it could probably happen, that shadows do not change where other tree shadows change since the pergolas are close to ground and movement of the shadow is small.

rarygit commented 2 years ago

Then we can ignore these artifacts as trivial in terms of the POI output; and will the testing code: https://github.com/UMEP-dev/UMEP/tree/PergolaShadows will be merged into master?

In the interim I will test the pergola method on a Tutorial dataset and will only report back if there is a significant anomaly.

Thanks for the work to fix this! Should we take this as a solution for now?

biglimp commented 2 years ago

Yes! I will include it in the next update of the model. Thanks for your detailed analysis of this issue.