Please provide the desired output for downscalling_example04

[dharanidharatdd]

I'm currently exploring the kalpana module and i seem to be getting a lot of errors while executing downscalling_example04.py, please provide the desired output or atleast make it a notebook file

[tomas19]

Hey, I maintain this repo in my spare time, and unfortunately, I don't have much. I plan to update the downscaling scripts to notebooks, but it will take me some time.

Can please provide more info about the error messages you are getting?

[dharanidharatdd]

I have successfully performed the downscaling headloss method using Kalpana. For this, I used the NLCD 2016 Land Cover (CONUS) dataset, which I downloaded from mlrc.gov. However, the file is quite large, around 30GB, so I attempted to extract just the North Carolina portion. This is where the error occurred. If possible, could you please tell me if there is a correct way to extract a specific part of the land cover data?

Additionally, I've noticed that the .shp file generated by the static downscaling does not correspond to the actual location. It seems there's a discrepancy between the expected and actual geographic coordinates. Any guidance on resolving these issues would be greatly appreciated.

Thank you for your assistance.

I'm adding the screenshots of the geotiff and shp files.

[tomas19]

I've never run the downscaling with a subset of the land cover dataset. Some possible issues that might arise are that the land cover may not cover the entire downscaling dem or that the format you saved the cropped land cover dem is incompatible with how Kalpana reads it. What format did you save it?

Regarding the .shp file, are you sure you are using the correct crs? Maybe the output crs you specify does not match the subdomain crs or the downscaling dem.

[dharanidharatdd]

I used raster extraction tool in QGIS to extract the data and I used the downscaling DEM layer as reference, so it can be of the same size and i exported it as .img format. Did you perform the downscaling using the entire CONNUS landcover data or just the NC landcover data?

Changing the CRS fixed the issue with .shp file, so thank you.

Also performing downscaling_example04 to calculate the cost surface took 7hrs, does it take that long? can you please share the time it took for you?

[tomas19]

I typically use the full landcover dataset. Can you send me a screenshot of the error message please?

Tomas

On Wed, May 22, 2024 at 11:13 PM Dharanidhara Chowdary Tammina < @.***> wrote:

I used raster extraction tool in QGIS to extract the data and I used the downscaling DEM layer as reference, so it can be of the same size and i exported it as .img format. Did you perform the downscaling using the entire CONNUS landcover data or just the NC landcover data?

Changing the CRS fixed the issue with .shp file, so thank you.

— Reply to this email directly, view it on GitHub https://github.com/ccht-ncsu/Kalpana/issues/101#issuecomment-2126152789, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD75PEIRZ5CH6BW4BD3EHUTZDVNGPAVCNFSM6AAAAABH4KWEAGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRWGE2TENZYHE . You are receiving this because you commented.Message ID: @.***>

-- Tomás

[dharanidharatdd]

hgridshift: could not find required grid(s). pipeline: Pipeline: Bad step definition: proj=hgridshift (File not found or invalid) hgridshift: could not find required grid(s). pipeline: Pipeline: Bad step definition: inv (File not found or invalid)

These are the errors that I have noticed in downscalling_example04.

Traceback (most recent call last): File "/home/kalpana/downscaling_example04.py", line 42, in preCompCostSurface(grassVer, createGrassLocation, pathGrassLocation, pathRasFiles, rasterFiles, epsg, manningRasPath, manningLandCover, File "/home/kalpana/downscalingHeadLoss.py", line 284, in preCompCostSurface costSurface(res, gs, slopeFactor, walkCoeefs, URConstant) File "/home/kalpana/downscalingHeadLoss.py", line 176, in costSurface pkg.run_command('r.walk', elevation="dem", friction="unitHeadLoss", File "/usr/local/grass/etc/python/grass/script/core.py", line 541, in run_command return handle_errors(returncode, result=None, args=args, kwargs=kwargs) File "/usr/local/grass/etc/python/grass/script/core.py", line 429, in handle_errors raise CalledModuleError(module=module, code=code, returncode=returncode) grass.exceptions.CalledModuleError: Module run r.walk --o --q elevation=dem friction=unitHeadLoss output=walkValsFromLoop start_raster=waterFinal stop_coordinates=1977400.0,35000.0 slope_factor=-0.2125 walk_coeff=0,1,-1,-1 ended with an error. The subprocess ended with a non-zero return code: -9. See errors above the traceback or in the error output.

This is the error that I faced when using the extracted data

Can you also specify the exact outputs I have to look at for both static and head-loss, so I can compare them ?

[tomas19]

I have no idea, I've never seen this error. I'll try to run Kalpana with a subset of the landcover dataset next week and let you know.

What do you mean by the exact outputs? The outputs of both downscaling methods are rasters with the peak total water level on a higher resolution than the ADCIRC mesh. The output of the static method should be more conservative, i.e. the water should be expanded further away inland.

Best Tomas

[dharanidharatdd]

I'm very new to this GIS, and other geo-related stuff. I'm just running this software so what I meant was, are final output (.tif) files? Example: maxele_florence_downscaled.tif Or something else?

Thanks for your patience and this final clarity will be greatly appreciated.

Also I ran static downscaling on my personal computer running ubuntu.

downscaling_example01.py just ran fine. But example 02 crashed after running: downscaling:postProcessStatic:661- Delete ground level. Upon monitoring my system, it seems it is using high memory and terminating itself. Can this be run only on HPC's? Or is there a way that it can be run without termination?

[tomas19]

Yes the outputs are rasters (.tif files).

Yes, the code is very memory-intensive. I have never tried to run it on a computer with less than 16GB of RAM. The code has multiple operations involving various copies (in memory) of the downscaling raster (already a couple of GB). Try to run it on a smaller area or try resampling the downscaling DEM to 25 or 50m to see if it works fine.

[dharanidharatdd]

Thank you so much for the reply.

[dharanidharatdd]

Can I resample DEM using kalpana or should I use grass for this process ?

[tomas19]

You can specify the resolution in Kalpana in theory, but I typically do it in QGIS beforehand. I think you should use QGIS to prepare the data so you can look it over before running Kalpana.