search

geodesign / django-raster

Django-raster allows you to create tiled map services (TMS) and raster map algebra end points for web maps. It is Python-based, and requires GeoDjango with a PostGIS backend.
BSD 3-Clause "New" or "Revised" License
96 stars 39 forks source link

How to deal with layers consisting of multiple input rasters? #50

Open smari opened 4 years ago

smari commented 4 years ago

Hello!

This is a pretty great package, but I seem not to be able to put multiple input rasters into the same RasterLayer. This would be useful for cases such as loading input tiles from satellite imagery of the entire Earth without first stitching them all together into one jumbo file (with associated memory requirements). Before I go ahead and add such functionality, I wanted to check if I'm missing some obvious way of doing this? It does seem like I could just give multiple layers the same name, but that seems like a dirty hack (and I haven't checked how the views treat that case).

What's the correct approach? Or do I just have to write my own?

Thanks!

yellowcap commented 4 years ago

Hi Smari This is a good use case, and its true that the package does not support this out of the box at the moment. Creating multiple layers would be possible, but very inefficient. The lookup on the view for the tiles is based on layer ID's and so creating multiple layers won't help much. So you could create multiple separate layers and add a lot of layers url endpoints to your visualization map (one per "region" so to speak). But that will be very inefficient and will trigger a lot of requests when browsing.

The raster package can handle pretty big files, so the jumbo file approach might work. But as you mentioned, this requires quite a large server and might be impossible at a global scale, depending on the resolution of your images.

If you want to contribute I would propose to make changes in the RasterLayerParser class

https://github.com/geodesign/django-raster/blob/adba9c79d44ee48c636f017331afe62bc5fd7b50/raster/tiles/parser.py#L26

It is used here to create the tiles:

https://github.com/geodesign/django-raster/blob/adba9c79d44ee48c636f017331afe62bc5fd7b50/raster/tasks.py#L12

An "add to rasterlayer" mode instead of "completely new rasterlayer" could be added. This is not a small task though. You could also try to create another parser just for "adding tiles" to a layer, and then hook it into the admin and celery as a separate task.

yellowcap commented 4 years ago

Maybe another relevant comment here is to remember that this package stores rasters directly in a PostGIS database. So the rasters are "in-db" not "out-of-db". This means that if you want to store a huge raster, you might need a huge database as well. So factor that in when considering the ingestion of global data. Storing many terrabytes of data in PostGIS is possible but has its own challenges as well.

jonathanmiller2 commented 3 years ago

I need this feature as well. I'm working with MODIS data, which is on a sinusoidal grid like so. This projection is very common in environmental science, as NASA uses it for their data products. The reprojection isn't a huge issue, but not being able to input the individual tile data as-is is a big roadblock.

Hopefully the jumbo-file approach will work.