PostGIS Support

[banesullivan]

We need an elegant solution to loading geo-referenced datasets from a PostGIS database into ParaView.

This issue will be a place to discuss what support for PostGIS will look like.

[fghorow]

It's not clear that it is elegant but creating an OGR .vrt file might do the trick if it were hooked up with Paraview's/Vtk's new-ish GDAL/OGR driver. (Assuming that said driver can be coerced into reading .vrt files, that is.) Maybe using this approach, PVGeo could support some kind of GUI front-end for such an operation?

[banesullivan]

Thanks for pointing this out @fghorow! Could you provide an example OGR .vrt file or point me to a webpage where I might be able to find a sample dataset to try this out?

[fghorow]

This appears to be an example: http://cartometric.com/blog/2014/09/24/ogr-vrt-connect-to-postgis-datasource/ I have not tested it, so can't verify that it works, but it looks promising. Here's the doc for the driver itself: https://www.gdal.org/drv_vrt.html

[fghorow]

Also, at the risk of stating the incredibly obvious, a solution going through GDAL/OGR would open up all of that library's extensive collection of file formats for import and some for export too if that is meaningful...

[banesullivan]

Thank you for sharing these resources! At the moment, I'm realizing that in order to use the GDAL/OGR drivers, a user would have to compile ParaView from source with support for those libraries. This is something I haven't been able to do and wouldn't expect a new user to do either.

On this note of how do we bring together datasets/formats from various software libraries into one visualization framework, most of our users are shifting towards using another project we've been developing for visualization to replace ParaView - vtki: A Streamlined Python Interface for the Visualization Toolkit. It's basically ParaView but in a Pure python environment! This avoids the need for ParaView all together and enables a user to let anaconda handle all the dependencies.

I'm curious about a few things from you @fghorow:

1. Are you familiar with any Python libraries for reading PostGIS databases?
2. How would you feel about using a Python-based visualization tool like vtki instead of ParaView?

We're rapidly shifting towards only using vtki as this allows us to connect all the various Python libraries out there for file IO and data access rather than building our own tools that are compatible with custom ParaView builds. If there are already established libraries for reading PostGIS databases and GDAL file formats, then we should leverage those libraries. This would enable us to focus on building the link between GDAL and VTK rather than the nuances of file IO for an incredibly sophisticated format/software suite.

[fghorow]

I’ve just successfully compiled Paraview, mostly to enable the new-ish GDAL driver (per the info on the PV discourse site, which I see you are also on, Bane: https://discourse.paraview.org/t/how-to-load-data-with-gdal/1184/4 ). I agree that it is not something you would want to force on a “normal” user. That being said, perhaps we could agitate for Kitware to actually enable this on the binary distribution builds.

I have written code interfacing my own (research) Python code with PostGIS, mostly through sqlalchemy and its spatial extension geoalchemy-2. In general, it’s not too terrible. But again, it is forcing a PostGIS solution, when a GDAL solution enables access to far more files.

I have not come across vtki before. Thanks for pointing that out! It looks interesting. For another pure Python approach to 3D viz, have a look at mayavi…

I’ve been using Paraview, VisIT, and mayavi for “quick and dirty” viz for quite some time. Yes, they are huge and complex, but then again so is OpendTect.

I don’t think there is a one-size-fits-all solution to this problem. However, about 20 years ago I was involved in a geo file data translator project (which ultimately died — long story). The main architectural lesson I learned from that experience is that one should solve the 2N problem of i/o to a library with a decent internal design, rather than the N**2 problem of all projects interacting with all other projects. GDAL/OGR certainly rings that 2N bell in the GIS-like spatial data space, and the project appears to have reasonable legs. (I’ve been using gdal for about a decade or so.)

I don’t have many other reactions, but am happy to answer other queries.

On Mar 10, 2019, at 9:22 PM, Bane Sullivan notifications@github.com wrote:

Thank you for sharing these resources! At the moment, I'm realizing that in order to use the GDAL/OGR drivers, a user would have to compile ParaView from source with support for those libraries. This is something I haven't been able to do and wouldn't expect a new user to do either.

On this note of how do we bring together datasets/formats from various software libraries into one visualization framework, most of our users are shifting towards using another project we've been developing for visualization to replace ParaView - vtki: A Streamlined Python Interface for the Visualization Toolkit. It's basically ParaView but in a Pure python environment! This avoids the need for ParaView all together and enables a user to let anaconda handle all the dependencies.

I'm curious about a few things from you @fghorow:

• Are you familiar with any Python libraries for reading PostGIS databases? • How would you feel about using a Python-based visualization tool like vtki instead of ParaView? We're rapidly shifting towards only using vtki as this allows us to connect all the various Python libraries out there for file IO and data access rather than building our own tools that are compatible with custom ParaView builds. If there are already established libraries for reading PostGIS databases and GDAL file formats, then we should leverage those libraries. This would enable us to focus on building the link between GDAL and VTK rather than the nuances of file IO for an incredibly sophisticated format/software suite.