search

locationtech / geotrellis

GeoTrellis is a geographic data processing engine for high performance applications.
http://geotrellis.io
Other
1.33k stars 361 forks source link

Update EMR Scripts so that They Install GDAL #2874

Open jbouffard opened 5 years ago

jbouffard commented 5 years ago

geotrellis-contrib and geotrellis-gdal are two libraries that will become more used and depended upon in the future. Therefore, we should ensure that our users will be able to use these two resources without issue. One way we can do that is by updating the EMR scripts so that they install GDAL on the create EMR clusters. This would allow users to fully utilize all of the features available in geotrellis-contrib and geotrellis-gdal.

metasim commented 5 years ago

As I've discussed quite a bit with @pomadchin , the dream endstate for me would be to package GDAL bindings in a manner similar to Java's bindings to BLAS/LAPACK, OpenGL/OpenCL, etc whereby all the necessary shared libraries are bundled in a platform-specific jar archive and unpacked on the fly. I've done this in the past with VTK, and while it's no easy task to get all the dependencies linked in properly and the various platform idiosyncrasies handled, the result is a very seamless experience for the end user and avoids a lot of compatibility issues down the road.

metasim commented 5 years ago

PS: I hope my comments above weren't interpreted as meaning we shouldn't update the EMR scripts for GDAL.... short term that's the right solution.

jpolchlo commented 5 years ago

Maybe it is worth pointing out that geotrellis/geotrellis-deployments provides bare-bones scripts to create such clusters already.