Open vincentsarago opened 4 years ago
@vincentsarago I don't see anything relevant in that link.
Planning on making a minor release this month, could you provide some more info here?
🤦♂ yeah link is not the good one
here is my current compilation setting:
# szip (for hdf)
RUN mkdir /tmp/szip \
&& curl -sfL https://support.hdfgroup.org/ftp/lib-external/szip/$SZIP_VERSION/src/szip-$SZIP_VERSION.tar.gz | tar zxf - -C /tmp/szip --strip-components=1 \
&& cd /tmp/szip \
&& CFLAGS="-O2 -Wl,-S" CXXFLAGS="-O2 -Wl,-S" ./configure --prefix=$PREFIX --disable-static \
&& make -j $(nproc) --silent && make install && make clean \
&& rm -rf /tmp/szip
# libhdf4
RUN mkdir /tmp/hdf4 \
&& curl -sfL https://support.hdfgroup.org/ftp/HDF/releases/HDF$HDF4_VERSION/src/hdf-$HDF4_VERSION.tar | tar xvf - -C /tmp/hdf4 --strip-components=1 \
&& cd /tmp/hdf4 \
&& CFLAGS="-O2 -Wl,-S" CXXFLAGS="-O2 -Wl,-S" ./configure \
--prefix=$PREFIX \
--with-szlib=$PREFIX \
--enable-shared \
--disable-static \
--disable-netcdf \
--disable-fortran \
&& make -j $(nproc) --silent && make install && make clean \
&& rm -rf /tmp/hdf4
# libhdf5
RUN mkdir /tmp/hdf5 \
&& curl -sfL https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-${HDF5_VERSION%.*}/hdf5-${HDF5_VERSION}/src/hdf5-$HDF5_VERSION.tar.gz | tar zxf - -C /tmp/hdf5 --strip-components=1 \
&& cd /tmp/hdf5 \
&& CFLAGS="-O2 -Wl,-S" CXXFLAGS="-O2 -Wl,-S" ./configure \
--prefix=$PREFIX \
--with-szlib=$PREFIX \
--enable-cxx \
--enable-thread-safe \
--disable-static \
&& make -j $(nproc) --silent && make install && make clean \
&& rm -rf /tmp/hdf5
# NetCDF
RUN mkdir /tmp/netcdf \
&& curl -sfL https://github.com/Unidata/netcdf-c/archive/v$NETCDF_VERSION.tar.gz | tar zxf - -C /tmp/netcdf --strip-components=1
RUN cd /tmp/netcdf \
&& CPPFLAGS="-I${PREFIX}/include" LDFLAGS="-L${PREFIX}/lib" \
./configure \
--with-default-chunk-size=67108864 \
--with-chunk-cache-size=67108864 \
--prefix=$PREFIX \
--disable-static \
--enable-netcdf4 \
--enable-dap \
--with-pic \
&& make -j $(nproc) --silent && make install && make clean \
&& rm -rf /tmp/netcdf
The important bits are --with-default-chunk-size
and --with-chunk-cache-size
options for netcdf that will speed HDF/NETCDF reads.
Right now the NetCDF/HDF driver are compiled with default options which are not really optimized for GDAL.
In this thread there are some info about how to make NetCDF reads faster https://trac.osgeo.org/gdal/timeline?from=2015-07-17T12%3A04%3A27-07%3A00&precision=second
@matthewhanson I'll open a PR to implement the change when I have time