Closed sr-murthy closed 1 month ago
Do you happen to have the config.log
and test-suite.log
files generated? That would shed a lot of light on what's happening under the hood.
Those would be inside the Docker container, but I could extract that and get back to you.
@WardF Here are the files extracted from the (Ubuntu 18.04) container - I just ran the build steps inside the container and reproduced the error.
P. S. The Dockerfile I'm using is here. Along with HDF4 and HDF5, I'm also trying to build HDF-EOS2 and H4CF from source. So the (attempted) build order inside the container is:
Steps 1-3 succeed - it's step #4 (netCDF-C) that fails.
Thanks, I'll take a look at this. Do you happen to have the apt.txt
file referenced by the Dockerfile you linked to? Also, is there any chance you have the test-suite.log file for the failing tests? The one you shared shows no errors.
@WardF The apt file is here..
Yes I can see the test-suite.log
didn't contain any errors, which is why I raised the issue - the chained build step clearly failed on make check
. Would there be a separate log file make check
, and where would I look for it?
I will try it again, and see if I can find anything else that can pinpoint the make check
errors.
@WardF I believe I am running configure
correctly, but do you see any issues with that?
It looks like you are running configure
correctly, my apologies for the delay and my thanks for your patience. I am circling back around to this.
@WardF Thanks, not a problem. I did try and find other files in the container that might indicate the cause of the problem, but no luck I'm afraid.
It's not urgent.
I'm at a bit of a loss. If I build the container from a modified Dockerfile
which doesn't build netcdf, and then run the container and build netCDF using the same arguments you are using in your script, it builds fine and the tests all pass. When it is automated, they do not, for reasons I'll have to keep investigating. I also don't observe this behavior in any of the Docker images we use for regression tests. I'll keep poking at this, but no obvious answer outside of "something in Docker" is immediately suggesting itself.
Thanks @WardF. I might look at this again if I have time, but I'm a little doubtful now about any possible resolution.
I'll keep this open for the moment, but at some point I suppose I may have to close it.
@sr-murthy I was having same build issue in the Docker and realized it was because of the ftp site not reachable (it happens when it's trying to download the hdf4 files because of --enable-hdf4-file-tests
in your docker file) .
Just sharing if your issue is same ? https://github.com/Unidata/netcdf-c/issues/2951
Note: the issue mentioned the netcdf v4.7.x but I had same issue building 4.9.2
If above shared issue is the case you need following in your Dockerfile (after configure and before make check)
# Alternate fix for using https instead of ftp until netcdf-c library doesn't release officially
# ftp no longer works
&& sed -i 's~ftp://ftp.unidata.ucar.edu/pub/netcdf/sample_data/hdf4/$1.gz~https://resources.unidata.ucar.edu/netcdf/sample_data/hdf4/$1.gz~g' hdf4_test/run_get_hdf4_files.sh \
&& sed -i 's~FTPFILE~DATAFILE~g' hdf4_test/run_get_hdf4_files.sh
@aafaque33 I can't see an FTP related error in the original build failure - from the logs it appears to be an ncgen3-related test failure.
Agreed, while the change in the ftp site was responsible for some errors, this one is recreatible on my end but only when building the image. If I modify the dockerfile to install everything before netCDF, and then run the dockerfile and install netCDF manually, the test failures do not occur (at least, in my local environment).
I'm leaning towards closing this out because it does not appear that the issue is with netCDF itself, but something in the docker environment.
I've created a modified Dockerfile, Dockerfile.slim
, and a companion script file, script_build.sh
. All of the provisioning has been offloaded into the bash script; when I build an image from this Dockerfile, connect to it, and invoke script_build.sh
, everything completes successfully except h4cflib
. Right now the error message is Can't link against gctp library in hdfeos2 library
, but that happens after the successful build and test of libnetcdf
, so I will leave it to you to investigate.
I have no idea why this provisioning fails when it is baked into the Dockerfile, but I feel that this demonstrates it is not something with netCDF.
I've attached the files here; I'd be curious to hear what happens on your end @sr-murthy. My process was pretty straightforward:
$ docker build -t tmp -f Dockerfile.slim .
$ docker run --rm -it tmp bash
(inside the running container) $ ./script_build.sh
I hope this helps! I'm going to close out this issue, but I'll gladly reopen if it so transpires that there is still a demonstrable issue with netCDF.
Thanks @WardF. It seems possible from what you're saying that the issue is not related to netCDF, but Docker. I will look at this again to see if I've missed anything. In any case, I will also try out your Dockerfile build, with an updated version of Docker.
I would mention that my recent version of the Dockerfile which led to this issue is more or less identical to the previous version 5 years ago, when everything worked. Obviously, since then we have had newer versions of all of the libraries I'm attempting to build, as well as newer versions of Docker, but I didn't expect this to change the build result.
I'm trying to build a Docker image (Ubuntu 18.04) with netCDF-C 4.9.2 from source - I'm following the installation guide for building it with HDF4 support. In the Dockerfile here are the chained instructions for the netCDF build:
The error occurs during
make check
:Earlier instructions build HDF4 (4.2.16) and HDF5 (1.14.4-2) from source, successfully. Here are the chained instructions for those:
My guess is there is some inconsistency in the way in which
configure
andmake
have been run for HDF4 vs netCDF, but I'm not sure.P. S. Of course 18.04 is deprecated, but I need to build one for it.