HDF5 problem.

[paullglebrun]

Hi Andrew,

I am trying to install Galacticus from source code, such that I can learn from the code a bit of Astrophysics.. I am running on a up to date CentOs9 system. After installing, as root, normal install, using usual yum of dnf, most dependency packkages (except qhull..for some unknown reason, it got re-install in galacticus-master/galacticusInstallWork), the script ran fine, but stop on F90 compiler error, handling the hdf5 package. The relevant part of the log file is:

building Galacticus --> make -j1 Galacticus.exe gfortran -c ./work/build/utility.IO.HDF5.p.F90 -o ./work/build/utility.IO.HDF5.o -ffree-line-length-none -frecursive -DBUILDPATH=\'./work/build\' -J./work/build/moduleBuild/ -I./work/build/ -fintrinsic-modules-path /home/lebrun/Galacticus/Tools/finclude -fintrinsic-modules-path /home/lebrun/Galacticus/Tools/include -fintrinsic-modules-path /home/lebrun/Galacticus/Tools/include/gfortran -fintrinsic-modules-path /home/lebrun/Galacticus/Tools/lib/gfortran/modules -fintrinsic-modules-path /usr/local/finclude -fintrinsic-modules-path /usr/local/include/gfortran -fintrinsic-modules-path /usr/local/include -fintrinsic-modules-path /usr/lib/gfortran/modules -fintrinsic-modules-path /usr/include/gfortran -fintrinsic-modules-path /usr/include -fintrinsic-modules-path /usr/finclude -fintrinsic-modules-path /usr/lib64/gfortran/modules -L/home/lebrun/Galacticus/Tools/lib -L/home/lebrun/Galacticus/Tools/lib64 -pthread -Wall -fbacktrace -ffpe-trap=invalid,zero,overflow -fdump-core -O3 -ffinite-math-only -fno-math-errno -fopenmp -g -DPROCPS -DOFDAVAIL -DFFTW3AVAIL -DANNUNAVAIL -DQHULLUNAVAIL -DMATHEVALUNAVAIL 2>&1 | ./scripts/build/postprocess.pl ./work/build/utility.IO.HDF5.p.F90 source/utility.IO.HDF5.F90; line 454 [preprocessed line 543]; code 43

543 | & H5T_NATIVE_DOUBLE, H5T_NATIVE_INT, H5T_NATIVE_INT_8, H5T_STD_I32BE , & | 1 Error: Symbol ‘h5t_native_int’ referenced at (1) not found in module ‘hdf5’ source/utility.IO.HDF5.F90; line 454 [preprocessed line 543]; code 59

543 | & H5T_NATIVE_DOUBLE, H5T_NATIVE_INT, H5T_NATIVE_INT_8, H5T_STD_I32BE , & | 1 Error: Symbol ‘h5t_native_int_8’ referenced at (1) not found in module ‘hdf5’ source/utility.IO.HDF5.F90; line 1342 [preprocessed line 1572]; code 90

....

No idea why H5T_NATIVE would cause problems.. The version of hfdf5 is

root@syn-2603-7080-ff01-8658-0000-0000-0000-1b5a lebrun]# yum info hdf5-devel Last metadata expiration check: 1:51:03 ago on Fri 05 Jul 2024 12:58:41 PM EDT. Installed Packages Name : hdf5-devel Version : 1.12.1 Release : 7.el9.1 Architecture : x86_64 Size : 3.9 M Source : hdf5-1.12.1-7.el9.1.src.rpm Repository : @System From repo : epel Summary : HDF5 development files URL : https://portal.hdfgroup.org/display/HDF5/HDF5 License : BSD Description : HDF5 development headers and libraries.

[root@syn-2603-7080-ff01-8658-0000-0000-0000-1b5a lebrun]#

Thanks

Paul

[abensonca]

Hi Paul. This is a known problem with v1.12 of HDF5 - I have an issue open to fix this but it hasn't been a high priority unfortunately.

[abensonca]

I will probably need to update the installer to avoid using this version and install an older version instead.

[abensonca]

I won't be able to work on this until next week, but you could try editing line 787 of the installation script:

maxVersion[$iPackage]="9.9.9"

Change that to:

maxVersion[$iPackage]="1.10.99"

and it should try installing an older version from source.

[paullglebrun]

Hi Andrew,

Thank you for your prompt reply!  This helps, but I am not out of the woods yet..

As, you suggested, I revert to an older version of HDF5, by making the change you recommended in the installation script. For some reasons, it found and installed, from source, the version 1.8.20 of HDF5.

I was able to run, and make trivial modification to an example in the fortran version of HDF5, so, I presume this HDF5 is as good as any for our purpose.

But the error remains...Same error as before, in fact. I suspec the pre-processing step is a fault here. I checked the I could insert in a short example

integer(kind=HID_T ), dimension(5) :: H5T_NATIVE_INTEGERS

or DOUBLES.... The generic type (or key word?) H5T_NATIVE is understood..

Poking around, I also found that you use ISO_Varying_string , a module that has been taken out of FORTRAN 90, about a decade ago, reading what I could find on the web....

Unfortunately, I don't know what I am talking about! I am definitely not a Fortran 90 user... My experience with FORTRAN dates back from the early 80's. The last time I touched FORTRAN when I was asked to translate an old FORTRAN 77 code to C++. I do recall the lengthy discussion in HEP whether to go for FORTRAN90 or adopt C++, and, as you know, we went for C++. Looking back, and looking at the code I am trying to install, frankly, I don't regret it!... At all. Even the use of HDF5 is bit outdated, as the computing industry has _finally - standardized on a standard to represent a double, an int, and so forth. A binary mode streamer would do just fine....

Let me know if some of your collaborators are lurking towards C++, I'll be glad to help!

Thanks!

Paul

---

From: Andrew Benson @.> Sent: Friday, 5 July 2024 14:31 To: galacticusorg/installationscripts @.> Cc: Paul Lebrun @.>; Author @.> Subject: Re: [galacticusorg/installationscripts] HDF5 problem. (Issue #3)

[EXTERNAL] – This message is from an external sender

I won't be able to work on this until next week, but you could try editing line 787 of the installation script:

maxVersion[$iPackage]="9.9.9"

Change that to:

maxVersion[$iPackage]="1.10.99"

and it should try installing an older version from source.

— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_galacticusorg_installationscripts_issues_3-23issuecomment-2D2211325298&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=rWbq940lyflnglWDrGZaLQ&m=acZ8YsC_OxIx1z85vRyTZvmVFBsRc_wmJ2wGcCtHOQCCWXSySE4MPIqcb-wpQpca&s=nCdk9KmC5sHdjdN_KeJzQL9UjDoW84IA51MpOVwPtVI&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ACQEOM64RVE62DZW6QSPBQTZK3YCPAVCNFSM6AAAAABKNT5R5GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMJRGMZDKMRZHA&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=rWbq940lyflnglWDrGZaLQ&m=acZ8YsC_OxIx1z85vRyTZvmVFBsRc_wmJ2wGcCtHOQCCWXSySE4MPIqcb-wpQpca&s=c5g0NQWEB_dG9KKX4fNdRMCSNPX7qsqoEPFdRJj-7KM&e=. You are receiving this because you authored the thread.

[abensonca]

I added a root-install RockyLinux-based (which should be similar to CentOS) build to my CI/CD for this repo. With a few updates this now builds correctly, including HDF5. It may be that on your system you want to get the updated installation script and start the install over to ensure it's getting the correct HDF5 for you - from your description above it sounds like it's still finding the system HDF5.

I'm always happy to receive any help in updating/improving code. A re-write to C++ is unlikely any time soon - I just don't have the funding to support such a major undertaking, and I'd probably want to first assess other possible language options to see what the optimal choice would be.

[paullglebrun]

Hi,

I should have mentioned that I "yum remove hdf5" from the system, last Sunday.. So the install script did install hdf5 in the ".../Tools" directory, as it should, and I presume, it did use that version. I think I simply messed up the gfortran include Unix path, or the pre-processor include path(s)...

I just did a

wget https://raw.githubusercontent.com/galacticusorg/installationscripts/installationScripts/galacticusInstall.sh

But... From where do I run this script ? At the onset, I created a directory /home/lebrun/Galacticus. In that directory, I have the galacticus-master.zip file, which I unziped. I go to the directory galacticus-master, I than got the galacticusInstall.sh bash script, and ran it from there.

Is this the correct place/directory to run the script?

It did created the .../Tools area, and install hdf5, Fox, and ann. Then, it declared "DONE DONE", but did not compiled and link/produced Galacticus.exe.

Upon re-running the same install script, it failed to compile utility.IO.HDF5.p.F90, same problem, wrong include path, or wrong pre-processed source code...

Simply typing make in the /home/lebrun/Galacticus/galacticus-mater/ also fails, again, I suspect search paths are not set correctly...

So, I'll need a bit more help...

Thanks

Paul

---

From: Andrew Benson @.> Sent: Tuesday, 9 July 2024 15:53 To: galacticusorg/installationscripts @.> Cc: Paul Lebrun @.>; Author @.> Subject: Re: [galacticusorg/installationscripts] HDF5 problem. (Issue #3)

[EXTERNAL] – This message is from an external sender

I added a root-install RockyLinux-based (which should be similar to CentOS) build to my CI/CD for this repo. With a few updates this now builds correctly, including HDF5. It may be that on your system you want to get the updated installation script and start the install over to ensure it's getting the correct HDF5 for you - from your description above it sounds like it's still finding the system HDF5.

I'm always happy to receive any help in updating/improving code. A re-write to C++ is unlikely any time soon - I just don't have the funding to support such a major undertaking, and I'd probably want to first assess other possible language options to see what the optimal choice would be.

— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_galacticusorg_installationscripts_issues_3-23issuecomment-2D2218712082&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=rWbq940lyflnglWDrGZaLQ&m=u3-SJsqHd1iSDp5D7k1AjE1xFqsVN21pi93LEsdsTCaCqYS6XHxRuNsdgkWAz7q3&s=cIsP6nIdFg9IxHlvsK7dDMyGNXeLmPFwxd8UdoJxjBc&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ACQEOM4PJYH3HADDLTXHORTZLRETVAVCNFSM6AAAAABKNT5R5GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMJYG4YTEMBYGI&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=rWbq940lyflnglWDrGZaLQ&m=u3-SJsqHd1iSDp5D7k1AjE1xFqsVN21pi93LEsdsTCaCqYS6XHxRuNsdgkWAz7q3&s=RaUk_nFVQEWKlj6drBhuryMEDH3g3PZVKEXRozWBe24&e=. You are receiving this because you authored the thread.Message ID: @.***>

[abensonca]

You can run the script from anywhere. You don't need to download galacticus-master.zip in advance - it will clone a copy of Galacticus using git after installing all of the libraries and tools needed.

You can specify the path for installation of libraries/tools via the --toolPrefix option (which will default to /usr/local/galacticus) and the path to install the Galacticus source via the --galacticusPrefix option to the script (defaults to $HOME/Galacticus/galacticus.

The DONE DONE was some debugging code I accidentally left in place after the last round of updates - I've removed it now.

To help further, can you paste here the exact command line you use to run the script? And, after it runs, there should be a file galacicusInstall.log created in the path where you ran the script. If you could share that I'll be able to see exactly what failed. If the file is too large to paste here you can put it somewhere else online and link to it.

[paullglebrun]

Hi Andrew,

If I run the galacticusInstall.sh script from a clean directory, and - important! - as root, allowing the script to write in ~root, it created a directory ~root/galacticus, and the script installed the missing dependencies, and the whole package installed properly. Flawlessly.. Everything seems to there.

I tend to use the default options, and if the default is to install under root, so be it. I appreciated you try to give the use the flexibilty, but life is short and hard..

So let me learn more about the package, not that I know it is there and I ( I think!) I know what it is!.

Thanks

Paul

---

From: Andrew Benson @.> Sent: Wednesday, 10 July 2024 08:36 To: galacticusorg/installationscripts @.> Cc: Paul Lebrun @.>; Author @.> Subject: Re: [galacticusorg/installationscripts] HDF5 problem. (Issue #3)

[EXTERNAL] – This message is from an external sender

You can run the script from anywhere. You don't need to download galacticus-master.zip in advance - it will clone a copy of Galacticus using git after installing all of the libraries and tools needed.

You can specify the path for installation of libraries/tools via the --toolPrefix option (which will default to /usr/local/galacticus) and the path to install the Galacticus source via the --galacticusPrefix option to the script (defaults to $HOME/Galacticus/galacticus.

The DONE DONE was some debugging code I accidentally left in place after the last round of updates - I've removed it now.

To help further, can you paste here the exact command line you use to run the script? And, after it runs, there should be a file galacicusInstall.log created in the path where you ran the script. If you could share that I'll be able to see exactly what failed. If the file is too large to paste here you can put it somewhere else online and link to it.

— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_galacticusorg_installationscripts_issues_3-23issuecomment-2D2220530292&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=rWbq940lyflnglWDrGZaLQ&m=ihrsBB_2KLEYy40U2UE6UwMB83LgD-o94JWWePJp_VxBdSt83dKx4CofKvVfTTTE&s=YDarqfM0u5sgIFxPkgqnYdUI5Fog1Ce_xJjLetk_xD4&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ACQEOMYZIIQMMH5WQXQEWXTZLU2FLAVCNFSM6AAAAABKNT5R5GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRQGUZTAMRZGI&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=rWbq940lyflnglWDrGZaLQ&m=ihrsBB_2KLEYy40U2UE6UwMB83LgD-o94JWWePJp_VxBdSt83dKx4CofKvVfTTTE&s=3sdAkmjVXUNYwkPncN6B_z5NPAmQvYp4p6MYsobKp7A&e=. You are receiving this because you authored the thread.

[abensonca]

That sounds good! Installing as root is certainly easier - it can make use of the package manager to install most things (and thereby avoid having to build from source).