FreeFem / FreeFem-sources

FreeFEM source code
https://freefem.org/
Other
770 stars 191 forks source link

Dependency management when building from source is not quite perfect. #318

Closed gl-yziquel closed 1 month ago

gl-yziquel commented 1 month ago

Hi.

Just met someone at a bar who referenced me to FreeFem. Just the kind of tool whose source code interests me, given the dependencies (interior point methods, SuiteSparse, ParMETIS, blah blah blah...). Had a shot a building it from source (which is my prefered mode of installation).

Frustration about the dependency management.

Failure below.

pkg/Ipopt-3.12.4.tgz                                        100%[====================[...]=============>]   4.31M  1.29MB/s    in 3.3s    

2024-09-22 11:29:37 (1.29 MB/s) - ‘pkg/Ipopt-3.12.4.tgz’ saved [4518646/4518646]

connected.
HTTP request sent, awaiting response... failed: Connection timed out.
Retrying.

failed: Connection timed out.
Retrying.

failed: Connection timed out.
Retrying.

failed: Connection timed out.
Retrying.

failed: Connection timed out.
Retrying.

failed: Connection timed out.
Retrying.

failed: Connection timed out.
Retrying.

failed: Connection timed out.
Retrying.

failed: Connection timed out.
Retrying.

failed: Connection timed out.
Retrying.

failed: Connection timed out.
Retrying.

--2024-09-22 11:30:05--  (try: 2)  http://glaros.dtc.umn.edu/gkhome/fetch/sw/metis/metis-5.1.0.tar.gz
Connecting to glaros.dtc.umn.edu (glaros.dtc.umn.edu)|160.94.40.33|:80... --2024-09-22 11:30:05--  (try: 2)  http://glaros.dtc.umn.edu/gkhome/fetch/sw/parmetis/parmetis-4.0.3.tar.gz
Connecting to glaros.dtc.umn.edu (glaros.dtc.umn.edu)|160.94.40.33|:80... --2024-09-22 11:30:05--  (try: 2)  https://www.ljll.math.upmc.fr/frey/ftp/archives/freeyams.2012.02.05.tgz
Connecting to www.ljll.math.upmc.fr (www.ljll.math.upmc.fr)|134.157.2.3|:443... --2024-09-22 11:30:05--  (try: 2)  https://www.ljll.math.upmc.fr/frey/ftp/archives/mshmet.2012.04.25.tgz
Connecting to www.ljll.math.upmc.fr (www.ljll.math.upmc.fr)|134.157.2.3|:443... --2024-09-22 11:30:05--  (try: 2)  http://104.46.50.187/pkg/gmm-4.2.tar.gz
Connecting to 104.46.50.187:80... --2024-09-22 11:30:05--  (try: 2)  https://www.ljll.math.upmc.fr/~tournier/boost_for_bemtool.tar.gz
Connecting to www.ljll.math.upmc.fr (www.ljll.math.upmc.fr)|134.157.2.3|:443... --2024-09-22 11:30:05--  (try: 2)  http://104.46.50.187/pkg/libpthread-google.tar.gz
Connecting to 104.46.50.187:80... --2024-09-22 11:30:05--  (try: 2)  http://104.46.50.187/pkg/mmg.zip
Connecting to 104.46.50.187:80... --2024-09-22 11:30:05--  (try: 2)  http://104.46.50.187/pkg/parmmg.zip
Connecting to 104.46.50.187:80... --2024-09-22 11:30:05--  (try: 2)  http://104.46.50.187/pkg/patch.tar.gz
Connecting to 104.46.50.187:80... --2024-09-22 11:30:05--  (try: 2)  http://104.46.50.187/pkg/arpack96.tar.gz
Connecting to 104.46.50.187:80... Read error (Connection timed out) in headers.
Retrying.

[...]

ERROR: ../bin/build/download http://104.46.50.187/pkg/gmm-4.2.tar.gz pkg/gmm-4.2.tar.gz  FAILED
ERROR 2: ../bin/build/download http://pkgs.freefem.org/gmm-4.2.tar.gz pkg/gmm-4.2.tar.gz  FAILED
ERROR2: INVALID MD5 for gmm-4.2.tar.gz  d41d8cd98f00b204e9800998ecf8427e
ERROR: ../bin/build/download http://104.46.50.187/pkg/libpthread-google.tar.gz pkg/libpthread-google.tar.gz  FAILED
ERROR 2: ../bin/build/download http://pkgs.freefem.org/libpthread-google.tar.gz pkg/libpthread-google.tar.gz  FAILED
ERROR2: INVALID MD5 for libpthread-google.tar.gz  d41d8cd98f00b204e9800998ecf8427e
ERROR: ../bin/build/download http://104.46.50.187/pkg/mmg.zip pkg/mmg.zip  FAILED
ERROR: ../bin/build/download http://104.46.50.187/pkg/parmmg.zip pkg/parmmg.zip  FAILED
ERROR 2: ../bin/build/download http://pkgs.freefem.org/mmg.zip pkg/mmg.zip  FAILED
ERROR2: INVALID MD5 for mmg.zip  d41d8cd98f00b204e9800998ecf8427e
ERROR 2: ../bin/build/download http://pkgs.freefem.org/parmmg.zip pkg/parmmg.zip  FAILED
ERROR2: INVALID MD5 for parmmg.zip  d41d8cd98f00b204e9800998ecf8427e
ERROR: ../bin/build/download http://104.46.50.187/pkg/patch.tar.gz pkg/patch.tar.gz  FAILED
ERROR: ../bin/build/download http://104.46.50.187/pkg/arpack96.tar.gz pkg/arpack96.tar.gz  FAILED
ERROR 2: ../bin/build/download http://pkgs.freefem.org/patch.tar.gz pkg/patch.tar.gz  FAILED
ERROR2: INVALID MD5 for patch.tar.gz  d41d8cd98f00b204e9800998ecf8427e
ERROR 2: ../bin/build/download http://pkgs.freefem.org/arpack96.tar.gz pkg/arpack96.tar.gz  FAILED
ERROR2: INVALID MD5 for arpack96.tar.gz  d41d8cd98f00b204e9800998ecf8427e
ERROR: ../bin/build/download http://104.46.50.187/pkg/metis-5.1.0.tar.gz pkg/metis-5.1.0.tar.gz  FAILED
ERROR: ../bin/build/download http://104.46.50.187/pkg/ParMetis-4.0.3.tar.gz pkg/ParMetis-4.0.3.tar.gz  FAILED
ERROR: ../bin/build/download http://104.46.50.187/pkg/freeyams.2012.02.05.tgz pkg/freeyams.2012.02.05.tgz  FAILED
ERROR: ../bin/build/download http://104.46.50.187/pkg/mshmet.2012.04.25.tgz pkg/mshmet.2012.04.25.tgz  FAILED
ERROR 2: ../bin/build/download http://pkgs.freefem.org/metis-5.1.0.tar.gz pkg/metis-5.1.0.tar.gz  FAILED
ERROR2: INVALID MD5 for metis-5.1.0.tar.gz  d41d8cd98f00b204e9800998ecf8427e
ERROR 2: ../bin/build/download http://pkgs.freefem.org/ParMetis-4.0.3.tar.gz pkg/ParMetis-4.0.3.tar.gz  FAILED
ERROR2: INVALID MD5 for ParMetis-4.0.3.tar.gz  d41d8cd98f00b204e9800998ecf8427e
ERROR 2: ../bin/build/download http://pkgs.freefem.org/freeyams.2012.02.05.tgz pkg/freeyams.2012.02.05.tgz  FAILED
ERROR2: INVALID MD5 for freeyams.2012.02.05.tgz  d41d8cd98f00b204e9800998ecf8427e
ERROR 2: ../bin/build/download http://pkgs.freefem.org/mshmet.2012.04.25.tgz pkg/mshmet.2012.04.25.tgz  FAILED
ERROR2: INVALID MD5 for mshmet.2012.04.25.tgz  d41d8cd98f00b204e9800998ecf8427e
ERROR: ../bin/build/download http://104.46.50.187/pkg/boost_for_bemtool.tar.gz pkg/boost_for_bemtool.tar.gz  FAILED
ERROR 2: ../bin/build/download http://pkgs.freefem.org/boost_for_bemtool.tar.gz pkg/boost_for_bemtool.tar.gz  FAILED
ERROR2: INVALID MD5 for boost_for_bemtool.tar.gz  d41d8cd98f00b204e9800998ecf8427e
ERROR: ../bin/build/download http://104.46.50.187/pkg/MUMPS_5.0.2.tar.gz pkg/MUMPS_5.0.2.tar.gz  FAILED
ERROR 2: ../bin/build/download http://pkgs.freefem.org/MUMPS_5.0.2.tar.gz pkg/MUMPS_5.0.2.tar.gz  FAILED
ERROR2: INVALID MD5 for MUMPS_5.0.2.tar.gz  d41d8cd98f00b204e9800998ecf8427e

P.S.: Ravi de voir que cela bosse bien à Jussieu... souvenirs...

prj- commented 1 month ago

1) use the develop branch 2) build PETSc first using make petsc-slepc

Then, you should not see such errors.

gl-yziquel commented 1 month ago
  1. use the develop branch

Thank you. Somewhat better. Though that should IMO be documented on the README.md in the master branch to avoid a clone/build/fail sequence for newcommers.

  1. build PETSc first using make petsc-slepc

Which then yields:

no write access in /usr/local/ff-petsc/r and /usr/local/ff-petsc/c

I do not install things like that with sudo except in some rare circumstances. Better instructions on the README.md would be... nice...

Then, you should not see such errors.

I can't wait.

Thanks.

prj- commented 1 month ago

Better instructions on the README.md would be... nice...

You need to read the instructions first to comment on the quality of said instructions. It is written, both on the master and develop branch: If you do not have administrator rights or do not want FreeFEM files scattered around on your machine, please use the --prefix option. See https://github.com/FreeFem/FreeFem-sources?tab=readme-ov-file#freefem.

gl-yziquel commented 1 month ago

Better instructions on the README.md would be... nice...

You need to read the instructions first to comment on the quality of said instructions.

I do not want to get into a fight on this. I grant you your point. However, I do expect, because I build a ton of stuff all the time, that the instructions on the README.md of the webpage I land on usually work out of the box. This one did not mention the develop branch, which is the main point. (except in a "the development occurs on the development branch", which does not immediately point to the reason of why the instructions of the master branch README.md fail when trying them on master).

I know this is not the 90s anymore and people use docker builds and what not (which is not the case here) everywhere and compiling from source has become kind of posh, but still...

I'm just saying "minimise friction, please", in your README.md. Not that the information isn't more or less there.

So let's leave it at that.

gl-yziquel commented 1 month ago

OK. For anyone stumbling on this discussion, if you wish to build and not install the source code (which is what I wanted to do as I do not wish to mix build and install steps), you may proceed (hopefully blindly) as follows.

git clone https://github.com/FreeFem/FreeFem-sources.git
cd FreeFem-sources
git checkout develop
autoreconf -vif
mkdir _prefix
./configure --enable-download --enable-optim --prefix=$(pwd)/_prefix
./3rdparty/getall -a
cd 3rdparty/ff-petsc
make petsc-slepc
cd -
./reconfigure
make

Too much steps to my taste, but this gets automated properly in my automated build system on Ubuntu 24.04.1 LTS. Haven't tried the binaries yet, though.

prj- commented 1 month ago

Too much steps to my taste

You can have three fewer steps.

git clone -b develop https://github.com/FreeFem/FreeFem-sources.git
cd FreeFem-sources
autoreconf -vif
./configure --enable-download --enable-optim --prefix=$(pwd)/_prefix
cd 3rdparty/ff-petsc
make petsc-slepc
cd -
./reconfigure
make
gl-yziquel commented 1 month ago

./3rdparty/getall -a is not required ?

prj- commented 1 month ago

Just if you want to download every packages upfront. But that's not really mandatory (especially if you build PETSc, which will fetch on its own most of the other packages).

gl-yziquel commented 1 month ago

In the long run, I'd like to build every package independently, and not download them at all. But, as of today, I have an unclear picture of how to separate the build of PETSc from that of FreeFEM.

prj- commented 1 month ago

But, as of today, I have an unclear picture of how to separate the build of PETSc from that of FreeFEM.

See page 198 ff. of https://joliv.et/FreeFem-tutorial/main.pdf ("Optimal compilation process").

gl-yziquel commented 1 month ago

But, as of today, I have an unclear picture of how to separate the build of PETSc from that of FreeFEM.

See page 198 ff. of https://joliv.et/FreeFem-tutorial/main.pdf ("Optimal compilation process").

Thank you very much. A bit busy now building the build2-toolchain repo, but as soon as my interest in sobolev spaces gets rekindled, I'll have a look at your document.

Bookmarked.