OPM / opm-core

Collection of utilities, solvers and other components.
http://www.opm-project.org
GNU General Public License v3.0
44 stars 50 forks source link

Meta-issue to track the readiness of the build system rollup #338

Closed rolk closed 11 years ago

rolk commented 11 years ago

This is a meta-issue to track the readiness to merge the following pull requests, all related to synchronizing the build system across modules (aka. support-building-with-DUNE-2.3):

When this issue is closed they are deemed to be ready and will be merged shortly and without further warning.

Currently however, the following pull requests are slated for inclusion and pending review:

If anyone sits on further improvements (to the build system), they are encourage to bring them forward now and add them to this list, so that we don't have to spend more time on this again before the release. (There certainly won't be more rollups from me).

rolk commented 11 years ago

Further issues that are related to configuration:

andlaus commented 11 years ago

pretty trivial, but still required for the rollup:

bska commented 11 years ago

I think all inbound PRs pertaining to the build system have been merged into opm-core:master. How much work is it to bring those changes into the roll-up PRs and fire off a (hopefully) final round of testing?

rolk commented 11 years ago

bring those changes into the roll-up PRs and fire off a (hopefully) final round of testing

I have updated all the pull-requests correspondingly. If anyone wants pre-canned statements to checkout the set of rollups, here they are:

git clone -b master http://github.com/rolk/opm-core
git clone -b 009_cmake http://github.com/rolk/opm-material
git clone -b 041_cmake http://github.com/rolk/dune-cornerpoint
git clone -b 063_cmake http://github.com/rolk/opm-porsol
git clone -b 071_cmake http://github.com/rolk/opm-upscaling
git clone -b 020_cmake http://github.com/rolk/opm-benchmarks
git clone -b 030_cmake http://github.com/rolk/opm-polymer
git clone -b 023_cmake http://github.com/rolk/opm-autodiff
git clone -b 027_cmake http://github.com/rolk/opm-verteq
blattms commented 11 years ago
git clone -b master http://github.com/rolk/opm-core
git clone -b 009_cmake http://github.com/rolk/opm-material
git clone -b 041_cmake http://github.com/rolk/dune-cornerpoint
git clone -b 063_cmake http://github.com/rolk/opm-porsol
git clone -b 071_cmake http://github.com/rolk/opm-upscaling
git clone -b 020_cmake http://github.com/rolk/opm-benchmarks
git clone -b 030_cmake http://github.com/rolk/opm-polymer
git clone -b 023_cmake http://github.com/rolk/opm-autodiff
git clone -b 027_cmake http://github.com/rolk/opm-verteq

I am not really sure whether this is related to the build system changes. But on my system dune-cornerpoint fails to compile when using dunecontrol without any additional options and the releases/2.2-cmake branches of the DUNE core modules:

[ 80%] Building CXX object CMakeFiles/finitevolume.dir/examples/finitevolume/finitevolume.cc.o In file included from /tmp/dune-cornerpoint/examples/finitevolume/finitevolume.cc:11:0: /tmp/dune-common/dune/common/mpihelper.hh:4:2: warning: #warning dune/common/mpihelper.hh is deprecated, please use dune/common/parallel/mpihelper.hh [-Wcpp] In file included from /tmp/dune-grid/dune/grid/common/grid.hh:26:0, from /tmp/dune-cornerpoint/dune/grid/CpGrid.hpp:51, from /tmp/dune-cornerpoint/examples/finitevolume/finitevolume.cc:23: /tmp/dune-grid/dune/grid/common/defaultgridview.hh: In instantiation of ‘Dune::DefaultLeafGridView<GridImp, pitype>::IntersectionIterator Dune::DefaultLeafGridView<GridImp, pitype>::iend(const typename Dune::DefaultLeafGridView<GridImp, pitype>::Codim<0>::Entity&) const [with GridImp = Dune::CpGrid; Dune::PartitionIteratorType pitype = (Dune::PartitionIteratorType)4u; Dune::DefaultLeafGridView<GridImp, pitype>::IntersectionIterator = Dune::cpgrid::IntersectionIteratorDune::CpGrid; typename Dune::DefaultLeafGridView<GridImp, pitype>::Codim<0>::Entity = Dune::cpgrid::Entity<0, Dune::CpGrid>]’: /tmp/dune-grid/dune/grid/common/gridview.hh:223:32: required from ‘Dune::GridView::IntersectionIterator Dune::GridView::iend(const typename Dune::GridView::Codim<0>::Entity&) const [with ViewTraits = Dune::DefaultLeafGridViewTraits<Dune::CpGrid, (Dune::PartitionIteratorType)4u>; Dune::GridView::IntersectionIterator = Dune::cpgrid::IntersectionIteratorDune::CpGrid; typename Dune::GridView::Codim<0>::Entity = Dune::cpgrid::Entity<0, Dune::CpGrid>]’ /tmp/dune-cornerpoint/examples/finitevolume/evolve.hh:60:55: required from ‘void evolve(const G&, const M&, V&, double, double&) [with G = Dune::CpGrid; M = Dune::LeafMultipleCodimMultipleGeomTypeMapper<Dune::CpGrid, P0Layout>; V = std::vector]’ /tmp/dune-cornerpoint/examples/finitevolume/finitevolume.cc:70:9: required from ‘void timeloop(const G&, double) [with G = Dune::CpGrid]’ /tmp/dune-cornerpoint/examples/finitevolume/finitevolume.cc:114:27: required from here /tmp/dune-grid/dune/grid/common/defaultgridview.hh:370:37: error: ‘const Entity’ has no member named ‘impl’ /tmp/dune-grid/dune/grid/common/defaultgridview.hh: In instantiation of ‘Dune::DefaultLeafGridView<GridImp, pitype>::IntersectionIterator Dune::DefaultLeafGridView<GridImp, pitype>::ibegin(const typename Dune::DefaultLeafGridView<GridImp, pitype>::Codim<0>::Entity&) const [with GridImp = Dune::CpGrid; Dune::PartitionIteratorType pitype = (Dune::PartitionIteratorType)4u; Dune::DefaultLeafGridView<GridImp, pitype>::IntersectionIterator = Dune::cpgrid::IntersectionIteratorDune::CpGrid; typename Dune::DefaultLeafGridView<GridImp, pitype>::Codim<0>::Entity = Dune::cpgrid::Entity<0, Dune::CpGrid>]’: /tmp/dune-grid/dune/grid/common/gridview.hh:216:34: required from ‘Dune::GridView::IntersectionIterator Dune::GridView::ibegin(const typename Dune::GridView::Codim<0>::Entity&) const [with ViewTraits = Dune::DefaultLeafGridViewTraits<Dune::CpGrid, (Dune::PartitionIteratorType)4u>; Dune::GridView::IntersectionIterator = Dune::cpgrid::IntersectionIteratorDune::CpGrid; typename Dune::GridView::Codim<0>::Entity = Dune::cpgrid::Entity<0, Dune::CpGrid>]’ /tmp/dune-cornerpoint/examples/finitevolume/evolve.hh:61:59: required from ‘void evolve(const G&, const M&, V&, double, double&) [with G = Dune::CpGrid; M = Dune::LeafMultipleCodimMultipleGeomTypeMapper<Dune::CpGrid, P0Layout>; V = std::vector]’ /tmp/dune-cornerpoint/examples/finitevolume/finitevolume.cc:70:9: required from ‘void timeloop(const G&, double) [with G = Dune::CpGrid]’ /tmp/dune-cornerpoint/examples/finitevolume/finitevolume.cc:114:27: required from here /tmp/dune-grid/dune/grid/common/defaultgridview.hh:363:39: error: ‘const Entity’ has no member named ‘impl’ /tmp/dune-grid/dune/grid/common/defaultgridview.hh: In member function ‘Dune::DefaultLeafGridView<GridImp, pitype>::IntersectionIterator Dune::DefaultLeafGridView<GridImp, pitype>::iend(const typename Dune::DefaultLeafGridView<GridImp, pitype>::Codim<0>::Entity&) const [with GridImp = Dune::CpGrid; Dune::PartitionIteratorType pitype = (Dune::PartitionIteratorType)4u; Dune::DefaultLeafGridView<GridImp, pitype>::IntersectionIterator = Dune::cpgrid::IntersectionIteratorDune::CpGrid; typename Dune::DefaultLeafGridView<GridImp, pitype>::Codim<0>::Entity = Dune::cpgrid::Entity<0, Dune::CpGrid>]’: /tmp/dune-grid/dune/grid/common/defaultgridview.hh:371:5: warning: control reaches end of non-void function [-Wreturn-type] /tmp/dune-grid/dune/grid/common/defaultgridview.hh: In member function ‘Dune::DefaultLeafGridView<GridImp, pitype>::IntersectionIterator Dune::DefaultLeafGridView<GridImp, pitype>::ibegin(const typename Dune::DefaultLeafGridView<GridImp, pitype>::Codim<0>::Entity&) const [with GridImp = Dune::CpGrid; Dune::PartitionIteratorType pitype = (Dune::PartitionIteratorType)4u; Dune::DefaultLeafGridView<GridImp, pitype>::IntersectionIterator = Dune::cpgrid::IntersectionIteratorDune::CpGrid; typename Dune::DefaultLeafGridView<GridImp, pitype>::Codim<0>::Entity = Dune::cpgrid::Entity<0, Dune::CpGrid>]’: /tmp/dune-grid/dune/grid/common/defaultgridview.hh:364:5: warning: control reaches end of non-void function [-Wreturn-type] make[2]: * [CMakeFiles/finitevolume.dir/examples/finitevolume/finitevolume.cc.o] Fehler 1 make[1]: * [CMakeFiles/finitevolume.dir/all] Fehler 2 make: *\ [all] Fehler 2

Somehow this seems familiar and I would have thought that I already fixed this.

bska commented 11 years ago

If anyone wants pre-canned statements to checkout the set of rollups, here they are:

Well, that was a metric disappointment. I tried to build it on CentOS 5.9 and here's what I got from my hackish build script

  dbg/opm-core
  --> cmake -C setup.cmake /home/bska/work/opm/src/test-rollup/opm-core
      Time = 9.9 [s]
  --> make -j15 all
      Time = 28.5 [s]: Success

  dbg/opm-autodiff
  --> cmake -C setup-noshadow.cmake /home/bska/work/opm/src/test-rollup/opm-autodiff
      Time = 7.53 [s]
  --> make -j15 all
      Time = 0.00287 [s]: FAILURE [exit = 2 (GNU Make: Build error)]

  dbg/opm-polymer
  --> cmake -C setup.cmake /home/bska/work/opm/src/test-rollup/opm-polymer
      Time = 9.55 [s]
  --> make -j15 all
      Time = 15.3 [s]: Success

  dbg/opm-verteq
  --> cmake -C setup.cmake /home/bska/work/opm/src/test-rollup/opm-verteq
      Time = 10.8 [s]
  --> make -j15 all
      Time = 3.68 [s]: FAILURE [exit = 2 (GNU Make: Build error)]

  dbg/opm-material
  --> cmake -C setup.cmake /home/bska/work/opm/src/test-rollup/opm-material
      Time = 6.29 [s]
  --> make -j15 all
      Time = 0.00284 [s]: FAILURE [exit = 2 (GNU Make: Build error)]

  dbg/dune-cornerpoint
  --> cmake -C setup.cmake /home/bska/work/opm/src/test-rollup/dune-cornerpoint
      Time = 13.2 [s]
  --> make -j15 all
      Time = 36.6 [s]: Success

  dbg/opm-porsol
  --> cmake -C setup.cmake /home/bska/work/opm/src/test-rollup/opm-porsol
      Time = 6.62 [s]
  --> make -j15 all
      Time = 0.00282 [s]: FAILURE [exit = 2 (GNU Make: Build error)]

  dbg/opm-upscaling
  --> cmake -C setup.cmake /home/bska/work/opm/src/test-rollup/opm-upscaling
      Time = 6.28 [s]
  --> make -j15 all
      Time = 0.00284 [s]: FAILURE [exit = 2 (GNU Make: Build error)]

  dbg/opm-benchmarks
  --> cmake -C setup.cmake /home/bska/work/opm/src/test-rollup/opm-benchmarks
      Time = 21.1 [s]
  --> make -j15 all
      Time = 9.21 [s]: FAILURE [exit = 2 (GNU Make: Build error)]

Other than -verteq, which I've never tried to build before and which appears to have deeper issues, all of the build failures are really configuration failures because there is no Makefile in many of the build directories. The remaining errors (notably opm-benchmarks) stem from failing build some/all of the prerequisites. As far as I can tell, the configuration errors all relate to failure to locate a dune.module file describing the Dune ISTL. That is a little strange, though, because opm-core finds the ISTL without any problems.

Upon closer inspection, I can think of a possible explanation for that, but I haven't checked yet whether or not that explanation is likely. Searching in the location (/home/bska/work/opm/inst/dbg) containing installed third-party software (i.e., the ERT and Dune core modules (version 2.2.1)), I find

/home/bska/work/opm/inst/dbg/lib64/dunecontrol/opm-material/dune.module
/home/bska/work/opm/inst/dbg/lib/dunecontrol/dune-istl/dune.module
/home/bska/work/opm/inst/dbg/lib/dunecontrol/dune-localfunctions/dune.module
/home/bska/work/opm/inst/dbg/lib/dunecontrol/dune-geometry/dune.module
/home/bska/work/opm/inst/dbg/lib/dunecontrol/dune-grid/dune.module
/home/bska/work/opm/inst/dbg/lib/dunecontrol/dune-common/dune.module

CentOS 5.9 is partially multilibed. Could it be that we're searching only in lib64 when we could/should be searching both lib and lib64?

bska commented 11 years ago

Oh, and a second comment. I can't reproduce @blattms's build failure for finitevolume.cc. I don't know why.

rolk commented 11 years ago

@blattms

defaultgridview.hh:370:37: error: ‘const Entity’ has no member named ‘impl’

Is the Entity typename defined in some conditional (#ifdef) section? What options does dunecontrol-cmake pass to the cmake invocation?

rolk commented 11 years ago

@bska

Other than -verteq, which I've never tried to build before and which appears to have deeper issues

That _may_ be due to inadvertedly using some features that does not exist in GCC 4.4; I try to write in a fairly conservative manner, but I haven't tested it on RHEL5.

CentOS 5.9 is partially multilibed

I think partially may be the key here. Libraries with x86_64 binaries should be in the lib64 directory, but I am not sure about dune.module which is inheritly noarch (Why is opm-material in lib64 BTW?).

The greater mystery is that I cannot recall having done anything with that code; it is probably fallout from some other patch.

bska commented 11 years ago

Other than -verteq, which I've never tried to build before and which appears to have deeper issues

That may be due to inadvertently using some features that does not exist in GCC 4.4; I try to write in a fairly conservative manner, but I haven't tested it on RHEL5.

If it is due to using features that don't exist in GCC 4.4, then it certainly manifests in strange ways. Here are some of the build failures I get (I can send a transcript offline if you wish):

In file included from /home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/verteq.cpp:6:
/home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/upscale.hpp: In constructor ‘Opm::Elevation::Elevation(int, double)’:
/home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/upscale.hpp:40: warning: declaration of ‘fraction’ shadows a member of 'this'
/home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/upscale.hpp:40: warning: declaration of ‘block’ shadows a member of 'this'
/home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/topsurf.cpp: At global scope:
/home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/topsurf.cpp:56: error: ISO C++ forbids declaration of ‘vector’ with no type
/home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/topsurf.cpp:56: error: expected ‘;’ before ‘<’ token
/home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/topsurf.cpp:61: error: ISO C++ forbids declaration of ‘vector’ with no type
/home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/topsurf.cpp:61: error: expected ‘;’ before ‘<’ token
/home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/topsurf.cpp:66: error: ISO C++ forbids declaration of ‘vector’ with no type
/home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/topsurf.cpp:66: error: expected ‘;’ before ‘<’ token
/home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/topsurf.cpp: In member function ‘void TopSurfBuilder::create_elements()’:
/home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/topsurf.cpp:122: error: ‘vector’ was not declared in this scope
/home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/topsurf.cpp:122: error: expected primary-expression before ‘int’
/home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/topsurf.cpp:122: error: expected ‘;’ before ‘int’
/home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/topsurf.cpp:130: error: expected primary-expression before ‘int’
/home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/topsurf.cpp:130: error: expected ‘;’ before ‘int’
/home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/topsurf.cpp:131: error: expected primary-expression before ‘int’
/home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/topsurf.cpp:131: error: expected ‘;’ before ‘int’
/home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/topsurf.cpp:149: error: ‘deep_k’ was not declared in this scope
/home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/topsurf.cpp:150: error: ‘high_k’ was not declared in this scope
/home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/topsurf.cpp:156: error: ‘act_cnt’ was not declared in this scope
/home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/topsurf.cpp:164: error: ‘act_cnt’ was not declared in this scope
/home/bska/work/opm/src/test-rollup/opm-verteq/opm/verteq/topsurf.cpp:165: error: ‘high_k’ was not declared in this scope
bska commented 11 years ago

CentOS 5.9 is partially multilibed

I think partially may be the key here. Libraries with x86_64 binaries should be in the lib64 directory, but I am not sure about dune.module which is inheritly noarch

It's certainly strange.

Why is opm-material in lib64 BTW?.

I have no idea. I just did make install some time ago and paid no attention to the specific location of the installed files. That said, it may be masking the real issue, so I'll remove it and try again from a source-based -material.

rolk commented 11 years ago

@bska [about opm-verteq]

topsurf.cpp:56: error: ISO C++ forbids declaration of ‘vector’ with no type

Upon inspection I think this may be because <vector> is implicitly included in the pre-compiled headers, but this is not done on GCC 4.4. Clearly a bug. I'll fix this eventually; please disregard this repo for now.

bska commented 11 years ago

Update: Other than -verteq, everything builds without issue and all tests pass on Ubuntu 10.04 LTS in both Debug and Release modes.

Second update: No change on CentOS 5.9 when using an opm-material module in the source tree rather than the install tree.

rolk commented 11 years ago

@bska

Other than -verteq, everything builds ... on Ubuntu 10.04 LTS

I have pulled in the latest changes to the code for opm-verteq; this should now be able to build without precompiled headers.

No change on CentOS 5.9

I will try to get CentOS running; however, I cannot start a VM while building opm-porsol...

andlaus commented 11 years ago

@blattms

I am not really sure whether this is related to the build system changes.

I had the same problem, but only in the current dune-grid master. if you change entity.impl to entity in the DefaultGridView it might work (well, at least compile). I thought that this was a temporary breakage of the dune-grid master branch, though...

bska commented 11 years ago

As a step towards debugging the build issues on CentOS 5.9, I made the following change to cmake/Modules/UseDuneVer.cmake in all modules:

diff --git a/cmake/Modules/UseDuneVer.cmake b/cmake/Modules/UseDuneVer.cmake
index ef917d0..247f1fc 100644
--- a/cmake/Modules/UseDuneVer.cmake
+++ b/cmake/Modules/UseDuneVer.cmake
@@ -63,6 +63,7 @@ function (find_dune_version suite module)
          get_filename_component (_lib_path "${_lib_path}" PATH)
        endif ()
        set (_dune_mod "${_lib_path}${_multilib}/dunecontrol/${suite}-${module}/dune.module")
+       message(FIND_DUNE_VERSION: DUNE_MOD = ${_dune_mod})
        if (NOT EXISTS "${_dune_mod}")
          # use the name itself as a flag for whether it was found or not
          set (_dune_mod "")

Summarising the output over all modules I get the following list of Dune modules (and dune.modules) being referenced:

DUNE_MOD=/home/bska/work/opm/inst/dbg/lib/dunecontrol/dune-grid/dune.module
DUNE_MOD=/home/bska/work/opm/inst/dbg/lib/dunecontrol/dune-geometry/dune.module
DUNE_MOD=/home/bska/work/opm/inst/dbg/lib/dunecontrol/dune-common/dune.module
DUNE_MOD=/home/bska/work/opm/inst/dbg/lib64/dunecontrol/dune-istl/dune.module

This is on a system where the dune core modules (common, geometry, grid, and istl) are installed in /home/bska/work/opm/inst/dbg, with

$ find /home/bska/work/opm/inst/dbg -name dune.module
/home/bska/work/opm/inst/dbg/lib/dunecontrol/dune-istl/dune.module
/home/bska/work/opm/inst/dbg/lib/dunecontrol/dune-geometry/dune.module
/home/bska/work/opm/inst/dbg/lib/dunecontrol/dune-grid/dune.module
/home/bska/work/opm/inst/dbg/lib/dunecontrol/dune-common/dune.module

so, for whatever reason the UseDuneVer.cmake module searches for ISTL's dune.module file in the wrong location. Is this possibly related to ISTL not creating a library?

rolk commented 11 years ago

Is this possibly related to ISTL not creating a library?

Possibly yes. If there is no library, it must guess the location from the header files (and it guesses wrong). I think I have a solution, probably done this evening. (Must get the CentOS 5 VM up and running with DUNE)

rolk commented 11 years ago

@akva2 @bska I am confused. If I install on CentOS 5, dune.module ends up in /usr/lib whereas the RPM packages (which are made for CentOS) have them in /usr/lib64. Grepping for dune.module in dunecontrol turns up a reference on line 917 where it seems it always use lib, so how come we ended up with lib64?

blattms commented 11 years ago

On Tue, Aug 27, 2013 at 02:59:46AM -0700, Andreas Lauser wrote:

@blattms

I am not really sure whether this is related to the build system changes.

I had the same problem, but only in the current dune master.

It turns out that I have problems using git correctly and ended up with the master branches of DUNE. With the 2.2 releases everything builds like expected.

bska commented 11 years ago

I am confused. If I install on CentOS 5, dune.module ends up in /usr/lib

Presumably that would be ${prefix}/lib if you defined a non-standard --prefix=? I don't know about the RPMs (I never tried those), but I can conform that ${prefix}/lib is where the dune.module files end up when I [dunecontrol] make install a particular set of Dune core modules. That was why I was surprised that our CMake system tried to retrieve ISTL's dune.module from ${prefix}/lib64 (but the others from ${prefix}/lib (!)). I think there might be some inconsistency, but I don't know enough about CMake to start analysing the system fully.

rolk commented 11 years ago

@bska I was able to get the configuration step going by applying the two patches in #350. I will give an update if I manage to link before I go to sleep...

bska commented 11 years ago

I will give an update if I manage to link before I go to sleep...

Best of luck. Meanwhile, the roll-ups work perfectly on my Ubuntu 12.04 LTS laptop without any external grid managers (ALUGrid, I'm looking at you) or MPI installed. That may not be particularly surprising for you (thank you for fixing -verteq, by the way) given that that configuration closely matches your primary development system, but I thought I'd share the news anyway. Thank you for the heroic efforts!

akva2 commented 11 years ago

On 08/27/2013 10:12 PM, Roland Kaufmann wrote:

@akva2 https://github.com/akva2 @bska https://github.com/bska I am confused. If I install on CentOS 5, dune.module ends up in |/usr/lib| whereas the RPM packages (which are made for CentOS) have them in |/usr/lib64|. Grepping for |dune.module| in dunecontrol turns up a reference on line 917 where it seems it always use |lib|, so how come we ended up with |lib64|?

— Reply to this email directly or view it on GitHub https://github.com/OPM/opm-core/issues/338#issuecomment-23366754.

iirc arch-independent files is supposed to go to lib64 (on 64bit) or lib32 (on 32 bit) in rhel. i guess somebody got inspired by this standard and joined in on the confusion or something?

rolk commented 11 years ago

@akva2

iirc arch-independent files is supposed to go to lib64 (on 64bit) or lib32 (on 32 bit) in rhel

That's what I thought too (and that's they way we (meaning OPM) behaved up until #350), but it doesn't seem that DUNE follow that convention (and there's no use of dune.module outside DUNE - we use it for configuration in the source tree, but not in the build)

akva2 commented 11 years ago

right. then it should go to lib to be compatible with dunecontrol.

rolk commented 11 years ago

Update: After I came to my senses and checked out the releases/2.2 -cmake branch of DUNE (yay for @blattms!), I managed to get opm-core to build and link with a GCC 4.4 toolchain on CentOS 5, with patchsets #350 and #351. It even passes all the tests.

andlaus commented 11 years ago

@blattms

It turns out that I have problems using git correctly and ended up with the master branches of DUNE. With the 2.2 releases everything builds like expected.

good to read that it compiles for you, too. Can you be so kind and check what the underlying problem with the dune master is? is it an opm or a dune issue? if opm, what interfaces do we need to change to make DefaultGridView in Dune 2.3 happy?

bska commented 11 years ago

With the patch set from #350 installed (manually) I too am able to build the roll-ups on CentOS 5.9. This is with installed Dune core modules (version 2.2.1). I did make a mistake in applying the change-set to -autodiff, so that does not (currently) work here, but I'll redo the change to confirm.

bska commented 11 years ago

I said

I did make a mistake in applying the change-set to -autodiff, so that does not (currently) work here, but I'll redo the change to confirm.

Upon closer inspection, I don't think that's accurate. The CentOS system on which I build this does not have Eigen installed. In commit OPM/opm-autodiff@d6563b2 (later perfected in a few other commits), @akva2 added code to the CMakeLists.txt to automatically download a copy of Eigen (from OPM/eigen3) if the Eigen software was not found. That code appears to have been rendered useless by the roll-up. At least, I now get configuration errors of the form

CMake Error at /work/encap/cmake-2.8.10.1+0/share/cmake-2.8/Modules/FindPackageHandleStandardArgs.cmake:97 (message):
  Could NOT find Eigen3 (missing: EIGEN3_INCLUDE_DIR EIGEN3_VERSION_OK)
  (Required is at least version "3.1")
Call Stack (most recent call first):
  /work/encap/cmake-2.8.10.1+0/share/cmake-2.8/Modules/FindPackageHandleStandardArgs.cmake:291 (_FPHSA_FAILURE_MESSAGE)
  cmake/Modules/FindEigen3.cmake:89 (find_package_handle_standard_args)
  cmake/Modules/OpmFind.cmake:146 (find_package)
  cmake/Modules/OpmFind.cmake:204 (find_and_append_package_to)
  cmake/Modules/OpmLibMain.cmake:90 (find_and_append_package_list_to)
  CMakeLists.txt:63 (include)

and the download code apparently doesn't run. How do I go about debugging this issue?

rolk commented 11 years ago

automatically download a copy of Eigen (from OPM/eigen3) if the Eigen software was not found

Although probably very elegant, is it really necessary? I don't image the sum of all people typing git clone http://github.com/OPM/Eigen3 will ever be larger than the time to debug it.

bska commented 11 years ago

How do I go about debugging [missing Eigen]?

Here's a work-around. It actually mirrors [parts of] OPM/opm-autodiff@d6563b2. With it, opm-autodiff builds without issue on my CentOS 5.9 system.

diff --git a/cmake/Modules/opm-autodiff-prereqs.cmake b/cmake/Modules/opm-autodiff-prereqs.cmake
index 650e533..5023329 100644
--- a/cmake/Modules/opm-autodiff-prereqs.cmake
+++ b/cmake/Modules/opm-autodiff-prereqs.cmake
@@ -19,5 +19,5 @@ set (opm-autodiff_DEPS
        dune-istl REQUIRED;
        opm-core REQUIRED"
        # Eigen
-       "Eigen3 3.1 REQUIRED"
+       "Eigen3 3.1"
        )
rolk commented 11 years ago

With it, opm-autodiff builds without issue

With the expense that everything that eventually depends on opm-autodiff gets confused if Eigen3 is not there.

bska commented 11 years ago

With it, opm-autodiff builds without issue

With the expense that everything that eventually depends on opm-autodiff gets confused if Eigen3 is not there.

Sorry, I don't follow. Could you provide a lay-man's terms explanation of why? Assume I know nothing of the details of "REQUIRED".

akva2 commented 11 years ago

On 08/28/2013 11:25 AM, Bård Skaflestad wrote:

    With it, opm-autodiff builds without issue

With the expense that everything that eventually depends on
opm-autodiff gets confused if Eigen3 is not there.

Sorry, I don't follow. Could you provide a lay-man's terms explanation of why? Assume I know nothing of the details of "REQUIRED".

— Reply to this email directly or view it on GitHub https://github.com/OPM/opm-core/issues/338#issuecomment-23401577.

REQUIRED means it will halt if it does not find Eigen3.

it works like this; look for Eigen3 without REQUIRED. if found use it, if not download and build.

rolk commented 11 years ago

In the dependency list three things are specified here. The first one is of course the package name, "Eigen3". The next one is the version number. If this was not specified then no specific version check would be done. Since it is, then the package must declare to be "compatible" with this version number in order to have status "found". Let us gloss over the entire version thing for now and just state that "3.0" is not compatible with "3.1", so if you have the former, this amounts to the package being "not found".

The last field is a flag that tells CMake's find_package how to proceed if the package is not found. If a package is marked as "required", then the configuration will halt with an error if the search for this package turns out unsuccessful. Otherwise, it will just continue, hoping that some variable is set (or not set) to indicate that this functionality is not available.

And this is what kicks in here, in opm-autodiff. if not eigen3_found then download.

But how does this work if anything depends on opm-autodiff? It will inherit the Eigen3 search, but also the non-requiredness and thus, if Eigen3 is not found, it will continue. Already downloaded when you built opm-autodiff you say? How does this work with a distro package, then? What if Eigen3 has been removed?

In my opinion, the download code is perfectly fine in an "umbrella" CMakeLists.txt that sets up the suite locally (which is what it was originally), but should never have been in opm-autodiff.

What I don't get is this: Why spend time on introducing more entropy into the build system just to save typing E-i-g-e-n-3 into some list of things to clone? I don't see the value.

akva2 commented 11 years ago

On 08/28/2013 11:42 AM, Roland Kaufmann wrote:

And this is what kicks in here, in opm-autodiff. if not eigen3_found then download.

But how does this work if anything depends on opm-autodiff? It will inherit the Eigen3 search, but also the non-requiredness and thus, if Eigen3 is not found, it will continue. Already downloaded when you built opm-autodiff you say? How does this work with a distro package, then? What if Eigen3 has been removed?

eigen3 is a pure template library and will never end up as a runtime dep. it's only a build time dep and if whomever builds the distro package removes it from the deps, guess who is to blame :)

In my opinion, the download code is perfectly fine in an "umbrella" CMakeLists.txt that sets up the suite locally (which is what it was originally), but should never have been in opm-autodiff.

What I don't get is this: Why spend time on introducing more entropy into the build system just to save typing E-i-g-e-n-3 into some list of things to clone? I don't see the value.

i agree in principle. i simply presented a possible solution and it was merged. i do think there is merit here though, as 1) this is a build-time only dep. ie only developers have the potential of ending up in trouble and they should be capable of handling it better than joe random. 2) it's not packaged in any distros yet, but an old version (3.0) is. ie it quells many potential end user questions.

rolk commented 11 years ago

Regarding Eigen3: I think this amounts to bundling one library into another, except that this is also download-over-Internet instead of having it in the repo. But I realize I haven't presented my case well enough to persuade the others, so I'm just going to let it rest.

Are there any other issues that are currently holding back the roll-up?

bska commented 11 years ago

Are there any other issues that are currently holding back the roll-up?

None that I know of. I'll test and (hopefully) merge OPM/opm-autodiff#26 at which time the roll-up should be good to go (once #350 and #351 are in). Maybe we should do a final build/install test just in case, but I think all (known) issues have been solved. The autodiff solution could possibly be revised at a later time.

rolk commented 11 years ago

All the pull-requests listed above have now been updated.

bska commented 11 years ago

@rolk

All the pull-requests listed above have now been updated.

That may be true, but I still see OPM/opm-porsol@f9fbd8a as the most recent commit in opm-porsol in a fresh clone. Did you push?

bska commented 11 years ago

That may be true, but I still see OPM/opm-porsol@f9fbd8a as the most recent commit in opm-porsol

Never mind. I can't read. Everything is FINE!

bska commented 11 years ago

Everything is FINE!

In that it works (builds and all tests pass in both Debug and Release modes) on

As far as I'm concerned, THIS META ISSUE CAN BE CLOSED AND ALL ROLL-UPS MERGED! Major milestone, and I must once more salute your persistence in getting this into the tree and for keeping up with my incessant nagging.

rolk commented 11 years ago

I have tested debug mode on Ubuntu 12.04 with ERT, DUNE 2.2.1, GCC 4.6, CMake 2.8.9 and Ninja myself. All modules build and all tests pass in that configuration, too.

I will leave this open until the afternoon (around 19:00 CEST) to allow anyone else to surface any issues; if nothing arise by then, everything will be merged.

andlaus commented 11 years ago

opm/opm-core#353 should also be considered for inclusion

rolk commented 11 years ago

As there is no more open issues pertaining to the rollup, I'll seize the opportunity and merge this thing.

bska commented 11 years ago

As there is no more open issues pertaining to the rollup, I'll seize the opportunity and merge this thing.

\o/
 |
/ \