ShyLU: ShyLU/ShyLUConfig.cmake:164 (include): could not find requested file: ../ShyLU_Node/ShyLU_NodeConfig.cmake

[eugeneswalker]

Bug Report

@srajama1

Description

Trilinos@13.4.1 built with +shylu using Spack contains a ShyLUConfig.cmake that wants to include a non-existent ShyLU_NodeConfig.cmake.

$> sed -n '164p' $(spack location -i trilinos+shylu)/lib/cmake/ShyLU/ShyLUConfig.cmake
include("${CMAKE_CURRENT_LIST_DIR}/../ShyLU_Node/ShyLU_NodeConfig.cmake")

$> file $(spack location -i trilinos+shylu)/lib/cmake/ShyLU/../ShyLU_Node/ShyLU_NodeConfig.cmake
...  (No such file or directory)

$> find $(spack location -i trilinos+shylu) -type f -name ShyLU_NodeConfig.cmake
$>

Here is the exact trilinos@13.4.1 +shylu that was installed:

$> spack find -lv trilinos+shylu
-- linux-ubuntu20.04-x86_64 / gcc@11.1.0 ------------------------
fwbxvsw trilinos@13.4.1~adelus~adios2+amesos+amesos2+anasazi+aztec~basker+belos+boost~chaco~complex~cuda~cuda_rdc~debug~dtk+epetra+epetraext~epetraextbtf~epetraextexperimental~epetraextgraphreorderings~exodus+explicit_template_instantiation~float+fortran~gtest~hdf5~hypre+ifpack+ifpack2+intrepid+intrepid2~ipo+isorropia+kokkos~mesquite+minitensor+ml+mpi+muelu~mumps+nox~openmp~panzer+phalanx+piro~python~rocm~rocm_rdc+rol+rythmos+sacado~scorec+shards+shared+shylu+stk+stokhos+stratimikos~strumpack~suite-sparse~superlu+superlu-dist+teko+tempus+thyra+tpetra+trilinoscouplings~wrapper~x11+zoltan+zoltan2 build_system=cmake build_type=Release cxxstd=14 generator=make gotype=long_long
==> 1 installed package

Steps to Reproduce

Using spack@develop (2dc76248d31 from Fri Jul 7 23:27:32 2023 +0200) install the following concrete spec.

trilinos-shylu.spec.json.txt

$> docker run -it ecpe4s/ubuntu20.04-runner-x86_64-gcc-11.1:2023.06.01
root@c01ef2221384:/ git clone https://github.com/spack/spack
root@c01ef2221384:/ (cd spack && git checkout 2dc76248d3154c500fa16edfbc47801d122c7859) 
root@c01ef2221384:/ . spack/share/spack/setup-env.sh
root@c01ef2221384:/ spack install -f ./trilinos-shylu.spec.json
... OK

Once trilinos+shylu is installed, check $(spack location -i trilinos+shylu)/lib/cmake/ShyLU/ShyLUConfig.cmake and you will see on line 164 the attempt to include a non-existent ShyLU_NodeConfig.cmake.

@nchaimov @wspear

[jhux2]

@trilinos/shylu

[eugeneswalker]

Any update on this?

[cgcgcg]

@trilinos/shylu

[ndellingwood]

Does this occur with a recent release of Trilinos like 14.2.0 or 14.4.0?

[eugeneswalker]

Does this occur with a recent release of Trilinos like 14.2.0 or 14.4.0?

Yes.

Here is with Trilinos 14.4.0 installed with Spack, +shylu, installed using spack@develop in the past couple days.

$> grep -n ShyLU_NodeConfig.cmake $(spack location -i trilinos@14.4.0)/lib/cmake/ShyLU/ShyLUConfig.cmake
135:  include("${CMAKE_CURRENT_LIST_DIR}/../ShyLU_Node/ShyLU_NodeConfig.cmake")

$> file $(spack location -i trilinos@14.4.0)/lib/cmake/ShyLU_Node/ShyLU_NodeConfig.cmake
# doesn't exist

$> ls -l $(spack location -i trilinos@14.4.0)/lib/cmake | grep ShyLU_Node
$>

[DaxLynch]

I just had this issue occur on perlmutter with e4s/23.05. After attempting to install xyce, which used trilinos@14.0.0%gcc@11.2.0.......+shylu+.........arch=linux-sles15-zen3, I got the error

     17      /global/homes/d/daxlynch/spack-workspace/perlmutter/software/linux-sles15-zen3/gcc-11.2.0/trilinos-14.0.0-7imkbs2czxflc
           o3s7ijirjhre2u2zdet/lib/cmake/Stratimikos/StratimikosConfig.cmake:148 (include)
     18      /global/homes/d/daxlynch/spack-workspace/perlmutter/software/linux-sles15-zen3/gcc-11.2.0/trilinos-14.0.0-7imkbs2czxflc
           o3s7ijirjhre2u2zdet/lib/cmake/NOX/NOXConfig.cmake:185 (include)
     19      /global/homes/d/daxlynch/spack-workspace/perlmutter/software/linux-sles15-zen3/gcc-11.2.0/trilinos-14.0.0-7imkbs2czxflc
           o3s7ijirjhre2u2zdet/lib/cmake/Trilinos/TrilinosConfig.cmake:123 (include)
     20      CMakeLists.txt:42 (find_package)
     21
     22
  >> 23    CMake Error at /global/homes/d/daxlynch/spack-workspace/perlmutter/software/linux-sles15-zen3/gcc-11.2.0/trilinos-14.0.0-
           7imkbs2czxflco3s7ijirjhre2u2zdet/lib/cmake/ShyLU/ShyLUConfig.cmake:135 (include):
     24      include could not find requested file:
     25
     26        /global/homes/d/daxlynch/spack-workspace/perlmutter/software/linux-sles15-zen3/gcc-11.2.0/trilinos-14.0.0-7imkbs2czxf
           lco3s7ijirjhre2u2zdet/lib/cmake/ShyLU/../ShyLU_Node/ShyLU_NodeConfig.cmake
     27    Call Stack (most recent call first):
     28      /global/homes/d/daxlynch/spack-workspace/perlmutter/software/linux-sles15-zen3/gcc-11.2.0/trilinos-14.0.0-7imkbs2czxflc
           o3s7ijirjhre2u2zdet/lib/cmake/Trilinos/TrilinosConfig.cmake:123 (include)
     29      CMakeLists.txt:42 (find_package)

[ndellingwood]

I tested a build of Trilinos outside of spack where I enable ShyLU explicitly but neither of the the sub-packages (ShyLU_Node or ShyLU_DD) and confirmed there is no ShyLU_NodeConfig.cmake file, as reported in this issue.

Some relevant output from CMake during configuration:

Explicitly enabled top-level packages on input (by user):  Kokkos ShyLU 2

Explicitly enabled packages on input (by user):  Kokkos ShyLU 2

Explicitly disabled top-level packages on input (by user or by default):  PyTrilinos NewPackage 2

Explicitly disabled packages on input (by user or by default):  ShyLU_NodeBasker ShyLU_NodeFastILU SEACASExotec2 STKMiddle_mesh STKSearchUtil STKTransferUtil  PanzerExprEval PyTrilinos NewPackage 9
...
-- Setting Trilinos_ENABLE_ShyLU_DD=ON because ShyLU has a required dependence on ShyLU_DD
-- Setting Trilinos_ENABLE_ShyLU_Node=ON because ShyLU has a required dependence on ShyLU_Node
-- Setting Trilinos_ENABLE_ShyLU_DDFROSch=ON because ShyLU_DD has a required dependence on ShyLU_DDFROSch
-- Setting Trilinos_ENABLE_ShyLU_DDCore=ON because ShyLU_DD has a required dependence on ShyLU_DDCore
...

In particular, enabling ShyLU triggers enables of ShyLU_DD and ShyLU_Node, as they are required dependencies; ShyLU_DD has two required component sub-packages (FROSch and Core) which are enabled, but none of the solvers in ShyLU_Node are required to be enabled which I'm guessing results in the issues seen and the unexpected behavior of the missing cmake file.

I think the easiest path around this is to change the dependency of ShyLU on ShyLU_Node to optional rather than required. Alternatively, we can make Tacho a requirement of ShyLU_Node since Tacho is PT, but Tacho is not compatible with the Threads backend so I don't know if this may cause issues for Threads builds? @iyamazaki or @srajama1 any preferences or additional thoughts?

Back to the actual issue, I'm not sure what to pinpoint as the defect here and where/what safeguard is missing:

• Should a "dummy" ShyLU_NodeConfig.cmake have been present regardless?
• Is the dependency structure incorrectly implemented here (user error) - that is, ShyLU depends on ShyLU_Node as a required dependency, but ShyLU_Node has no required dependencies, should this type of dependency chain be disallowed? If so, changing ShyLU_Node to an optional dependency seems the right thing to do, but how do we prevent this situation from slipping through (where and how to check for that)?

[DaxLynch]

I tried to get around this by removing the offending line, /global/homes/d/daxlynch/spack-workspace/perlmutter/software/linux-sles15-zen3/gcc-11.2.0/trilinos-14.0.0-7imkbs2czxflco3s7ijirjhre2u2zdet/lib/cmake/ShyLU/ShyLUConfig.cmake:135. This did not work.

I also tried to get around it by changing packages/shylu/shylu_node/cmake/Dependencies.cmake

  #SubPackageName       Directory         Class    Req/Opt
  #
  # New ShyLU subpackages:
  HTS                   hts               ST       OPTIONAL
  Tacho                 tacho             PT       OPTIONAL
  Basker                basker            EX       OPTIONAL
  FastILU               fastilu           EX       OPTIONAL
  ) 

into

SET(SUBPACKAGES_DIRS_CLASSIFICATIONS_OPTREQS
  #SubPackageName       Directory         Class    Req/Opt
  #
  # New ShyLU subpackages:
  HTS                   hts               ST       REQUIRED
  Tacho                 tacho             PT       REQUIRED
  Basker                basker            EX       REQUIRED
  FastILU               fastilu           EX       REQUIRED
  )

I thought this might trigger ShyLU_NodeConfig.cmake to be made but it didn't seem to.

One thought with making ShyLU_Node optional is that it seems like it then won't get built, but it seems likely its required for ShyLU functionality, hence why we should require it if we want ShyLU. Take this concern with a grain of salt however, as I have barely any idea what is going on.

[ndellingwood]

I tested two separate types of changes to the Dependencies files:

1. Change shylu/cmake/Dependencies: Make ShyLU_Node an optional lib of ShyLU Test 1: Trilnos_ENABLE_ShyLU=ON Result: In /lib/cmake, there are directories for ShyLU, ShyLU_DD, ShyLU_DDCore, ShyLU_DDFROSch, all with their expected Config.cmake and Targets.cmake files; ShyLU_Node is not present (and thus there is not a missing Config.cmake file that was expected) Test 2: Trilnos_ENABLE_ShyLU=ON and Trilnos_ENABLE_ShyLU_NodeTacho=ON Result: As above, with additional directories for ShyLU_Node and ShyLU_NodeTacho containing their expected Config.cmake and Targets.cmake files

2. Change shylu/shylu_node/cmake/Dependencies: Make Tacho a REQUIRED dependency Test 1: Trilnos_ENABLE_ShyLU=ON Result: In /lib/cmake, there are directories for ShyLU, ShyLU_DD, ShyLU_DDCore, ShyLU_DDFROSch, all with their expected Config.cmake and Targets.cmake files; ShyLU_Node and ShyLU_NodeTacho present and also containing their expected Config.cmake and Targets.cmake files

For now I put in PR #12245 which makes ShyLU_Node an optional dependency of ShyLU and seemed reasonable considering the solvers of ShyLU_Node are all designated as Optional dependencies, and the result is that when only using Trilnos_ENABLE_ShyLU=ON to enable ShyLU capabilities the install will have directories and corresponding cmake configuration files for ShyLU, ShyLU_DD, ShyLU_DDCore, ShyLU_DDFROSch. Request for any component of ShyLU_Node must be made explicity, which maintains the current behavior, and doing so will result in the install directories and cmake config files as expected.

ShyLU_DD and ShyLU_Node are independent subpackages, so making ShyLU_Node optional has no impact on ShyLU_DD.

An alternative is to make Tacho a required dependency of ShyLU_Node, but I am uncertain what the requirements are for making such a change, and I think #12245 is the safest and fastest path to workaround the issue

[eugeneswalker]

@ndellingwood Since ShyLU can be on while all ShyLU_Node solvers are off, is it fair to say there may be a case where someone would actually want this? Would someone ever want ShyLU on but have no node solvers? We are thinking about how to update the Trilinos Spack package and are wondering if it would make sense to make some variants for each of the ShyLU Node solvers and require that at least one of them be turned on in the case that someone wants +shylu.

What do you think?

[github-actions[bot]]

This issue has had no activity for 365 days and is marked for closure. It will be closed after an additional 30 days of inactivity. If you would like to keep this issue open please add a comment and/or remove the MARKED_FOR_CLOSURE label. If this issue should be kept open even with no activity beyond the time limits you can add the label DO_NOT_AUTOCLOSE. If it is ok for this issue to be closed, feel free to go ahead and close it. Please do not add any comments or change any labels or otherwise touch this issue unless your intention is to reset the inactivity counter for an additional year.