module reload/refresh fails when cycles exist in modulefiles

[ben-bowers]

Describe the bug

If modulefiles contain a "cycle" where one modulefile prereqs another modulefile that also does a prereq, a cycle is created and cannot be resolved by either reload or refresh. A standard module load does not apparently have an issue with breaking the circular dependency.

Why would one wish to have a cycle? In the case where some modulefile may want to load settings from another modulefile, but it is uncertain which modulefile the user would load. For example if doing module load AB, then also prereq file BA. If doing module load BA, then also prereq file AB.

To Reproduce

Steps to reproduce the behavior:

$ module load BA
Loading BA
  Loading requirement: AB

$ module refresh
ERROR: Cannot reload modules, some of their constraints are not satistied

$ module reload
ERROR: Cannot reload modules, some of their constraints are not satistied

Location and content of any modulerc or modulefile involved:

$ cat AB
#%Module1.0 # -*- tcl -*-
prereq BA
setenv AA aa

$ cat BA
#%Module1.0 # -*- tcl -*-
prereq AB
setenv BB bb

Expected behavior

Ideally both reload and refresh could break the cycle and not fail to execute. I do not understand the internals of modules (or what breadcrumbs are saved in the environment) well enough to know if this is possible.

If this is not possible, is there a suggested workaround for this limitation?

Error and debugging information

Using debug mode enough output is printed to help me diagnose that some cycle was found, which presumably is the root issue.

$ module reload -D
...
DEBUG runModulerc: running...
DEBUG cmdModuleReload: reloading BA AB
DEBUG getUnloadMatchOrder: unload_match_order set to 'returnlast'
DEBUG setModuleDependency: set an unmet requirement on 'AB' for 'BA'
DEBUG setModuleDependency: set prereq violation state for 'BA'
DEBUG doesModuleConflict: 'BA' conflicts with '' (declared as '')
DEBUG setModuleDependency: set requirements of 'BA' to ''
DEBUG setModuleDependency: set NPO requirements of 'BA' to 'AB'
DEBUG doesModuleConflict: 'AB' conflicts with '' (declared as '')
DEBUG setModuleDependency: refresh requirements targetting 'AB'
DEBUG getDependentLoadedModuleList: get loaded mod dependent of 'BA' (strong=0, direct=0, nporeq=0, loading=1, sat_constraint=0, being_unload=0)
DEBUG getDependentLoadedModuleList: got 'AB'
DEBUG setModuleDependency: skip deps refresh for 'BA' as dep cycle detected with 'AB'
DEBUG setModuleDependency: update NPO dependent of 'AB' to ''
DEBUG setModuleDependency: set NPO prereq violation state for 'BA'
DEBUG setModuleDependency: update NPO requirement of 'BA' to '{}'
DEBUG setModuleDependency: set requirements of 'AB' to 'BA'
DEBUG setModuleDependency: set NPO requirements of 'AB' to 'BA'
DEBUG cacheCurrentModules: 2 loaded
ERROR: Cannot reload modules, some of their constraints are not satistied
DEBUG renderSettings: called.
DEBUG renderSettings: 1 error(s) detected.
DEBUG renderFalse: called.

Modules version and configuration

$ module --version
Modules Release 4.3.1 (2019-09-21)

$ module config --dump-state
...
MODULES_AUTO_HANDLING     1
...

Additional context

[xdelaruelle]

Thanks for the report. What you observe is the current expected behavior for these reload/refresh commands: an error is returned and nothing happens if current loaded environment is seen inconsistent. This is done not to worsen the bad situation the user is currently in.

Dependency cycles should be avoided. load or unload cope well with them but as a result the first loaded module is seen inconsistent as its requirement is loaded after it and not before it.

I am interested to get a concrete use case example where a dependency cycle is needed.

[xdelaruelle]

I am interested to get a concrete use case example where a dependency cycle is needed.

This mutual dependency can usually be worked-around either by:

• merging both modules into a single one (modules A and B become a single module AB)
• creating a third module that is the real common dependency of the 2 initial modules (add module C which is required by module A and module B)

[ben-bowers]

Hi @xdelaruelle I am not sure there is a "concrete use case", but more accurately multiple use cases (and a 3+ node cycle) that involved different users trying to use modules a slightly different way with a different entry point (but net result was trying to load A,B,C in some sequence)

I have been told that they intend to remove the cycle and so I am fine with closing this issue, or could be left as a case of improving documentation.

[xdelaruelle]

Hi @ben-bowers I will add mention of this in the documentation, so the issue can be left open for the moment.