Parallel run with method `scotch` not working

[massisenergy]

Hi Guanyang Xue,

Happy that it's working and single core performance seems better than my desktop with i9 + 32GB RAM. Found another problem now. Parallel run with the method simple is working but it blurts out the following when I tried to use scotch:

/*---------------------------------------------------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  v2106                                 |
|   \\  /    A nd           | Website:  www.openfoam.com                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
Build  : _c15bfde3cb-20210624 OPENFOAM=2106
Arch   : "LSB;label=32;scalar=64"
Exec   : decomposePar -force
Date   : Jan 01 2022
Time   : 20:30:35
Host   : coldandwhite.local
PID    : 77116
I/O    : uncollated
Case   : /Users/massisenergy/OpenFOAM/pitzDaily
nProcs : 1
trapFpe: Floating point exception trapping - not supported on this platform
fileModificationChecking : Monitoring run-time modified files using timeStampMaster (fileModificationSkew 5, maxFileModificationPolls 20)
allowSystemOperations : Allowing user-supplied system call operations

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time

Decomposing mesh

Removing 8 existing processor directories
Create mesh

Calculating distribution of cells
Decomposition method scotch [8] (region region0)

--> FOAM FATAL ERROR: (openfoam-2106)
Attempted to use <scotch> without the scotchDecomp library loaded.
This message is from the dummy scotchDecomp stub library instead.

Please install <scotch> and ensure libscotch.so is in LD_LIBRARY_PATH.
The scotchDecomp library can then be built from src/parallel/decompose/scotchDecomp.
Dynamically loading or linking this library will add <scotch> as a decomposition method.

    From virtual Foam::labelList Foam::scotchDecomp::decompose(const Foam::polyMesh &, const Foam::pointField &, const Foam::scalarField &) const
    in file dummyScotchDecomp.C at line 102.

FOAM exiting

Is it working for you? How do I go about this?

I have installed scotch from home-brew:

$brew info scotch
scotch: stable 6.1.3 (bottled), HEAD
Package for graph partitioning, graph clustering, and sparse matrix ordering
https://gitlab.inria.fr/scotch/scotch
/opt/homebrew/Cellar/scotch/6.1.2 (76 files, 5.7MB) *
  Poured from bottle on 2021-12-28 at 12:08:27
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/scotch.rb
License: CECILL-C
==> Dependencies
Required: open-mpi ✔
==> Options
--HEAD
    Install HEAD version
==> Analytics
install: 53 (30 days), 227 (90 days), 524 (365 days)
install-on-request: 51 (30 days), 220 (90 days), 499 (365 days)
build-error: 0 (30 days)

[BrushXue]

The homebrew scotch won’t work because they refuse to add a compiler flag. You need to to use the scotch.rb in my repo.

[BrushXue]

Or, you can add a issue so they can know the severity of the problem and hopefully they’ll make a change.

[BrushXue]

Also seems like your scotch library is not properly linked. You probably need to modify etc/config.sh/scotch and change the path(mentioned in README).

[massisenergy]

Hello again,

Sorry I didn't find any instruction in the Readme.md, how to use the scotch.rb. I checked in my home-brew scotch installation, but there also exists no scotch.rb, that I could just replace to make it work. Can you please instruct me in detail?

[BrushXue]

This is not related to OpenFOAM so I’m not really responsible for this part. I suggest you learn how to use OpenFOAM on Linux then try it on Mac. You need to download the script and do brew install --build-from-source scotch.rb. Then change the PATH in etc/config.sh/scotch to /opt/homebrew/opt/scotch

[massisenergy]

Thanks so much Xue! Decomposition using the scotch method is working now 🙂. I learnt how to use a custom .rb with brew install --build-from-source.

My OpenFoam installation in ArchLinux just worked out of the box, never posed the requirement of any modification during building so...

Maybe add this part download the script and do brew install --build-from-source scotch.rb into the readme for newbies like me?

[gerlero]

https://github.com/Homebrew/homebrew-core/pull/95320