Compiling params.h with new parameters

[pasansherath]

Hi @wsja,

I am trying to compile fraysum with increased values for maxlay (from 15 to 30) and maxph (from 40000 to 80000). Are there any other parameters that needs changing to accommodate new parameters?

Also, can you please explain the procedure to compile with new params.h?

Thanks very much.

[wasjabloch]

Thanks, I'm glad you ask! This is a point we haven't well automatized, yet, and the whole method is somewhat messy.

I'd like to send a warning ahead: In you computations, it may be important stick to Control(mults=0), because, with the factorial increase in phases being computed, mults=1 and mults=2 will most likely result in indefinite computing times.

First, to see what's going on, check in a Python or IPython what the current max dimensions of the arrays are:

from fraysum import run_bare
run_bare?

Watch out for the 15 in

Parameters
----------
thick : input rank-1 array('f') with bounds (**15**)

Now, navigate to: PyRaysum/pyraysum/src

open params.h and change:

      parameter (namelen=40,maxlay=15,maxtr=500,maxseg=45)
      parameter (maxph=40000,buffsize=120)

to

      parameter (namelen=40,maxlay=30,maxtr=500,maxseg=90)
      parameter (maxph=100000,buffsize=120)

Note to also change maxseg to three time maxlay and, perhaps just to be sure, to also increase maxph (this is the variable affected by the factorial increase when computing multiples).

In the next lines, you will: (1) pretend as if you have changed some source file, in order to trigger re-compilation (2) Go back to the root directory of the package (3) Compile the source files (4) Install PyRaysum form the newly compiled source

touch seis-spread.f
cd ../../
python setup.py build
pip install .

Now you can go and check in a Python console if everything worked. The array dimension should have changed to 30:

from fraysum import run_bare
run_bare?
(...)
Parameters
----------
thick : input rank-1 array('f') with bounds (**30**)

Does this work for you?

Next, when you do your computations, watch out to also increase the corresponding array dimensions on the Python side of the code. That is, when setting up your computations, do:

from pyraysum import Control, Model, Geometry
mod = Model(maxlay=30)
cntl = Control(maxseg=45, maxph=100000)

Please let me know about any success of failure.

Happy modelling!

[pasansherath]

Thanks @wsja. I followed your instructions but I get an error when I recompile using the python setup.py build command.

error: Command "/home/pasan/miniconda3/envs/rfpy/bin/x86_64-conda-linux-gnu-gfortran -Wall -g -Wall -g -shared -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,--allow-shlib-undefined -Wl,-rpath,/home/pasan/miniconda3/envs/rfpy/lib -Wl,-rpath-link,/home/pasan/miniconda3/envs/rfpy/lib -L/home/pasan/miniconda3/envs/rfpy/lib build/temp.linux-x86_64-3.7/build/src.linux-x86_64-3.7/fraysummodule.o build/temp.linux-x86_64-3.7/build/src.linux-x86_64-3.7/build/src.linux-x86_64-3.7/fortranobject.o build/temp.linux-x86_64-3.7/pyraysum/src/buildmodel.o build/temp.linux-x86_64-3.7/pyraysum/src/eigenvec.o build/temp.linux-x86_64-3.7/pyraysum/src/eispack-cg.o build/temp.linux-x86_64-3.7/pyraysum/src/matrixops.o build/temp.linux-x86_64-3.7/pyraysum/src/phaselist.o build/temp.linux-x86_64-3.7/pyraysum/src/raysum.o build/temp.linux-x86_64-3.7/pyraysum/src/readwrite.o build/temp.linux-x86_64-3.7/pyraysum/src/seis-spread.o build/temp.linux-x86_64-3.7/pyraysum/src/trace.o build/temp.linux-x86_64-3.7/build/src.linux-x86_64-3.7/fraysum-f2pywrappers.o -L/home/pasan/miniconda3/envs/rfpy/bin/../lib/gcc/x86_64-conda-linux-gnu/11.3.0/../../../../x86_64-conda-linux-gnu/lib/../lib -L/home/pasan/miniconda3/envs/rfpy/bin/../lib/gcc/x86_64-conda-linux-gnu/11.3.0/../../../../x86_64-conda-linux-gnu/lib/../lib -lgfortran -o build/lib.linux-x86_64-3.7/fraysum.cpython-37m-x86_64-linux-gnu.so" failed with exit status 1 Do you have a clue what this is about?

[wasjabloch]

It looks like it is not compiling, even though the compiler looks good to me. Are you in the right environment? That is, do you have setuptools and fortran-compiler installed it that environment? (It looks like you have, but that is the only guess I'm having right now)

[wasjabloch]

Could you test if

python setup.py build

works on a pristine copy of the repository? That is, could you do:

git restore pyraysum/src/params.h
python setup.py build

I'd like to know if the code is just not compiling, or not compiling because of the changes made.

[pasansherath]

I checked again, and I am in the correct environment with both setuptools and fortran-compiler installed.

I created a new environment with obspy, setuptools and fortran-compiler. Then I newly cloned the PyRaysum repo and did pip install . after cding into the PyRaysum dir. Then I tried importing fraysum into ipython. It returns an error ModuleNotFoundError: No module named 'fraysum.

I also tried changing the previously discussed parameters in params.h and recompiled it, but not sure if it compiled correctly. However, I get the same ModuleNotFoundError: No module named 'fraysum when I try to import it in ipython (in the same env).

[wasjabloch]

Thank you. This means that the Fortran part of the code (fraysum) is not compiling in the first place. I assume there are some error messages in the middle of of your pip install . output. Can you perhaps look out for those and post them here? Also, could you try if python setup.py build works with the pristine installation, or post the error message in case it doesn't?

If you see some error related to an unknown matrix eye(1,1), your NumPy Version is too new. In that case you would need to downgrade to <1.23.

[wasjabloch]

P.S.: Remember to test all this in a python console. When using IPython, remember to conda install ipython from within the same environment.

It might be necessary to conda install the dependencies one after another, in case conda cannot resolve the dependencies right away. I.e.:

conda create -n prs "python<=3.11" numpy<1.23 setuptools
conda activate prs
conda install fortran-compiler -c conda-forge
conda install obspy -c conda-forge
python setup.py build

[pasansherath]

I get these warnings with python setup.py build command, with the newly modified params.h

WARN: CCompilerOpt.__init__[977] : unable to detect compiler type which leads to treating it as GCC. this is a normal behavior if you're using gcc-like compiler such as MinGW or IBM/XLC.check dist_info:<<
('linux-x86_64', '/home/pasan/miniconda3/envs/prs2/bin/x86_64-conda-linux-gnu-cc', '-Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -Wstrict-prototypes -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/pasan/miniconda3/envs/prs/include -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/pasan/miniconda3/envs/prs/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/pasan/miniconda3/envs/prs/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/pasan/miniconda3/envs/prs/include')
>>

WARN: Could not locate executable armflang

When I run pip install . I get the following error at the end.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pyraysum
  Running setup.py clean for pyraysum
Failed to build pyraysum
ERROR: Could not build wheels for pyraysum, which is required to install pyproject.toml-based projects

[pasansherath]

Running python setup.py and subsequent pip install . on a pristine copy of PyRaySum (without making any modifications to params.h does not result in any error and I can import run_bare in fraysum in an ipython console. So I presume modifying params.h breaks the installation? Did it work fine for you when you tried?

[wasjabloch]

Yes, it did work for me. However, I needed multiple attempts, too. The warnings your are getting are safe to ignore.

I once also ran into the ERROR you are stating, after I aborted an installation process with Ctrl+C. I solved this issue by force-deleting the root package folder

rm -rf PyRaysum/

and cloning a clean repository

git clone https://github.com/paudetseis/PyRaysum

There may be cleaner ways to archive this, but at least this one worked for me.

After that, you can proceed to edit pyraysum/src/params.h and should be able to

touch pyraysum/src/seis-spread.f
python setup.py build
pip install .

Then, when you open a python console (just to be sure there are no IPython version conflicts), do:

from fraysum import run_bare
run_bare?

and you should see that the size of the thickness array (and the others) has, in fact, increased to the number you previously specified in params.h.