[JOSS] Installation issues - S4

[EricaEgg]

I have installed S4 and rayflare as described in the docs. However, when importing functions for rayflare, I get an error.

This is the import I tried:

from rayflare.matrix_formalism import process_structure, calculate_RAT, get_savepath

This was the resulting error:

ImportError: dlopen(/Users/user/mainenv/lib/python3.9/site-packages/S4.cpython-39-darwin.so, 2): Library not loaded: @rpath/libboost_serialization.dylib
  Referenced from: /Users/user/mainenv/lib/python3.9/site-packages/S4.cpython-39-darwin.so
  Reason: image not found

I am using the macOS teriminal and Jupyter notebooks.

Thank you for the help!

[phoebe-p]

(sorry, closing the issue was a wrong click!)

Just to confirm, did you run make boost before making S4, and did it seem to execute successfully? Since the issue seems to be related to the boost library.

Also to check what's going on with S4, could you try just (in Python):

import S4

I guess that will give an error but if not, what happens with:

S = S4.New(((1,0),(0,1)), 5)

I don't currently have access to a Mac but have successfully installed S4 on it in the past so hopefully we can make it work!

[phoebe-p]

Another thing: RayFlare is designed to still work even when S4 is not installed (if you want to use it on Windows, or if you are simply not interested in the RCWA functionality). When I run the same imports as you in Windows (S4 not installed), I get:

from rayflare.matrix_formalism import process_structure, calculate_RAT, get_savepath
WARNING: The RCWA solver will not be available because an S4 installation has not been found.
Backend TkAgg is interactive backend. Turning interactive mode on.
WARNING: The RCWA solver will not be available because an S4 installation has not been found.

So it warns me S4 is not installed, rather than throwing an error. I can run the examples which don't involve RCWA as well. It seems because you have attempted to install S4 but something went wrong, you are now in a situation where RayFlare can't recognize you actually don't have the install, so now nothing works anymore. Hopefully we can fix the S4 install so everything works, but I think the first step has to be deleting the version of S4 you currently have from the Python package tree since it's obviously faulty in some way.

[phoebe-p]

@EricaEgg any luck deleting the current S4 install/figuring out what was wrong with it?

[EricaEgg]

Another thing: RayFlare is designed to still work even when S4 is not installed (if you want to use it on Windows, or if you are simply not interested in the RCWA functionality). When I run the same imports as you in Windows (S4 not installed), I get:

from rayflare.matrix_formalism import process_structure, calculate_RAT, get_savepath
WARNING: The RCWA solver will not be available because an S4 installation has not been found.
Backend TkAgg is interactive backend. Turning interactive mode on.
WARNING: The RCWA solver will not be available because an S4 installation has not been found.

So it warns me S4 is not installed, rather than throwing an error. I can run the examples which don't involve RCWA as well. It seems because you have attempted to install S4 but something went wrong, you are now in a situation where RayFlare can't recognize you actually don't have the install, so now nothing works anymore. Hopefully we can fix the S4 install so everything works, but I think the first step has to be deleting the version of S4 you currently have from the Python package tree since it's obviously faulty in some way.

Thanks for this tip! I uninstalled S4, and I was able to run all examples that do not require RCWA.

[phoebe-p]

I am not sure if this was the cause of your issue, but several users have identified a problem when installing S4 on MacOS. This must be a fairly recent problem since I have successfully installed S4 on Apple machines in the past, so I am unsure when it started, but here is a snippet of the (very long) error I'm getting:

    S4/main_python.c:674:11: error: implicit declaration of function 'Simulation_LoadSolution' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        err = Simulation_LoadSolution(&(self->S), fname);
              ^
    S4/main_python.c:674:11: note: did you mean 'Simulation_InitSolution'?
    S4/S4.h:346:5: note: 'Simulation_InitSolution' declared here
    int Simulation_InitSolution(Simulation *S);
        ^
    S4/main_python.c:686:11: error: implicit declaration of function 'Simulation_SaveSolution' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        err = Simulation_SaveSolution(&(self->S), fname);
              ^
    S4/main_python.c:686:11: note: did you mean 'Simulation_InitSolution'?
    S4/S4.h:346:5: note: 'Simulation_InitSolution' declared here
    int Simulation_InitSolution(Simulation *S);
        ^
    S4/main_python.c:1489:23: warning: incompatible pointer types passing 'PyObject *' (aka 'struct _object *') to parameter of type 'PyArrayObject *' (aka 'struct tagPyArrayObject_fields *') [-Wincompatible-pointer-types]
      PyArray_ENABLEFLAGS(Earr, NPY_ARRAY_OWNDATA);
                          ^~~~
    /Users/phoebe/mainenv/lib/python3.9/site-packages/numpy/core/include/numpy/ndarraytypes.h:1644:36: note: passing argument to parameter 'arr' here
    PyArray_ENABLEFLAGS(PyArrayObject *arr, int flags)
                                       ^
    S4/main_python.c:1500:23: warning: incompatible pointer types passing 'PyObject *' (aka 'struct _object *') to parameter of type 'PyArrayObject *' (aka 'struct tagPyArrayObject_fields *') [-Wincompatible-pointer-types]
      PyArray_ENABLEFLAGS(Harr, NPY_ARRAY_OWNDATA);
                          ^~~~
    /Users/phoebe/mainenv/lib/python3.9/site-packages/numpy/core/include/numpy/ndarraytypes.h:1644:36: note: passing argument to parameter 'arr' here
    PyArray_ENABLEFLAGS(PyArrayObject *arr, int flags)
                                       ^
    S4/main_python.c:2172:14: warning: unused variable 'layerName' [-Wunused-variable]
            const char *layerName;
                        ^
    S4/main_python.c:2171:15: warning: unused variable 'kwlist' [-Wunused-variable]
            static char *kwlist[] = {"Layer", "Simulations", NULL};
                         ^
    5 warnings and 2 errors generated.
    error: command '/usr/bin/clang' failed with exit code 1
    ----------------------------------------

The failures are caused by the two errors, both of which are caused by error: implicit declaration of function 'Simulation_LoadSolution' is invalid in C99. I have to say I do not understand why this was not an issue previously but is now, since the installation on Linux similarly uses the gnu99/C99 standard (this is specified in gensetup.py.sh in S4) but doesn't cause an error. My knowledge of C++ is very rusty, but this appears to be relatively easy to avoid, so I will make a pull request shortly in my [forked S4 repository]() which hopefully addressed this issue.

[phoebe-p]

I think PR #39 should address this issue - @EricaEgg I think your issue was due to the second point in #39, the boost library not being linked correctly. This was fixed on my own macOS system by installing boost via homebrew (brew install boost) and then compiling the S4 extension again.