Open git-tea opened 2 years ago
Sorry to hear this—not aware of this problem, but it's been a while since I was working on this project. Can you try installing everything from source (i.e. from the src
directory of this repo)?
Thank you for the quick response!
I have tried to install via src, and I get the output in the following file: graphkernels_src_install.txt
I do not see any obvious errors during the installation. However, I seem to come across a second error later on.
When I try to call CalculateWLKernel from my script, I get the following error: kernel_mat = gk.CalculateWLKernel( relabeled_graphs, n_iters ) File "/opt/GraphKernels/src/graphkernels/graphkernels/kernels.py", line 160, in CalculateWLKernel E, V_label, V_count, E_count, D_max = GetGKInput(G) File "/opt/GraphKernels/src/graphkernels/graphkernels/utilities.py", line 71, in GetGKInput E = gkCpy.VecMatrixXi() AttributeError: module 'GKextCPy' has no attribute 'VecMatrixXi'
Is there a separate path that I need to link to be able to locate these attributes?
That's weird—can you load the module and tell me the output of the following command, please:
import pathlib, GKextCPy; pathlib.Path(GKextCPy.__file__).resolve().parent
The following is what is returned from running that command in the python interpreter:
">>> import pathlib, GKextCPy; pathlib.Path(GKextCPy.file).resolve().parent
Traceback (most recent call last):
File "/opt/GraphKernels/src/GKextCPy/GKextCPy.py", line 14, in swig_import_helper
return importlib.import_module(mname)
File "/opt/anaconda3/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "
This looks to me like the extension module was not installed. Can you try to install this manually as well (from the GKextCPy
folder)?
I believe that I have now installed GKextCPy from src
using the corresponding setup.py file.
I ran python setup.py build
and then python setup.py install --user
like with graphkernels. I then repeated the build and installation of graphkernels.
I get the same error when I repeat the test from above (import pathlib, GKextCPy; pathlib.Path(GKextCPy.__file__).resolve().parent
)
I don't understand this fully. Is the module maybe already installed (if somewhat partially)? The setup script might refuse to overwrite it.
I have cleared all the references I can find to the module and tried to install it from source from scratch.
After clearing everything, the entire local GraphKernels directory was gone, and there was no listing of either GKextCPy or graphkernels in pip list
.
I re-cloned the repository and both built and installed GKextCPy and then graphkernels. I also built and installed with sudo permissions to ensure no access problems. The following files correspond to each command, such that the setup.py
files are the ones inside their corresponding package.
GKextCPy - sudo ./setup.py build
: gkcpy_build.txt
GKextCPy - sudo ./setup.py install --user
: gkcpy_install.txt
graphkernels - sudo ./setup.py build
: gk_build.txt
graphkernels - sudo ./setup.py install --user
: gk_install.txt
When I then run the command above import pathlib, GKextCPy; pathlib.Path(GKextCPy.__file__).resolve().parent
from a python interpreter in my home directory, I get the following error:
->>> import pathlib, GKextCPy; pathlib.Path(GKextCPy.file).resolve().parent
Traceback (most recent call last):
File "
Sorry for the hassle here...
It seems that the packages are being installed for the root user only. If you use sudo
, you can drop the --user
flag. This should install the packages globally (might cause other problems, though; maybe a virtual environment would be best).
It's no problem. I appreciate you taking the time to help with this.
I've deleted and re-installed it again to make sure there's no conflict, and I installed it without the --user flag. I seem to be getting the same ModuleNotFoundError when running the test command.
Do I need to run the command from within a specific directory to ensure that all packages are locate-able?
When I run the test from somewhere outside of the GraphKernels directories, I get the ModuleNotFoundError above: ->>> import pathlib, GKextCPy; pathlib.Path(GKextCPy.file).resolve().parent Traceback (most recent call last): File "", line 1, in ModuleNotFoundError: No module named 'GKextCPy'
When I run within GraphKernels/src
, I get the following error:
->>> import pathlib, GKextCPy; pathlib.Path(GKextCPy.file).resolve().parent
Traceback (most recent call last):
File "
And when I run within the GraphKernels/src/GKextCPy
directory, I get the following errors:
->>> import pathlib, GKextCPy; pathlib.Path(GKextCPy.file).resolve().parent
Traceback (most recent call last):
File "/opt/GraphKernels/src/GKextCPy/GKextCPy.py", line 14, in swig_import_helper
return importlib.import_module(mname)
File "/opt/anaconda3/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "
Still looks like there's something wrong with the installation. I see that you have Anaconda active as well. Can you try installing it in the Anaconda environment using --user
and no use of sudo
?
(I just pushed a simplified module installation fix, but that should not apply to your issue, unfortunately. If we can't get anywhere, the main developers @mahito-sugiyama or @eghisu might want to weigh in.)
When you say to install it in the Anaconda environment, do you mean using pip to install it? Or do you mean to install it into the directories for the conda packages?
I mean installing it using setup.py
while the Anaconda environment is active, but without using sudo
. From what I saw in the logs, your previous installation used a different Python version than the one your Anaconda environment uses.
The package is within my /opt directory, so it has limited permissions by default.
To account for that, I have now added write permissions (using chmod +w) to each of the directories of GraphKernels so that the build process can create what it needs to.
Afterward, I ran python3 setup.py build
on the setup.py file in GKextCPy, and it returned the following error. It appears to be the same error that I saw when installing via pip:
Requirement already satisfied: numpy in /opt/anaconda3/lib/python3.9/site-packages (1.21.5)
Requirement already satisfied: pkgconfig in /opt/anaconda3/lib/python3.9/site-packages (1.5.5)
running build
running build_py
creating build/lib.linux-x86_64-3.9
copying GKextCPy.py -> build/lib.linux-x86_64-3.9
running build_ext
building '_GKextCPy' extension
creating build/temp.linux-x86_64-3.9
gcc -pthread -B /opt/anaconda3/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /opt/anaconda3/include -I/opt/anaconda3/include -fPIC -O2 -isystem /opt/anaconda3/include -fPIC -I/opt/anaconda3/lib/python3.9/site-packages/numpy/core/include -I -I/opt/anaconda3/include/python3.9 -c GKextCPy.cpp -o build/temp.linux-x86_64-3.9/GKextCPy.o -std=c++11 -O3
gcc -pthread -B /opt/anaconda3/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /opt/anaconda3/include -I/opt/anaconda3/include -fPIC -O2 -isystem /opt/anaconda3/include -fPIC -I/opt/anaconda3/lib/python3.9/site-packages/numpy/core/include -I -I/opt/anaconda3/include/python3.9 -c GKextCPy_wrap.cxx -o build/temp.linux-x86_64-3.9/GKextCPy_wrap.o -std=c++11 -O3
GKextCPy_wrap.cxx:173:11: fatal error: Python.h: No such file or directory
173 | # include ~~~~~
compilation terminated.
error: command '/usr/bin/gcc' failed with exit code 1
Do you have the dev files for Python installed within your Anaconda environment? I think the command you ran at the very beginning only installs them for your global installation.
I am not sure I understand the discussion on the /opt
dir. What I would suggest is to clone the package into your home, activate your conda env, then run setup.py
from there (after potentially installing dev headers; the package is named python-devtools
or something).
I confirmed that I already have the devtools within the conda environment:
$ conda install -c conda-forge python-devtools Collecting package metadata (current_repodata.json): done Solving environment: done All requested packages already installed.
Additionally, I have cloned the repository into my home directory and tried to build GKextCPy from there, but I seem to get the same 'Python.h: No such file or directory' error.
Specifically, when I run python setup.py build
from the GKextCPy directory, I get the output at the following file:
gkcpy_build.txt
Things are getting curiouser and curiouser. Can you check where the file exists for your current Python installation? (It seems at this point that this is not necessary a problem with our setup.py
since we cannot do more than rely on the paths provided in the system.)
The following are the paths I know of related to Python:
Python binary: /opt/anaconda3/bin/python
Python.h header: /usr/include/python3.9/Python.h
Is there a separate python file or directory that would be helpful to locate?
What I find weird is that the second directory is not being used during the setup process. Can you try providing this directory as an additional CFLAGS
argument before calling setup.py
?
Thank you for being patient while I work on this.
It seems that the place that /usr/include/python3.9
was not included was in the sysconfig
settings. So I updated the GKextCPy setup file such that the extra compile arguments takes this form:
extra_compile_args = ['-std=c++11', '-O3', '-I/usr/include/python3.9/']
This update allowed GKextCPy and graphkernels to build and install from src seemingly without errors.
However, when I try to run the test import command above (import pathlib, GKextCPy; pathlib.Path(GKextCPy.__file__).resolve().parent
), I get the following error. The test command was run from outside of the Graphkernels directory after the installation completed:
>>> import pathlib, GKextCPy; pathlib.Path(GKextCPy.file).resolve().parent
Traceback (most recent call last):
File "/opt/GraphKernels/src/GKextCPy/GKextCPy.py", line 14, in swig_import_helper
return importlib.import_module(mname)
File "/opt/anaconda3/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named '_GKextCPy'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "", line 1, in
File "/opt/GraphKernels/src/GKextCPy/GKextCPy.py", line 17, in
_GKextCPy = swig_import_helper()
File "/opt/GraphKernels/src/GKextCPy/GKextCPy.py", line 16, in swig_import_helper
return importlib.import_module('_GKextCPy')
File "/opt/anaconda3/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named '_GKextCPy'
Hmm, what I don't understand about this issue is that the Anaconda headers should 'live' under a different path than /usr/include
. But in any case: the module seems to build and install now—but I am guessing that from your output, not even the normal import worked...
Can you please attach the output of python setup.by install --user
for the GKextCPy
module? Seems like something is very wrong with the paths.
(Not the main developer here; at this point I would really appreciate any additional input from @mahito-sugiyama or @eghisu)
I was able to locate a separate version of the Python headers under the Anaconda installation at /opt/anaconda3/pkgs/python-3.9.12-h12debd9_0/include/python3.9/Python.h
.
I also removed any existing versions of the Graphkernels repository on my system to ensure that there is not an existing installation elsewhere on the system being identified. And I recloned it into my home directory.
Then I take the following steps:
python setup.py build
. This produces the following output, which doesn't seem to have any obvious errors: gkcpy_build.txtpython setup.py install --user
and get the following output: gkcpy_install.txtimport pathlib, GKextCPy; pathlib.Path(GKextCPy.__file__).resolve().parent
from a python interpreter. It produces the following output:>>> import pathlib, GKextCPy; pathlib.Path(GKextCPy.__file__).resolve().parent
Traceback (most recent call last):
File "/home/exouser/.local/lib/python3.9/site-packages/GKextCPy-0.4.1-py3.9-linux-x86_64.egg/GKextCPy.py", line 14, in swig_import_helper
return importlib.import_module(mname)
File "/opt/anaconda3/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 666, in _load_unlocked
File "<frozen importlib._bootstrap>", line 565, in module_from_spec
File "<frozen importlib._bootstrap_external>", line 1173, in create_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
ImportError: /home/exouser/.local/lib/python3.9/site-packages/GKextCPy-0.4.1-py3.9-linux-x86_64.egg/_GKextCPy.cpython-39-x86_64-linux-gnu.so: undefined symbol: _ZSt28__throw_bad_array_new_lengthv
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/exouser/.local/lib/python3.9/site-packages/GKextCPy-0.4.1-py3.9-linux-x86_64.egg/GKextCPy.py", line 17, in <module>
_GKextCPy = swig_import_helper()
File "/home/exouser/.local/lib/python3.9/site-packages/GKextCPy-0.4.1-py3.9-linux-x86_64.egg/GKextCPy.py", line 16, in swig_import_helper
return importlib.import_module('_GKextCPy')
File "/opt/anaconda3/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ImportError: /home/exouser/.local/lib/python3.9/site-packages/GKextCPy-0.4.1-py3.9-linux-x86_64.egg/_GKextCPy.cpython-39-x86_64-linux-gnu.so: undefined symbol: _ZSt28__throw_bad_array_new_lengthv
OK, now there appears to be an issue with the linker dependencies. Is this the same environment? Might be an issue between the system C++ compiler and the Anaconda one. Can you check https://github.com/stan-dev/pystan/issues/294 for a potential workaround?
At this point, it seems that we are fighting a losing battle—maybe consider trying the GraKeL library?
Hello!
I want to install graphkernels, and am receiving the output at the following file when using the command:
pip --no-cache-dir install graphkernels
: graphkernels_install_err.txtThe main error from this seems to be "Python.h: No such file or directory". My understanding is that this usually means that pythonX.Y-dev must be installed according to the python version used.
I have confirmed that I do have python3.9-dev installed corresponding to my version of python using the command
sudo apt-get install python3.9-dev
, which gave the output "python3.9-dev is already the newest version (3.9.13-1+jammy1)." And the Python.h file exists on my system at/usr/include/python3.9/Python.h
.Is there something else I should do to install the package?