setup.py file doesn't work properly

[AslantheAslan]

Thank you very much for sharing your valuable code here. However, I couldn't compile it due to the "sparse_tools" library dependency. Once I try to run "python setup.py develop", it sets the sparse_tools==0.0.0 which is non-sense. I suppose it happens due to a deletion of the sparse_tools library on the internet. Therefore, I believe the setup.py file should be rewritten.

To be more concise, the error I get is the following:

AttributeError: module 'sparse_tools' has no attribute 'spspmm_diag_sym_ABA'

I think it happens due to the problem I mentioned in the first paragraph.

Thanks for your time and effort.

[Yangxc13]

Thanks for your attention. Sorry I cannot locate the bug, could you please check the following things to provide more help?

After running "python setup.py develop", a package named "sparse_tools" would be installed into your python environment. In my expectation, an error may occur if you already have another python package with the same name. Please run python -c "import sparse_tools; print(sparse_tools.__path__)" to see where this package points to.

If "sparsetools.__path_\" points to the folder ./sparse_tools/sparse_tools and there is no "*.so" files under the folder ./sparse_tools/sparse_tools or ./sparse_tools/build, then it may be a compilation error, please run the following commands under the folder ./sparse_tools and provide me the logs.

pip uninstall sparse_tools
python setup.py develop

These commands will uninstall the old "sparse_tools" package and try to re-install it.

---

By the way, I actually did not provide the compiled library but only the source C++ files. In the expectation, when running "python setup.py develop", the source files would be compiled to generate the library.

[AslantheAslan]

Thank you very much for pointing out how I should uninstall and reinstall the sparse_tools library. However, I got different bugs now, regarding the compilation of the C++ file. The entire output on the terminal is too long but here are some useful lines about my problem:

LINK : warning LNK4044: unrecognized option '/s'; ignored LINK : error LNK2001: unresolved external symbol PyInit__spspmm_diag_unsym_cpu build\temp.win-amd64-3.9\Release\csrc\cpu_spspmm_diag_unsym_cpu.lib : fatal error LNK1120: 1 unresolved externals error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\link.exe' failed with exit code 1120

Also,python -c "import sparse_tools; print(sparse_tools.__path__)" points to the folder ./sparse_tools, not ./sparse_tools/sparse_tools. Likewise, I don't have any "*.so" files under those folders you mentioned.

Since my knowledge about developing libraries with C++ is not quite sufficient, I couldn't resolve the issue I had.

I would be more than happy to hear from you regarding this issue. Thanks for your time and effort, again.

[Yangxc13]

Please use files of this to overwrite files under ./sparse_tools/csrc. Then uninstall the old package and re-install it.

If it still does not work, please provide me the logs.

[AslantheAslan]

Once again, thank you very much for the detailed information. I have applied the revisions you offered and now it looks like the library was compiled successfully. However, I am facing another issue in the *\large\main.py file after compiling the sparse_tools. It looks like an execution error and now I am trying to solve it. The logs are as follows:

Traceback (most recent call last):
  File "C:\Users\User\PycharmProjects\sehgnn\large\main.py", line 13, in <module>
    from utils import *
  File "C:\Users\User\PycharmProjects\sehgnn\large\utils.py", line 28, in <module>
    import sparse_tools
  File "c:\users\user\pycharmprojects\sehgnn\sparse_tools\sparse_tools\__init__.py", line 10, in <module>
    torch.ops.load_library(importlib.machinery.PathFinder().find_spec(
  File "C:\Users\User\PycharmProjects\competition\venv\lib\site-packages\torch\_ops.py", line 255, in load_library
    ctypes.CDLL(path)
  File "C:\Users\User\AppData\Local\Programs\Python\Python39\lib\ctypes\__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: [WinError 193] %1 is not a valid Win32 application

[Yangxc13]

It seems like a problem about the python environment now. Maybe you can try Anaconda for python environment setup.

I have tested the codes on my friend's windows PC, and it works. He installed python through Anaconda on windows.

By the way, I think Anaconda could also help if you want to explore more machine learning or deep network algorithms in the future.

[AslantheAslan]

I am fully agreed. Actually, using pip rather than conda does not make any sense but I got used to it since I did it quite often previously. I am sticking with conda now and will let you know if I can make it work. Please don't close the issue until I run it so that I can inform the others who may face with the issue that I had.

Thank you very much for your support. Your code and paper are just amazing!

[AslantheAslan]

It seems like a problem about the python environment now. Maybe you can try Anaconda for python environment setup.

I have tested the codes on my friend's windows PC, and it works. He installed python through Anaconda on windows.

By the way, I think Anaconda could also help if you want to explore more machine learning or deep network algorithms in the future.

I created a conda environment with all dependencies satisfied but I still have the same issue as follows:

Traceback (most recent call last):
  File "C:\Users\User\PycharmProjects\sehgnn\large\main.py", line 13, in <module>
    from utils import *
  File "C:\Users\User\PycharmProjects\sehgnn\large\utils.py", line 28, in <module>
    import sparse_tools
  File "c:\users\user\pycharmprojects\sehgnn\sparse_tools\sparse_tools\__init__.py", line 10, in <module>
    torch.ops.load_library(importlib.machinery.PathFinder().find_spec(
  File "C:\ProgramData\Anaconda3\envs\sehgnn\lib\site-packages\torch\_ops.py", line 255, in load_library
    ctypes.CDLL(path)
  File "C:\ProgramData\Anaconda3\envs\sehgnn\lib\ctypes\__init__.py", line 382, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: [WinError 193] %1 is not a valid Win32 application

[Yangxc13]

Maybe try to remove the previous compiled libraries? That is, remove ./sparse_tools/build and all ".pyd" files under ./sparse_tools/sparse_tools. Or more specifically, only keep ".py", ".cpp", ".h" files in sparse_tools. Then uninstall sparse_tools package and re-install it.

By the way, if you use pycharm as IDE, please make sure you have changed the python interpreter of pycharm to conda.

[AslantheAslan]

That actually solved my problem! I deleted ".pyd" files and then reinstalled sparse_tools into my conda environment and now it works. I am having another error related to "Torch not compiled with CUDA enabled" but I think I can solve it on my own. Again, thank you very much for your very helpful guidance. Though, I wonder if your can be compiled by utilizing CPU only. I tried to change the large/main.py row 132 as device = 'cuda' if torch.cuda.is_available() else 'cpu' but I came up with the error:

line 294, in main
label_emb = label_emb[init2sort]
UnboundLocalError: local variable 'label_emb' referenced before assignment

[Yangxc13]

Actually I did not expect to run only on cpu. But it is ok with a few modifications. Please update the codes to the latest version, and take care of the following things during running:

• Add the option '--cpu' in commands
• Avoid using the option '--amp', it only support GPU with tensor cores for acceleration (gpu with the turing or ampere architecture, e.g. T4/V100/A100/3060 or other latest gpu cards).
• For ComplEx embedding generation, change the option '--gpu 1' in the file ./data/complex_nars/train_graph_emb.sh to '--gpu -1' (about 2 times slower than a T4 GPU for embedding generation)

I still do not recommend to run on cpu only, as it is about 15 times slower than running on a single T4 GPU.

By the way, the error about 'label_emb' occurs when not using the '--label-feats' option in commands, and it has been fixed. But not using this option damages model effects. I recommend to follow the instructions and command examples in Readme.

[AslantheAslan]

I can not describe my gratitude for your efforts. Thank you very much, now I can compile it by utilizing my CPU. Afterwards, I will make it run on GPUs of my server and get high-speed computation as you mentioned. I will try to experience your SOTA model's effectiveness on other heterogeneous datasets as well.

Wish you a great day and lots of success on your studies.

Best regards.

[Yangxc13]

Thanks for your help in promoting SeHGNN as well, and good luck with your studies.

With no more problems, I will close this issue soon. Feel free to reopen it or contact me through emails if you meet difficulties later.