rusty1s
commented
2 years ago Can you ensure that you are not loading any data that was generated using PyG 2.0? This may require that you remove the processed/ directory in your dataset folder.
Closed waqar12868 closed 2 years ago
rusty1s
commented
2 years ago Can you ensure that you are not loading any data that was generated using PyG 2.0? This may require that you remove the processed/ directory in your dataset folder.
waqar12868
commented
2 years ago Thank you for your reply.
I update the PyG 1.7.2 with PyG 2.0.0 and then run the code; this error occurs:
RuntimeError: The 'data' object was created by an older version of PyG. If this error occurred while loading an already existing dataset, remove the 'processed/' directory in the dataset's root folder and try again.
I deleted the processed directory from my dataset folder, but again this error occurred.
When I tried to run the code with the older version of PyG 1.7.2, then this type of error occurred: ModuleNotFoundError: No module named 'torch_geometric.data.storage'
That is the two-year old code. https://github.com/flandolfi/kplex-pool
Please guide me; where is the problem?
rusty1s
commented
2 years ago This is a bit weird. Are you sure you have removed any data inside processed/? What's the recommended PyG version of the code base?
waqar12868
commented
2 years ago Yes, I removed all data files inside the processed/ folder. The PyG version is not mentioned in the code requirements.
Here is the code requirements:
conda create -n kplex-pool python=3.7 -y conda activate kplex-pool conda install -y -c nvidia -c rapidsai -c numba -c conda-forge -c defaults \ dask-cudf dask-cuda cugraph=0.15 cudatoolkit=10.1 c-compiler cxx-compiler
--find-links https://download.pytorch.org/whl/torch_stable.html --find-links https://pytorch-geometric.com/whl/torch-1.7.0+cu101.html torch==1.7.0+cu101 torchvision==0.8.1+cu101 torch-scatter torch-sparse torch-cluster torch-spline-conv torch-geometric scikit-learn skorch tqdm
I installed every library package using the above commands. And I am using cugrph 21.08 because the old version is no more compatible. You can see from the above commands, the version of PyG is not mentioned. This is two-year old code.
rusty1s
commented
2 years ago Can you post the full traceback? Where exactly does the code crash in https://github.com/flandolfi/kplex-pool?
waqar12868
commented
2 years ago Here is the full traceback:
kplex-pool1) waqar.ali@gpu1:~/kplex-pool-master$ cat slurm-1629.out
Outer CV: 0%| | 0/10 [00:00<?, ?it/s]
Grid Search: 0%| | 0/128 [00:00<?, ?it/s, graph_sage=1, hidden=64, k=1, num_layers=2, lr=0.001]
Outer CV: 0%| | 0/10 [00:00<?, ?it/s]
Traceback (most recent call last):
File "cv.py", line 269, in
Traceback (most recent call last):
File "cv.py", line 269, in
Traceback (most recent call last):
File "cv.py", line 269, in
Traceback (most recent call last):
File "cv.py", line 269, in
Traceback (most recent call last):
File "cv.py", line 269, in
Traceback (most recent call last):
File "cv.py", line 269, in
Traceback (most recent call last):
File "cv.py", line 269, in
Traceback (most recent call last):
File "cv.py", line 269, in
rusty1s
commented
2 years ago Can you do me a favor and test out what happens if you move thesuper(Cover, self).__init__() call at the top of the __init__ function, see here? Also replace the line below
def __inc__(self, key, value):with
def __inc__(self, key, value, *args, **kwargs):I did the same as you told me above. Here is the error:
Traceback (most recent call last):
File "cv.py", line 15, in
rusty1s
commented
2 years ago Really sorry for the late reply. Not really sure what is causing this error, but what happens if you replace all occurrences of super(Cover, self) with just super()?
waqar12868
commented
2 years ago No problem. I did it, but the error is the same: I think I have to replicate the complete code with the new versions of libraries such as PyTorch and PyTorch geometric.
"The 'data' object was created by an older version of PyG. " RuntimeError: The 'data' object was created by an older version of PyG. If this error occurred while loading an already existing dataset, remove the 'processed/' directory in the dataset's root folder and try again.
rusty1s
commented
2 years ago Can you try to send me the data over that you are trying to load? I can take a look and try to convert it :)
waqar12868
commented
2 years ago Thanks for your help. In this kplex project, the authors downloaded the data from the given below link:
https://github.com/FilippoMB/Benchmark_dataset_for_graph_classification/tree/master/datasets
After performing some algorithms, they save it and then load it again for training the graph neural network.
rusty1s
commented
2 years ago This fails for me with:
Traceback (most recent call last):
File "/Users/rusty1s/miniconda3/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/Users/rusty1s/miniconda3/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/Users/rusty1s/github/forks/kplex-pool/benchmark/cv.py", line 273, in <module>
net = NeuralNetClassifier(
File "/Users/rusty1s/miniconda3/lib/python3.8/site-packages/skorch/classifier.py", line 141, in fit
return super(NeuralNetClassifier, self).fit(X, y, **fit_params)
File "/Users/rusty1s/miniconda3/lib/python3.8/site-packages/skorch/net.py", line 1213, in fit
self.initialize()
File "/Users/rusty1s/miniconda3/lib/python3.8/site-packages/skorch/net.py", line 816, in initialize
self._initialize_module()
File "/Users/rusty1s/miniconda3/lib/python3.8/site-packages/skorch/net.py", line 715, in _initialize_module
self.initialize_module()
File "/Users/rusty1s/miniconda3/lib/python3.8/site-packages/skorch/net.py", line 567, in initialize_module
module = self.initialized_instance(self.module, kwargs)
File "/Users/rusty1s/miniconda3/lib/python3.8/site-packages/skorch/net.py", line 544, in initialized_instance
return instance_or_cls(**kwargs)
TypeError: __init__() missing 1 required positional argument: 'cover_fun'But it looks like the data is loaded in correctly, independent of the PyG version.
waqar12868
commented
2 years ago Thank you for your reply. If data is loaded correctly, what could be the issue with PyG? Then why do I get this error; data object created by the old version of PyG? Can you share the above code?
rusty1s
commented
2 years ago What is the command you run that receives this error? I still assume that the issue is that you are loading a pre-processed dataset, otherwise I cannot explain myself this issue.
waqar12868
commented
2 years ago Yes, I am loading a pre-processed dataset.
In the first step, the maximal clique algorithm was applied to the graph to get one node (step by step coarsened the graph).
In the second step, save the hierarchy of the graph and then use it during training the graph neural network.
I am using this commond:
python -m benchmark.cv \ --dense \ --model CoverPool \ --dataset ENZYMES \ --k_step_factor 0.5 \ --node_pool_op add max
rusty1s
commented
2 years ago Why don't you remove the pre-processed graph first? The following modification works for me:
class Cover(Data):
def __init__(self, cover_index=None, num_clusters=None, **kwargs):
super().__init__(**kwargs)
self.cover_index = cover_index
if num_clusters is not None:
self.__num_clusters__ = num_clustersThank you for your guidance. Now there is no error, and the code is successfully run. All credit goes to you @rusty1s. I think so. I removed the pre-processed graph. I am new in this domain (graph neural network). Can you explain why your above-modified code is running correctly? Again thanks for helping me.
rusty1s
commented
2 years ago PyG 2.0 introduced some new concepts of data handling, and therefore the super().__init__() call needs to be called before any other data attribute assignment :)
waqar12868
commented
2 years ago Thank you. I have a question. I am running the kplex-pool code with the same architecture of graph neural network and the same parameters. Still, I got different results compared to the original paper( for instance, the original paper's accuracy was 75.6%, and I got 60.4 % on the same dataset with the same code). What can be the reasons?
The only difference between the PyTorch-geometric library and the cugraph library. I am using the latest version of PyTorch-geometric 2.0.3, and the authors used the old one, 1.6.0 I am using the latest version of cugraph library 21.01, and the authors used the old one, 15.0
Can the different libraries' versions affect the accuracy of the model?
rusty1s
commented
2 years ago In general, reproducibility is not guaranteed across PyTorch and PyG versions, e.g., there might be differences in weight initializations. In general, we try to keep these things pretty stable though. Note that there may be also differences in dataset splitting, etc which might explain these differences. I sadly don't have a good explanation for you, except for reaching out to the authors :(
waqar12868
commented
2 years ago I got your point. You can close this issue. Thank you for your guidance.
😵 Describe the installation problem
I install the old version of pytroch-geometric 1.7.2 with torch==1.7.0+cu101 and torchvision==0.8.1+cu101. When I run my code, then this type of error occurs: ModuleNotFoundError: No module named 'torch_geometric.data.storage' I don't know what the problem with pytroch-geometric 1.7.2 is? Please guide me if anybody faced this type of error.
Environment
--find-links https://download.pytorch.org/whl/torch_stable.html --find-links https://pytorch-geometric.com/whl/torch-1.7.0+cu101.html torch==1.7.0+cu101 torchvision==0.8.1+cu101 torch-scatter torch-sparse torch-cluster torch-spline-conv