narges-tk
commented
7 months ago By modifying the code (please find it below), I managed to create a some data. However, when I use them for classification, e.g., PointCNN, I get an error. I think the data folder is missed during data creation (please see figure below).
code: import os.path as osp import os import glob import torch
from torch_geometric.io import fs, read_off
from typing import Callable, List, Optional from torch_geometric.data import Data,InMemoryDataset,Dataset
class MyOwnDataset(Dataset): def init(self, root,train=True, transform=None, pre_transform=None, pre_filter=None): super().init(root, transform, pre_transform, pre_filter) path = self.processed_paths[0] if train else self.processed_paths[1] print('path:',path)#NNN
self.load(path)
#self.raw_dir=
@property
def raw_file_names(self)-> List[str]:
return ['alder','aspen','lime']
@property
def processed_file_names(self)-> List[str]:
return ['training.pt', 'test.pt']
def download(self):
# Download to `self.raw_dir`.
#path = download_url(url, self.raw_dir)
pass
# os.unlink(path)
#folder = osp.join(self.root, f'titan_FGI')
#fs.rm(self.raw_dir)
# os.rename(folder, self.raw_dir)
def process(self):
idx = 0
for raw_path in self.raw_paths:
# Read data from `raw_path`.
# print('raw_path:',raw_path)#NNN
#titan_FGI
data = Data(...)
if self.pre_filter is not None and not self.pre_filter(data):
continue
if self.pre_transform is not None:
data = self.pre_transform(data)
torch.save(data, osp.join(self.processed_dir, f'data_{idx}.pt'))
idx += 1
#----------------------------------
#torch.save(self.process_set('train'), self.processed_paths[0])
#torch.save(self.process_set('test'), self.processed_paths[1])
def process_set(self, dataset: str) -> List[Data]:
categories = glob.glob(osp.join(self.raw_dir, '*', ''))
categories = sorted([x.split(os.sep)[-2] for x in categories])
data_list = []
for target, category in enumerate(categories):
folder = osp.join(self.raw_dir, category, dataset)
paths = glob.glob(f'{folder}/{category}_*.txt')
for path in paths:
data = read_off(path)
data.y = torch.tensor([target])
data_list.append(data)
if self.pre_filter is not None:
data_list = [d for d in data_list if self.pre_filter(d)]
if self.pre_transform is not None:
data_list = [self.pre_transform(d) for d in data_list]
return data_list
def len(self):
return len(self.processed_file_names)
def get(self, idx):
data = torch.load(osp.join(self.processed_dir, f'data_{idx}.pt'))
return datadata_root=r'data/titan_FGI/'
train_dataset=MyOwnDataset(data_root,train=True)#raw_dir
test_dataset=MyOwnDataset(data_root,train=False)#raw_dir
print('train_dataset:',train_dataset)
print('test_dataset:',test_dataset)
rusty1s
🐛 Describe the bug
To do point cloud classification for custom datasets, I followed
. However, I am getting the following error.
I wonder if you could provide me with a more clear instruction of how to use the point classification codes for custom datasets.
Below is my code:
import os.path as osp
import torch from torch_geometric.data import Dataset, download_url from torch_geometric.io import fs, read_off
class MyOwnDataset(Dataset): def init(self, root,train=True, transform=None, pre_transform=None, pre_filter=None): super().init(root, transform, pre_transform, pre_filter) path = self.processed_paths[0] if train else self.processed_paths[1] print('path:',path)#NNN self.load(path)
self.raw_dir=
data_root=r'/media/emre/Data/Downloads/pytorch_geometric-master/benchmark/points/data'
train_dataset=MyOwnDataset(data_root,train=True)#raw_dir
test_dataset=MyOwnDataset(data_root,train=False)#raw_dir print('train_dataset:',train_dataset) print('test_dataset:',test_dataset)
Versions
ints$ curl -OL https://raw.githubusercontent.com/pytorch/pytorch/main/torch/utils/collect_env.py % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 22068 100 22068 0 0 89707 0 --:--:-- --:--:-- --:--:-- 89707
ints$ python3 collect_env.py Collecting environment information... PyTorch version: 2.1.1 Is debug build: False CUDA used to build PyTorch: 11.8 ROCM used to build PyTorch: N/A
OS: Ubuntu 20.04.6 LTS (x86_64) GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0 Clang version: Could not collect CMake version: version 3.16.3 Libc version: glibc-2.31
Python version: 3.11.5 (main, Sep 11 2023, 13:54:46) [GCC 11.2.0] (64-bit runtime) Python platform: Linux-5.15.0-84-generic-x86_64-with-glibc2.31 Is CUDA available: True CUDA runtime version: 10.1.243 CUDA_MODULE_LOADING set to: LAZY GPU models and configuration: GPU 0: NVIDIA GeForce RTX 2080 Ti Nvidia driver version: 545.29.06 cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5 HIP runtime version: N/A MIOpen runtime version: N/A Is XNNPACK available: True
CPU: Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian Address sizes: 43 bits physical, 48 bits virtual CPU(s): 32 On-line CPU(s) list: 0-31 Thread(s) per core: 2 Core(s) per socket: 16 Socket(s): 1 NUMA node(s): 2 Vendor ID: AuthenticAMD CPU family: 23 Model: 8 Model name: AMD Ryzen Threadripper 2950X 16-Core Processor Stepping: 2 Frequency boost: enabled CPU MHz: 2200.000 CPU max MHz: 3500,0000 CPU min MHz: 2200,0000 BogoMIPS: 6999.13 Virtualization: AMD-V L1d cache: 512 KiB L1i cache: 1 MiB L2 cache: 8 MiB L3 cache: 32 MiB NUMA node0 CPU(s): 0-7,16-23 NUMA node1 CPU(s): 8-15,24-31 Vulnerability Gather data sampling: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Retbleed: Mitigation; untrained return thunk; SMT vulnerable Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; Retpolines, IBPB conditional, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Not affected Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid amd_dcm aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb hw_pstate ssbd ibpb vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 clzero irperf xsaveerptr arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif overflow_recov succor smca sme sev sev_es
Versions of relevant libraries: [pip3] numpy==1.26.2 [pip3] torch==2.1.1 [pip3] torch-cluster==1.6.3 [pip3] torch_geometric==2.4.0 [pip3] torch-scatter==2.1.2 [pip3] torchaudio==2.1.1 [pip3] torchvision==0.16.1 [pip3] triton==2.1.0 [conda] blas 1.0 mkl
[conda] ffmpeg 4.3 hf484d3e_0 pytorch [conda] libjpeg-turbo 2.0.0 h9bf148f_0 pytorch [conda] mkl 2023.1.0 h213fc3f_46344
[conda] mkl-service 2.4.0 py311h5eee18b_1
[conda] mkl_fft 1.3.8 py311h5eee18b_0
[conda] mkl_random 1.2.4 py311hdb19cb5_0
[conda] numpy 1.26.2 py311h08b1b3b_0
[conda] numpy-base 1.26.2 py311hf175353_0
[conda] pytorch 2.1.1 py3.11_cuda11.8_cudnn8.7.0_0 pytorch [conda] pytorch-cluster 1.6.3 py311_torch_2.1.0_cu118 pyg [conda] pytorch-cuda 11.8 h7e8668a_5 pytorch [conda] pytorch-mutex 1.0 cuda pytorch [conda] pytorch-scatter 2.1.2 py311_torch_2.1.0_cu118 pyg [conda] torch-geometric 2.4.0 pypi_0 pypi [conda] torchaudio 2.1.1 py311_cu118 pytorch [conda] torchtriton 2.1.0 py311 pytorch [conda] torchvision 0.16.1 py311_cu118 pytorch