Creating custom dataset from images

[Comp-Engr18]

Hi, I want to know how can I use a custom large dataset of images in pytorch geometric. Can someone share me complete code of converting custom dataset of images into graph neural network? Any implementation please share. I am unable to find a solution in the entire web.

[rusty1s]

Can you be a bit more specific? How should graphs be created from those images?

In general, you can do this via transforms, e.g., you take in an image dataset from torchvision and apply transforms that take in an image and convert it to a graph representation suitable for PyTorch Geometric. In particular, the ToSLIC transform and the KNNGraph transform are very useful to do this.

[Comp-Engr18]

Thanks for your reply. I don't want a torchvision dataset. My dataset is provided by my university lab. I load it in python using cv2. I want to make a graph from images. How can I accomplish it? Do I need to make 1 graph per image? Nodes =number of pixels? What will be the nodes, edges and labels or features? I then want to use pytorch geometric to apply graph neural network on that graph.

[rusty1s]

I see. This depends on what you want to do. If you simply want to process images using a neural network, it's best to stay in the CNN domain. However, there exists some works that try to learn from images in a graph-based fashion, e.g., by converting an image to its superpixel representation. You can do that in PyTorch Geometric via the ToSLIC transform that takes in a torch.Tensor representing an image and converts it to its superpixel representation.

If you want to create a graph that represents relations between images, I suggest to first convert your images to a feature vector representation (e.g., via a pre-trained ResNet), and then connect each node/image using a k-nearest neighbor search.

[Comp-Engr18]

I want to classify the images which are basically obtained from electroencephalogram data. Can you share me a code of converting image into graph. Do you mean superpixel representation is a graph?

[rusty1s]

I think there might be a misunderstanding. This package is about learning on graphs (containing nodes/objects and their connections), see here. It's not about learning over function plots.

[Comp-Engr18]

I don't want to convert electroencephalogram data to image via pytorch geometric. I'll do this in MATLAB. I want to use pytorch geometric to implement graph neural network by using images taken from electroencephalogram data. Your answer to earlier question was to use TOSLIC for it. I review TOSLIC function but want to know what is meant by super pixel representation. Do TOSLIC converts image into graph which has nodes, edges, labels which I can use to train graph neural network in pytorch geometric?

[Comp-Engr18]

I have read from link that TOSLIC works with any torch vision dataset. While my images are custom from lab and not from torch vision dataset. Can I still use TOSLIC function?

https://pytorch-geometric.readthedocs.io/en/latest/modules/transforms.html#torch_geometric.transforms.ToSLIC

[rusty1s]

The ToSLIC transform works on any image that is represented as a PyTorch tensor, e.g.:

transform = ToSLIC(...)
img = ... # torch.Tensor of shape [C, H, W]
data = transform(img)

[Comp-Engr18]

When I get data from above code, I don't need to use the following to make graph?

data = Data(x=x, edge_index=edge_index, y=y)

You mean superpixel representation is now a graph in data object without mentioning X, y and edge_index?

Moreover, please tell what is C, H,W shape for torch.tensor?

[rusty1s]

Yes, the Data object will be returned by calling ToSLIC, and it will return node colors in x, node positions in pos, which you can further convert into a graph using KNNGraph or RadiusGraph.

The C, H, W shape correspond to image channels, image height, and image width.

[Comp-Engr18]

TOSLIC will produce data object for a single image. How should I get dataset object that contains entire output results from TOSLIC transform for all images in dataset. E.g., dataset= TUDataset returns entire dataset in dataset object. As my data is custom consisting of images, how can I concatinate output of data for each image in dataset object? In this case I think dataset object will be a tensor. Can you write sample code here for my understanding? Moreover what is image channels in C,H, W format?

[rusty1s]

A dataset will hold a set of Data objects. In your case, each image will be represented as a single Data/graph object, and a list of those Data objects will describe your dataset. This list can then be inputted into the torch_geometric.data.DataLoader for creating mini-batches:

images = [...]  # A list holding images of shape [C, H, W]
transform = ToSLIC(...)
data_list = [transform(image) for image in images]
loader = DataLoader(data_list, batch_size=...)

The number of channels denote the number of color channels, i.e., 3 for RGB images.

[Comp-Engr18]

Can you write the code of how to make images list e.g., for 2 images of 2 rows and 2 columns each for simplicity here. What has to be written in the parameters of TOSLIC function?

[rusty1s]

The ToSLIC transform expects any arguments passed to skimage.segmentation.slic, e.g.:

import torch
from torch_geometric.transforms import ToSLIC

image1 = torch.randn(3, 2, 2)  # Create a dummy image of size 2x2 with 3 channels
image2 = torch.randn(3, 2, 2)  # Create a dummy image of size 2x2 with 3 channels
images = [image1, image2]

transform = ToSLIC(n_segments=4)

data_list = [transform(image) for image in images]