im2im: Automatically Converting In-Memory Image Representations using A Knowledge Graph Approach

The im2im package provides an automated approach for converting in-memory image representations across a variety of image processing libraries, including scikit-image, opencv-python, scipy, PIL, Matplotlib.plt.imshow, PyTorch, Kornia and Tensorflow. It handles the nuances inherent to each library's image representation, such as data formats (numpy arrays, PIL images, torch tensors, and so on), color channel (RGB or grayscale), channel order (channel first or last or none), device (CPU/GPU), and pixel intensity ranges.

Im2im was developed for the use in Visual Programming Language for image processing (VPL4IPs) to completely removes the conversions steps required to manually manage image transformations, drastically improving accessibility, specifically for non-expert users. It also addresses the tension between low-level implementation details necessary for compatibility and the high-level image processing operations that VPL4IPs aim to provide. However, the library is a conventional Python package, and as such, it can also be used by directly invoking its functions from any Python image processing program to automate image conversion steps. Considering the relatively low memory footprint and computational overhead of the system, using the library to simplify conventional Python programming is an interesting option as well.

Installation

Install the package via pip:

pip install im2im

Usage

import numpy as np
from im2im import Image, im2im

# Assume an image that is a numpy.ndarray with shape (20, 20, 3) in uint8 format
to_be_converted = np.random.randint(0, 256, (20, 20, 3), dtype=np.uint8)

# Convert the image using the input preset "numpy.rgb_uint8" for the source image and "torch.gpu" for the target image.
converted: Image = im2im(Image(to_be_converted, "numpy.rgb_uint8"), "torch.gpu")
# Access the converted image data via 'converted.raw_image', which is now a torch tensor with shape (1, 3, 20, 20),
# in float32 format, normalized to the range [0, 1], and transferred to the GPU.

For other APIs like im2im_code, please refer to public APIs.

For integration to visual programming language, please refer to Comparative Analysis.

Evaluation

Comparative Analysis The effectiveness and benefits for VPL4IPs are validated through Comparative Analysis. For detailed comparisons, please refer to the following studies: , and third study.

Accuracy All conversion code snippets are thoroughly verified through execution checks to ensure their correctness.

Performance Profiling Performance is analyzed using the cProfile module, with detailed results available in the profiling notebooks.

Contribution

We welcome all contributions to this project! If you have suggestions, feature requests, or want to contribute in any other way, please feel free to open an issue or submit a pull request. For detailed instructions on developing, building, and publishing this package, please refer to the README_DEV.

Cite

Todo

License

This project is licensed under the MIT License. See the LICENSE file for details.