Quantum Image Processing - Filters

[robertloredo]

Description

The idea of this project is build a quantum image processing filter that illustrates the potential which quantum computers have in the processing of large amounts of image data. We'll begin by exploring the various image filters that are most widely used in various industries such as medical imaging (life-sciences industry), or risk analysis (insurance industry) and familiarize ourselves with existing quantum image processing techniques. Afterwards we will identify areas in which we can work on implementing them using Qiskit and real image data. This project could be then extended towards a research paper or topic, or at a minimum be a nice blog post and candidate to the Qiskit textbook applications chapter.

Mentor/s

Robert Loredo (@robertloredo), IBM Quantum Ambassador worldwide lead, Qiskit Advocate, IBM Master Inventor

Type of participant

The ideal participant for this project would be someone who is interested in the area of quantum image processing and with familiarity of quantum computing. An understanding of the various techniques used to filter or segment objects within an image using classical computers is an advantage. The participant will also need to be able to run and deploy the algorithms using Aqua and be familiar with the latest changes to the Aqua module.

Number of participants

2-3

Deliverable

Technical note, Qiskit textbook chapter or a short research paper in an image processing or quantum journal.

[q-inho]

Since Quantum and classical image processing are my recent main interest, I always wanted to develop and dive into this topic and its application. This project topic looks really interesting and can't wait to join this project.

[Denny-Hwang]

I am interested in this topic. Although I am not familiar with quantum image processing, nowadays I'm working with classical Image processing (like Opencv..) as a preprocessing of my ML model. Also, I am ready to deep dive into this project!

[darshkaushik]

I have experience in classical convolutional filters (CNN classifiers mostly) and Gradient Class Activation Maps (Grad-CAM) for medical imaging, and VQAs like QAOA and QSVM. This project seems to be perfectly timed with my learning path, exploring quantum image filters and hopefully contributing towards better filters.

[q-inho]

It would be great if we can team up for an application and be able to discuss it as soon as possible.

[SaashaJoshi]

Hey! Even I'm interested in joining you in working on this project. I have experience dealing with medical image processing (denoising/filtering, cropping, segmentation, etc.) using CNNs and working on image metrics. Count me in if you guys discuss anything!

[q-inho]

Qiskit textbook Chapter 4.2.2 seems to be a good material to be familiar with quantum image processing. This project idea is to see if there are other areas that might illustrate quantum speed up in the area of quantum image processing. I am looking at other classical image processing techniques and determine whether there is some quantum speed up potential. I will share them with you once available.

[HuangJunye]

I found someone has published a paper on this topic on Qiskit Slack. Might be useful.

[HuangJunye]

Qiskit textbook Chapter 4.2.2 seems to be a good material to be familiar with quantum image processing. This project idea is to see if there are other areas that might illustrate quantum speed up in the area of quantum image processing. I am looking at other classical image processing techniques and determine whether there is some quantum speed up potential. I will share them with you once available.

If I am not mistaken, @robertloredo is a co-author of this chapter.

[darshkaushik]

I have been trying to read this paper which is also referred the qiskit textbook chapter.

Section 3 and before are clear to me, but as I move on to section 4 and 5, I am able to somewhat understand the preliminary quantum theories individually but cannot put them together in section 5. Can anyone help with this?

Also are we looking for the filters which are similar to the kind of kernels stored in QRAM in section 5? Or should we shift our focus to other kinds?

[robertloredo]

List of ideas:

1. Quantum Noise: Find ways to create/remove noise from images using Quantum techniques. i.e. blur/sharpen, etc.
2. Quantum augmentation of image data: Clustering an image how can we refine the image. Add additional pixel information (review the paper titled: Explainable and scalable machine-learning algorithms for detection of autism spectrum disorder using fmri data section titled: Simulating fMRI data for superior deep-learning training: Data augmentation using linear interpolation.
3. Quantum 3D images: Constructing 3D images from 2d image sets by integrating quantum computing techniques to the image processing process (pre or post): https://cvgl.stanford.edu/teaching/cs231a_winter1415/prev/projects/CS231a-FinalReport-sgmccann.pdf

Please include any other ideas as well, and then vote on your top three in the comment section.

[gines-carrascal]

Have you seen this scene in Blade Runner navigating into the image: zoom-enhance-zoom-enhance...

Some resources on (classical) augmentation of image data:

https://medium.com/@zhuocen93/an-overview-of-espcn-an-efficient-sub-pixel-convolutional-neural-network-b76d0a6c875e

https://github.com/Lornatang/ESPCN-PyTorch

It may be possible to use quantum layers in this neural network. Could we obtain better results?

An example of convolutional quantum neural networks: https://link.springer.com/content/pdf/10.1007/s11433-021-1734-3.pdf

[HuangJunye]

@AmauryDM can please comment on this issue so that I can assign you on the issue?

[AmauryDM]

Yes @HuangJunye sir thank you!

[Denny-Hwang]

I also have an idea 'Edge detection enhancement using quantum computing'. But, I'm still finding clues, I don't have a vivid guide to research now.

ref 1: https://www.osapublishing.org/oe/fulltext.cfm?uri=oe-28-24-35415&id=442483 ref 2: https://www.spiedigitallibrary.org/conference-proceedings-of-spie/7497/749724/A-new-quantu[…]on-algorithm-for-medical-images/10.1117/12.832499.short ref 3: https://ieeexplore.ieee.org/document/9262964 Anyone has related ideas, or anything, please comment!

• (there was already a quantum edge detection example, in the qiskit textbook) https://qiskit.org/textbook/ch-applications/quantum-edge-detection.html (edited)

[AmauryDM]

#1 Quantum Image Processing - Filters.pdf

[AmauryDM]

Checkpoint 2

In this project, our goal was to study how we could enhance images using quantum operations. After a brief survey of papers on the subject of quantum image processing and practical applications, we realized there were not a lot of research in this area. Thus, we had to go back at the start and study everything that was related to image processing both in classical and quantum computing. In this manner, we studied how convolutional filters work on images and how some were translated in quantum such as the QSobel for edge detection. Moreover, the area of quantum image encoding was very important. Indeed, after developing hybrid quantum-classical neural network models to approach the ESPCN, the classical model for image enhancement, we realized it was not adapted to this task: the quantum part is used as an activation function and works best on classification where the output can be mapped to 0 or 1 but the image enhancement is about intensities of the pixels to add information to the image. Thus, we studied how to implement a full quantum image processing circuit and this is how the image encoding part was important. Indeed, we based on the work that was done such as FRQI and amplitude encoding that was really helpful to work with encoded images in qubits so that we could perform operations and retrieve interesting characteristic on the image. After encoding the image, the idea was to find some interesting operations that could lead us to our goal which is scaling up a given image. To do so, we experimented different known circuit such as the Quantum Fourier Transform which is an important area of image analysis in the classical counterpart. That is how we obtained the following output image which was interesting to study as it resemble the classical version. However, this analysis is not sufficient enough for our project because the image is obtained with the state vector of the encoded image and not by measuring the qubits. This study only helped us to perform operations on the image in the Fourier basis in order to see if we could obtain similar results than in classical image processing. Moreover, it is not close enough for us in our search for quantum enhancement of images, but it is an interesting starting point when comparing to classical computing. Based on all these different leads for our work, the next step is to implement theoretical results of image scaling in classical and quantum computing, study transformations that could help us analyse an image by only using quantum operations and find interesting results on the use of QFT in different kinds of images.

[AmauryDM]

#1 Quantum Image Processing - Filters - Final Showcase.pdf