Hybrid Algorithm to Explore Properties of GPT in Quantum Transformer Models

hykavitha commented 1 year ago

Description

This project is intended to explore couple papers in literature of Quantum Transformer models [self attention model: https://arxiv.org/abs/2205.05625 , Quantum vision transformers : https://arxiv.org/abs/2209.08167, GPT models: https://arxiv.org/pdf/2303.12712.pdf] . There are a couple limitations to produce a quantum GPT model, one being that transformers generally require a lot of data to be more accurate then convolutional or recurrent architectures, and another being the size of the model, inputs, and outputs, Input is the key property of transformers - each block updates the existing input vector in the sequence, transforming it slightly, and this updated vector is passed to the next block.
In this project we want to explore and build a quantum circuit as a transformer block - if we have some inputs encoded in some way as input, and some other set of inputs corresponding to the last measurement outputs of the model - using the quantum circuit for part of the transformer blocks - i.e., as a sub component of a classical transformer neural net.

This project should take approximately 3 months to reasonably explore the literature of the Quantum transformer model and generate Qiskit Code. To Build Quantum Machine Learning - technical skills, and work on meaningful projects we want to integrate into Qiskit or maintain; Deliverables

Interactive tutorial Jupyter Notebook
Journal / conference paper

Mentors details Mentor 1 Name: Kavitha Yogaraj, Brian Quanz
GitHub ID: @hykavitha, @bquanz What they do:

Kavitha: IBM Quantum Computational Scientist, IBM Qiskit Advocate-2020, IBM Quantum
Brian : IBM Senior Research Scientist, IBM Quantum

Number of mentees required : 3

Type of mentees Mentee 1

Required:

Deep Learning - RNN, CNN Expert, deep generative models - Classical

Knowledge in Quantum Transformer model

Experience in qiskit

Proficient in Linear Alzebra

Gradient descent

Convert Pennylane code to Qiskit.

Paper implementation knowledge

Mentee 2

Required:

Proficient in Qiskit & QNN Hybrid algorithms, Quantum Machine Learning

Deep Learning - RNN, CNN Expert, deep generative models - Classical

Knowledge in Quantum Transformer model

Experience in Qiskit open source contributions

Proficient in Linear Alzebra

Gradient descent

Convert Pennylane code to Qiskit.

Paper implementation knowledge

Mentee 3

Required:

Proficient in Linear Alzebra

Deep Learning - RNN, CNN Expert, deep generative models - Classical

Knowledge in Quantum Transformer model

Experience in Qiskit open source contributions

Proficient in Linear Alzebra

Gradient descent

Convert Pennylane code to Qiskit.

Paper implementation knowledge

Tarun2021 commented 1 year ago

I am interested to work in this project

divshacker commented 1 year ago

I am also interested in this project

poig commented 1 year ago

interested

ichen17 commented 1 year ago

Interesting.

harshdeeps036 commented 1 year ago

Interested.

AnuVadali commented 1 year ago

I am interested to work on this project.

This is very intriguing considering the fact that just some years ago, there were studies on how to simulate quantum circuits using transformer models (https://arxiv.org/abs/1912.11052). And now we are studying how we can make transformer models using quantum circuits. Very interesting :D

Genuine question : Won't we have an issue with the number of qubits?

hykavitha commented 1 year ago

I am interested to work on this project.

This is very intriguing considering the fact that just some years ago, there were studies on how to simulate quantum circuits using transformer models (https://arxiv.org/abs/1912.11052). And now we are studying how we can make transformer models using quantum circuits. Very interesting :D

Genuine question : Won't we have an issue with the number of qubits?

Thats a study we want to do here.

hykavitha commented 1 year ago

@GemmaDawson : Please assign these interested people as the mentees for this project.