Async calculation on remote devices

[fretchen]

Feature details

Pennylane can send jobs to remote devices. However, whenever such a job is sent out pennylane typically seems to wait for the answer within the code before the user can continue with his life. As example see here. At least this is what I got from a quick search and question in the forum

While this is of great help for the design of the language, this assumption can create numerous serious problems in real life:

• If the calculation takes hours or days it is really annoying to have the system blocked for all the time, even if the calculation runs on a remote machine.
• Sometimes the remote machines just forget about you, jobs are not properly treated and god knows. Right now it is hard for the user to investigate any kind of problems in this regard.
• There might be an enormous queue to wait in...

Implementation

A very helpful feature would be to allow users to have some kind of "non-blocking" interaction, if they wish and if it makes sense. Then the user submits the calculation lives his life and comes back to see the result when it might be time.

How important would you say this feature is?

1: Not important. Would be nice to have.

Additional information

qiskit has this basically build in for qubits and as a design choise, but it seems a bit strange to go always through qiskit just because I have some kind of "remote"-quantum hardware.

For the interaction with more complex hardware like cold atoms etc this seems like a crucial feature. But who am I too judge if this is crucial. pennylane survived until now without this feature too.

[trbromley]

Thanks @fretchen! I agree that this is indeed a pain point for long-running remote executions. Non-blocking/asynchronous execution is something we'd love to build up, we just need to work out the details to make it work nicely :100:

[fretchen]

Sounds good. Feel free, to tell me if you are looking for input. Just as ressources:

• @RohitPrasadBhatt came up with a first solution. An example can be found here. However, it felt so foreign to the standard procedure of operating pennylane that we felt that we should not pursue it within the plugin but rather upstream.
• The way we typically communicate with the remote device is with a fairly standard REST API. See here for documentation.

I am looking forward to the implementation :wink:.