More realistic Use case

[NicoRenaud]

We have been using so far a very simple system with 5 antennas in a line and no noise (?) We have to move towards more realistic setting. One option would be to integrate our different LS into existing libraries for calibration a few options are :

• https://github.com/HERA-Team/hera_cal (https://arxiv.org/pdf/2003.08399.pdf)
• https://github.com/bnikolic/simpleredcal (http://reionization.org/wp-content/uploads/2013/03/HERA084__A_Generalized_Approach_to_Redundant_Calibration_with_JAX.pdf)
• https://gitlab.com/ska-telescope/sim/ska-sim-low (not sure it does calibration)
• https://gitlab.com/ska-telescope/ska-low-mccs
• https://ska-telescope.gitlab.io/external/rascil/index.html
• https://github.com/lofar-astron/DP3
• https://github.com/lofar-astron/factor
• SAGECal ?
• ASTRON library ?

[broekema]

What we discussed previously is that we can approach a more realistic approach to our quantum experiments in two different ways:

1. we can look at software used in actual telescopes and introduce a quantum "accelerated" module, or
2. we can keep using our simple experimental software and feed more realistic data to it (i.e. more antennas, add noise, etc)

In my initial estimation, the first would be more useful, the second easier to achieve.

I've been looking at the ASTRON DP3 software. It uses some LAPACK solvers in its calibration routines. This may be a good starting point, assuming we have drop-in replacements for these functions.

https://git.astron.nl/RD/DP3/-/tree/master/ddecal/linear_solvers

[broekema]

@NicoRenaud would a more generic task of "find drop-in replacements for often used LAPACK solvers" be viable?

[broekema]

Spoke with Stefan Wijnholds. DP3 is a good representative software package. We could also look at RASCIL, which is a python based prototype at SKA. There are some solvers used there as well, but these are provided by numpy.

So a drop-in replacement in numpy could also be useful here.

https://gitlab.com/ska-telescope/external/rascil/-/blob/master/rascil/processing_components/calibration/solvers.py

[NicoRenaud]

@NicoRenaud would a more generic task of "find drop-in replacements for often used LAPACK solvers" be viable?

Yes I think replacing Lapack/Numpy solver by qiskit solver is what this issue is about. Replacing the Lapack/Numpy call in any of these libraries would allow us to use their example to illustrate the method

[broekema]

In that case I would suggest starting with a quick spike to select either Lapack or numpy. I feel the lapack route would add more value since I suspect numpy would use this under the hood anyway. Furthermore, the ddecal code is in my initial estimation easier to understand than the rascil python code.

The following Lapack functions could be targeted: cposv cgels cgelss

[NicoRenaud]

it would be easier to integrate with a python code so that we don't have to call python in C++ and then run the IBMQ API there. So I would be in favor to try with rasciil first and see which numpy linsolve we can replace by vqls or hhl.

[broekema]

I don't mind that, but be aware that that code does not seem to call any particular solve routine, but rather implements it itself through a number of individual linear algebra routines. I believe this is due to the fact that numpy only provides one solver that does not match the requirements of this usecase. (scipy would have been a better choice I suspect)

This could still be viable, but tricky.

[NicoRenaud]

Ok I'll take a deeper look at it :) but if Numpy fails to solve the system i'm not sure VQLS will do better ...

[NicoRenaud]

The lapack routines mentioned by Chris :

• cposv complex system of equation through Cholesky where the matrix is hermitian and positive definite
• cgels over/under determined complex system through QR
• cgelss minimum norm solution of complex least square through SVD

So VQLS can do what cposv does now would it be faster than Cholesky, I'm not sure :)

[NicoRenaud]

rascil only implements selfcal that is a minimization process (if I get it correctly) see e.g. Stefan's paper : http://cas.tudelft.nl/pubs/wijnholds09eusipco.pdf

So we might be able to use a QUBO formulation similar to what we have for linear system but with a different Q matrix, and then use DWAVE for that. But VQLS is not the way to replace solvers in rascil

[NicoRenaud]

hera and and simpleredcal are impossible to use ! The code and doc are out of sync and the data they use us unfindable. It seems to only exists on their local cluster ... awesome

[NicoRenaud]

Onmical : https://github.com/jeffzhen/omnical is an even less documented code to do redcal ..

[NicoRenaud]

The documentation of DP3 is kinda sparse ... However I found that repo with this notebook where they seem to be doing some sort of calibration for LOFAR. They also provide the data so that we can run the notebook. @broekema do you think that is a use case where we could replace the calibration with VQLS ?

The calibration is done by this line that almost look like the solution of a linear system except that they are taking the element wise division of the vis by the gain matrix ...

[broekema]

Yes, that looks fine. I appears to be a set of tools used by well known astronomers at astron. Code is a bit old, but the contributors are known to me.

[broekema]

Note that this is doing gain calibration, essentially normalising the fluctuations in antenna gains over the elements in a station to ensure each antenna contributes equally to the total gain of the station.

[broekema]

Check page 23 and further of the LOFAR cookbook for an explanation of DP3, available here: https://git.astron.nl/ao/sdco/lofar_imaging_cookbook/-/blob/master/source/docs/LOFARImagingCookbook_latest.pdf

[NicoRenaud]

There is an html of the lofar imaging cookbook : https://support.astron.nl/LOFARImagingCookbook/index.html It describe how to use the software on the Astron cluster only

[NicoRenaud]

So the easiest path is I think to try redcal with hera. They have some data they use for training/testing that I can see how to use.

[NicoRenaud]

Hera cal pipeline in place