phase coherent readout with multiplexed readout

[Cameron-Spence-Riverlane]

Hi, I am attempting to perform phase coherent readout using a ZCU216, with a bitfile slightly modified from the q3diamond. My DAC and ADC modules are:

Board: ZCU216

    Global clocks (MHz): tProcessor 349.997, RF reference 245.760

    7 signal generator channels:
    0:  axis_signal_gen_v6 - tProc output 0, envelope memory 65536 samples
        DAC tile 2, ch 0, 32-bit DDS, fabric=430.080 MHz, fs=6881.280 MHz
        ...
    6:  axis_sg_mux4_v2 - tProc output 6, envelope memory 0 samples
        DAC tile 0, ch 0, 32-bit DDS, fabric=430.080 MHz, fs=1720.320 MHz

    4 readout channels:
    0:  axis_pfb_readout_v2 - controlled by PYNQ
        ADC tile 2, ch 0, 35-bit DDS, fabric=307.200 MHz, fs=2457.600 MHz
        maxlen 1024 (avg) 1024 (decimated), trigger bit 8, tProc input 0
        ...
    3:  axis_pfb_readout_v2 - controlled by PYNQ
        ADC tile 2, ch 0, 35-bit DDS, fabric=307.200 MHz, fs=2457.600 MHz
        maxlen 1024 (avg) 1024 (decimated), trigger bit 11, tProc input 3

However the phase is always incoherent since the fabric clock frequencies are different. Is there a way to align the phase in software, or will I need to modify the clock rate in firmware? Cheers

[meeg]

What's your definition of phase coherent? If you mean "if I take multiple shots with the same settings, separated by an arbitrary amount of time, I should always get the same phase" that should be possible here . . . unless you are reloading the FPGA between shots or something?

[Cameron-Spence-Riverlane]

I am referring to coherence between the DAC output and the ADC LO (across different frequencies etc.); loopback tests show that they are incoherent. I expect an offset that I need to calibrate, as in demo 01, but that relies on the ADC and DAC having the same clock frequency. As it is, instead of a linear relationship I have a constantly fluctuating phase difference. When you say it should be possible, do you mean with post-processing?

[meeg]

No, there's no post-processing involved. I would expect the following:

1. if you make multiple measurements of the phase at a given frequency, you will consistently get the same value - this offset will change when you reload the FPGA
2. if you make measurements of the phase at different frequencies, you will see a linear relationship with a very steep slope (you can see in the 01 demo that the slope gives you a 2pi shift in a few kHz) - this slope will also change when you reload the FPGA

Two cautions:

1. you must follow the rules given under "how to ensure frequency matching" in the 00 demo
2. if your frequency range is too wide, your phase vs. freq plot will be aliased and may look random

[Cameron-Spence-Riverlane]

Thanks for the cautions. I have ensured the frequency matches the adc frequency, and am measuring with <kHz resolution, so that shouldn't be a problem.

The phase is a constant value for repeated measurements at a given frequency (although it seems to take a few measurements ~40 to settle at the constant value - maybe this is the source of the problem loopback_phase.pdf ). Even over a small frequency sweep of ~100 Hz, the phase offset still appears random, I don't see any relationship with the frequency.

[meeg]

I just noticed that one of your generators is the mux gen - is that the generator you're using? This generator (and this generator only) is not phase coherent with respect to frequency shifts - if you measure, shift the frequency, and measure at the new frequency, you cannot predict the new phase, and if you shift back to the original frequency and measure again, you will not get the original phase.

[Cameron-Spence-Riverlane]

That is interesting, it is not the generator I am using to test (that is DAC 0), but I was hoping to use it for coherent readout. Does that mean it is not possible to do phase-coherent frequency sweeps with the mux generator? What is the cause - could I predict it and account for it in software? Would it work keeping the mux frequencies fixed and sweeping the mixer frequency?

[meeg]

It is not possible to do phase-coherent frequency sweeps with the mux generator, you can only use it for amplitude measurement as a function of frequency or full IQ measurement at a fixed frequency. The mixer is also not phase-coherent wrt frequency shifts. The mixer is actually just the RF data converter's built-in NCO, and that is phase-continuous not phase-coherent (you can't have both!). So if you shift frequency and shift back, you accumulate a phase lag/advance that is proportional to the time integral of the frequency shift.

It is not possible to account for this, because you would need to know the exact times at which you changed frequencies but the frequency changes were not done at exact times. I believe the mux generator's issue is closely related - the DDS oscillators used in the mux gen are free-running (for ease of design) and therefore phase-continuous, the DDSes in the other generators are referenced to master clocks and therefore phase-coherent.

But OK, you're using a standard generator. Can you try to reproduce your problem on one of the original q3diamond firmwares and/or the standard ZCU216 firmware, and give me a minimal program or notebook? I would expect that the phase vs. frequency sweep should work on any of these (it's certainly been tested on the standard firmware, and I think also on q3diamond).

[Cameron-Spence-Riverlane]

I have the same trouble using the q3diamond (216) bitfile. I have attached my code below, I have made it as similar as possible to that in demo 01 (indeed, running demo 01 with the correct dac channels swapped in also yields the same results).

Is this limitation of the mux generator documented? I would imagine phase coherent sweeps being a fairly common use case (we can work around it though)

I appreciate the help!

from typing import List

import numpy as np
from qick import AveragerProgram, QickConfig, QickSoc

def calculate_phase(s21_data_complex):
    [x_real, x_imag] = s21_data_complex
    x = x_real + 1j * x_imag

    x_avg = np.mean(x)
    fi = np.remainder(np.angle(x_avg, deg=True) + 360, 360)
    return [fi, np.abs(x_avg), np.std(x)]

class loopbackProgram(AveragerProgram):
    def __init__(self, soccfg, cfg):
        super().__init__(soccfg, cfg)

    def initialize(self):
        cfg = self.cfg
        dac = cfg["dac"]
        adc = cfg["adc"]
        nqz = cfg["nqz"]

        freq = soccfg.adcfreq(cfg["pulse_freq"], gen_ch=dac, ro_ch=adc)

        self.declare_readout(
            ch=adc, length=cfg["readout_length"], freq=freq
        )

        idata = 30000 * np.ones(16 * cfg["length"])
        self.declare_gen(ch=dac, nqz=nqz)
        self.add_pulse(ch=dac, name="measure", idata=idata)

        freq_int = self.soccfg.freq2reg(freq)
        self.trigger(pins=[0], t=0)

        self.set_pulse_registers(
            ch=dac,
            style="arb",
            freq=freq_int,
            phase=0,
            gain=cfg["pulse_gain"],
            waveform="measure",
            mode="periodic",
        )
        self.synci(200)

    def body(self):
        adc = self.cfg["adc"]
        dac = self.cfg["dac"]
        self.trigger(
            adcs=[adc], adc_trig_offset=0
        )
        self.pulse(ch=dac, t=0)
        self.wait_all()
        self.sync_all(200)

def send(
    soc: QickSoc,
    soccfg: QickConfig,
    config: dict,
    axis: List[float],
) -> List[List[float]]:

    results = []
    for x in axis:
        config["pulse_freq"] = x
        prog = loopbackProgram(soccfg, config)
        avg_data = prog.acquire(soc, load_pulses=True, progress=False, debug=False)
        length = config["readout_length"]
        data = [[prog.di_buf[i]/length, prog.dq_buf[i]/length] for i in range(1)]
        results.append([calculate_phase(d) for d in data])

    soc.reset_gens()
    return results

if __name__ == "__main__":
    import matplotlib.pyplot as plt
    soc = QickSoc()
    soccfg = soc
    config = {
            "dac": 0, # signal gen v6
            "adc": 0,
            "reps": 100,
            "nqz": 2,  # Nyquist zone
            "length": 100,  # [Clock ticks]
            "readout_length": 100,  # [Clock ticks]
            "pulse_gain": 30000,  # [DAC units]
            "pulse_freq": 4999,  # [MHz]
        }

    f_axis = list(np.linspace(4999,5000,1000))
    results = np.asarray(send(soc, soccfg, config, f_axis))

    plt.plot(results[:,0,0])   
    plt.savefig("loopback_phase.pdf", format="pdf")
    print("Test complete")

[meeg]

Thanks, I can run this later today.

To be honest the mux and int4 generators, and the PFB readout, are not documented at all because they're not part of the standard firmware, and are basically still in testing and only available by request. I recognize that this is hardly best practice and our docs do need to catch up.

[meeg]

OK, sorry for the delay - but the PFB readout that is used in q3diamond and your firmware has the same property as the mux gen. It uses a free-running DDS, it is not phase-coherent with respect to frequency changes, etc. I do see that your test, and the 01 demo, fail on q3diamond, and clearly I was wrong when I thought I'd done that test previously.

[Cameron-Spence-Riverlane]

I see! Thank you for testing it.

Do you have a firmware that has a mux gen, PFB adc, signal gen and standard adc? I can perform the characterisations with the mux gen and IQ tests with a single tone sweep.

[meeg]

I don't. Could you do what you need by switching between a mux-gen, mux-RO firmware like q3diamond and a standard-gen, standard-RO firmware like the default firmware?

[Cameron-Spence-Riverlane]

Yes, that's a good solution, thank you!