Urukul v1.0 detailed testing

[hartytp]

• [x] Channel-to-channel cross-talk: adjacent channels, with agressor, victim switches opened and switches closed and with attenuators at max/min
• [x] Output phase noise/amplitude noise (0.1 Hz - 10 MHz)
• [x] OIP3, OIP2 vs frequency for min attenuator setting and medium/large amplitude (ASF)
• [x] RF switch settling behavior for on edge at max power (worst case)
• [x] Attenuator glitches
• [x] Nyquist attenuation, clock leakage
• [ ] RF switch settling behavior for off edge at max power (worst case)
• [ ] RF switch phase chirp
• [ ] Temperature coefficient of power and phase
• [ ] Are we happy running the IC11 and running the LVDS interface directly from the 3V6 rail? (Try this and check performance)
• [ ] Check performance of clock buffers when running in LVDS mode from the 1V8A rail
• [x] Optimize PLL loop filters (@weidazhang can you do this for the AD9910 boards, please?)
• [ ] Board-to-board crosstalk (in proximity to Kasli, DIO pulsing 5 V into 50 R, Urukul)

Use some fixed reference frequency for all tests, such as 80MHz.

[a-shafir]

@dhslichter there was only one more SMA cable in the lab and no 50R terminators at all. Any way the probes are OK to check how the driver controls.

[jordens]

The screenshots are crappy. tl;dr: All good. The only issue are phase noise spurs at -75dBc at ~1.2 kHz offset with many harmonics, especially x11. We didn't figure out where exactly they came from. The spur offset is surprisingly low. Their offset is temperautre dependent (increases with temperature). And coupling different parts of the PCB (by finger), especially around the underside near the clock fanout, makes them worse. That spur needs to be debugged.

Setup

• Urukul v1.0 (with BIAS and resistor patches)
• urukul-cpld v1.0rc6
• very clean 100 MHz reference, 10 dBm, un-un, into SMA, selected
• divide-by-4 from clock fanout
• PLL locked, multiplier 40, 387µA, VCO5, 1 GHz VCO
• CFR1=0x00000002, CFR2=0x01400820
• output frequency 80 MHz (0x147ae148) or 81 MHz (0x14bc6a7f) unless otherwise noted
• un-un on output
• using channel 1 unless otherwise noted
• 12 V lab power supply or random 12V 6A brick: no difference

low frequency limit

• 100 kHz out: -20 dBm max (attenuator at 63 digital, switch open)
• 1 MHz out: 10 dBm

switch isolation

• 70 dB (attenuator at 63 digital)

crosstalk

• -84 dB (victim attenuator at 63, switch open, DDS off, aggressor switch open, attenuator at 63)
• -110 dB (same as above but victim switch closed)

cross-channel intermodulation

• -85 dBc (agggressor 1 MHz, 10 dBm, victim 80.2 MHz, IMD product at 81.2 MHz)
• -90 dBc (agggressor 1.234 MHz, 10 dBm, victim 80.123 MHz, IMD product at 81.357 MHz)
• <-100 dBc (agggressor 80.123 MHz, 10 dBm, victim 81.234 MHz, IMD product at 82.345 MHz)

Attenuator step from 20 to 60 digital (16+4dB switch glitch)

Attenuator step from 31 to 32 digital (major carry glitch)

Harmonics DC to 1 GHz, clock and ref leakage below RBW

Harmonics 2 and 3 vs power

6dBm outputs, 3rd harmonic at -53dB

10dBm outputs, 3rd harmonic at -25dB

Close in spurs (1.2 kHz multiples, especially 11x)

• increase and change the harmonics distribution when touching the underside of the PCB near the clock fanout
• increase when pushing the underside into the antistatic bag (and thereby coupling parts of the underside)
• increase in frequency with increasing temperature
• no power rail or testpoint found where whey originate
• power supply does not make a difference
• shielding by covering with a massive Aluminum case does not change them
• adding more un-uns does not change them
• cable routing does not affect them
• distance to kc705 does not affect them

PLL bump and 1 MHz spurs

• spurs at 1 MHz and multiples from the SMPS at -85 dBc on the same level as the specified spurs of the DDS.

Phase noise 100 mHz to 10MHz

Phase noise 10 Hz to 10 MHz

Phase noise 1 kHz to 1MHz

RF switch turn on transient

Nyquist rejection 400 MHz to 600 MHz

Nyquist rejection 450 MHz to 550 MHz

[gkasprow]

@jordens such spur may be caused by PSU instability. Then 1.2kHz ripples caused by SMPS or LDO feedback loop may cause such spurs. So you can have a look with scope on critical power rails.

[jordens]

Ack. We didn't find anything noticable on the rails. We'll do more prodding and poking.

[gkasprow]

@jordens I can have a look with spectrum analyzer or when necessary with SSA. Is ADI devkit sufficient to recreate this spur?

[jordens]

I don't know. But I'd be surprised if you see them on the devkit. I'll give you details about the setup later.

[gkasprow]

No, I mean Urukul controlled by ADI devkit instead of the Artiq

[jordens]

I think you should see them on Urukul with your setup. I added the description of the setup to the post above.

[a-shafir]

@jordens what reference clock used for the test? KC705 output from the FPGA pll possible has the low freq modulation from the power or even fan. And the AD9910 pll most likely will unable to clean it.

[jordens]

@a-shafir the reference is clean. And that hypothesis is already excluded be the other observations.

[jordens]

AD9912 variant also works fine (https://github.com/m-labs/artiq/commit/a940550e4755f92c96408acfd560099e2e2ee3ed) @gkasprow did you get a chance to hunt for the 1.2 kHz spurs?

[gkasprow]

@jordens not yet. The University and my lab was closed during Xmass. I could not even cross the main gate. They open tomorrow.

[jordens]

Ack. That's fine!

[jordens]

The spur might be a beat due to the on board oscillator leaking through the mux (-61 dB isolation, 25 ppm spec, 1.2 kHz is 12 ppm). But the magnitude is still high: the capacitor population should have created much more isolation. The symptoms are otherwise fitting (maybe not the harmonics, though). @gkasprow if you get to testing this before I do, try with and without L9.

[hartytp]

@jordens Based on those measurements are you happy to scrap the screening clips for the next revision? We could also DNP them, but scrapping them frees up quite a bit of space (and, anyway, AFAICT, there wasn't enough space on Urukul to do the screening properly anyway, so the current arrangement is a bit of a bodge!). Or, do you want to take measurements with the screening in place first to see if it actually makes a difference?

[jordens]

Why would you remove the clips now? The pending changes certainly don't need the additional space. And yes, I'd like to test and quantify the crosstalk in a rack with Urukul in proximity to other Urukuls, Kaslis, or DIOs pulsing 5 V into 50 Ohm.

[hartytp]

Why would you remove the clips now?

No hurry for this, just came up now because I was reviewing my to do list for these boards.

My guess is that most of the screening we've added to the EEMs is unnecessary and can be removed at some point once we've finished testing, but there is no rush for that.

[gkasprow]

@jordens tomorrow I plan to spend full day on this issue.

[gkasprow]

@jordens I got such spectra on my setup

[jordens]

@gkasprow those are not the ones I am interested in. Look above for a description, photos and analysis of the ~kHz spurs.

[gkasprow]

above there are mainly setup windows :)

[gkasprow]

I brought a spectrum analyser (N9030A) with SSA function and got such plot. I confirm that the level of spurs gets increased by 20dB when I touch the clock chip

[gkasprow]

I removed L9 and it helped. So in next revision will add MOSFET switch.

[gkasprow]

[jordens]

Excellent, Greg. Yes. A mosfet would work.

[gkasprow]

@jordens do you want to control MLVDS receiver type via FPGA pin ?

[jordens]

Yes. If there is a pin available.

[hartytp]

Closing since Urukul v1.1 has gone to manufacture, so I assume Urkul v1.0 testing is complete (reopen if I'm wrong about that).

[jordens]

I still want those tests.

[hartytp]

Okay, sorry. Overly enthusiastic in my pruning.

[jordens]

No problem.

[jordens]

Another quick shot at verifying that the clock distribution with Kasli and Urukul fundamentally works (modulo the known issues). Instrument noise floor still at -130 dBc/Hz. I would expect flicker phase (0.9 GS/s, 24x PLL, 150 MHz carrier, L(1kHz)=-116 dBc/Hz) to be a bit better given the datasheet graph (1 GS/s, 20x PLL, 200 MHz carrier, L(1kHz)=-120 dBc/Hz). But this may also be a setup or Kasli issue as I measured L(1kHz)=-126 dBc/Hz for Urukul itself, 80 MHz carrier and 1 GHz/40x PLL previously which exactly matches the datasheet. This is between two arms of the ADCLK948.

Edit: The analysis was not properly rejecting common mode aperture jitter. This one is with fixed numerics: 83 MHz carrier, L(1kHz) = -125 dBc/Hz.

[hartytp]

We took some data today using the AD9910 Urukul with a Wenzel oscillator source. Will upload tomorrow.

[hartytp]

Courtesy of @WeiDaZhang: Urukul AD9910 phase noise. Urukul Phase Noise Measurement - Ver1.0 w- XTAL Disabled.pdf

[hartytp]

Urukul noise performance basically matches the data sheet curve.

[hartytp]

Urukul Amplitude Modulation Noise Measurement - Ver1.0 w- XTAL Disabled.pdf

Uruku AM noise. cf

[hartytp]

Conclusions:

• the loop filter isn't quite optimal, which isn't a surprise since the VCO characteristics aren't particularly well specified (not even clear how well controlled they are). We could probably make the servo bump about 7dB smaller with optimisation, but probably not worth the effort.
• the LDO dropout voltage doesn't seem to have too much of an impact. There are a few spurs on the AM output, which may be SMPS leakage. Might be worth adding inductors between the SMPS and LDOs (cheap to do).

Other than that, all looks really good! Congratulations @gkasprow

[hartytp]

NB haven't bothered to check performance without the PLL. We may not quite hit the noise floor of the DDS in that case, but clearly there is no drastic problem.

[jbqubit]

Why did @WeiDaZhang switch to higher phase noise E4422B Ref vs Wenzel?

Looks like <5 dB deviation from Datasheet at single frequency. Nice job! Is it similar at other output frequencies?

[hartytp]

Before I forget: @cjbe tested phase synchronisation with FPGA. All looks good!

[jordens]

Nice. @cjbe @hartytp with the FPGA driving SYNC? Or just among the DDS?

[cjbe]

@jordens with FPGA driving sync. I tested synchronisation between channels on the same board and channels on different boards.

The DDS sync signal was generated by a TTLClkGen, and IO_UPDATE by a TTL SERDES output. I adjusted the sync input delay on each channel to sit in the centre of the eye - thus the SYNC_CLK of each DDS is aligned to better than 1 DDS clk period (=1ns @ 1GHz).

I choose an offset between IO_UPDATE and the sync TTLClkGen such that IO_UPDATE aligned roughly on the falling edge of the DDS SYNC_CLK. (I did not look at the CPLD timing report to work out if this would work with a fixed value over PVT).

I saw no synchronisation errors over ~10^10 IO_UPDATES, but I did see at least one SYNC_SMP_ERR from most of the AD9910 over this 10^10 events using the standard 300ps validation window - I have not looked to see if this happens with slightly smaller validation windows.

[jordens]

Ok. Nice!

[hartytp]

(I did not look at the CPLD timing report to work out if this would work with a fixed value over PVT).

This still concerns me, and is something we need to look at. @jordens do you know how well constrained this timing is atm?

[jordens]

SYNC is not routed through the CPLD. That can't be related to SYNC_SMP_ERRs. How are you checking for IO_UPDATE synchronization errors? I don't know about the constraints there (other than the 7.7ns nominal propagation delay) and whether that's even specified.

[cjbe]

There are two considerations here:

1) SYNC_SMP_ERRs: This is just due to the jitter of the FPGA generated SYNC relative to the AD9910 1GHz clock. I see a narrower eye using the FPGA generated SYNC vs the AD9910 sync (~6 taps = ~450ps for the FPGA generated clock). I am using a validation window of 300ps, so it is not surprising that at least 1 in 10^10 events closes the eye to <300ps, but may be fine for a validation window one tap smaller.

2) IO_UPDATE sync errors: I am setting CFR1 bit 11 so the phase resets on IO_UPDATE, I am then triggering a scope on a signal synchronous with IO_UPDATE, and looking at the RF phase relative to this trigger with a scope on persist. As the RTIO fine clock is 1ns, and the SYNC_CLK period is 4ns, there are 4 possible IO_UPDATE alignments. With one of these I see frequent phase alignment errors on the scope, but with the other 3 I have not seen any - I choose the best out of these three by looking at the IO_UPDATE timing at the AD9910.

My only worry is how large the PVT change in the 7.7ns propagation delay is - if it is larger than ~2ns (which is probably is) things are getting marginal (the IO_UPDATE setup/hold is 1.75ns/0ns). We could register IO_UPDATE on the DDS0_SYNC_CLK at the CPLD - from the CPLD datasheet it looks like the PVT on the clock input buffer and standard output buffer sum to ~0.9ns, so this may well work out.

[jordens]

That's what I thought. I don't think P is relevant for us (you'll have to tweak it anyway). And I don't think VT are more than 500 ps for our VT variability. Registering io update doesn't help AFAICT because (1) there is no way to delay the cpld output to meet sync clk s/h and (2) the s/h window on the cpld is unlikely to be significantly larger than the 2.25 ns of the DDS.

[hartytp]

NB we have an IO_UPDATE return on EEM1 in the current design, so if we see any PVT issues with the phase synchronisation, we should be able to use that to calibrate the delays in the CPLD buffers. Given that, I no longer have any concerns about phase synchronisation with Urukul!

[jordens]

The warmest chip is the clock fanout. With just convective cooling it is ~75C wenn flat on a table.

Pulled fresh out of a crate after thermalizing with convective cooling upright the DDS also get into the 70C.

[hartytp]

What do people think about maybe sprinkling a few cheap, low-profile SMT/THT heatsinks on Urukul (and maybe/other EEMs with significant power consumption) near the most power-hungry ICS?