external clocking

[sbourdeauducq]

Do we want to use Metlino as a DRTIO master that distributes an external clock to the DRTIO downstream devices? If so, we need to send that clock to the FPGA transceivers, and currently the only path that allows this is FRONT_SMA_CLK_IN -> AUX_CLK -> Si5324 in bypass mode -> CDR_CLK_CLEAN.

Does the Si5324 bypass mode have phase noise good performance? If we start developing fancy WR-style clock recovery circuits and use DRTIO to distribute high-quality clocks, we should be careful about this.

[hartytp]

Weida measured the noise of the Si5324 in bypass mode (I posted the data on one of the issues). It’s not good and we shouldn’t rely on it.

IIRC on Sayma we had a means of getting an ext SMA to an FPGA pin without going through the Si5324 or anything.

What we can potentially do is use the FPGA as a mux to select which clocks goes to the Si5324.

[sbourdeauducq]

Weida measured the noise of the Si5324 in bypass mode (I posted the data on one of the issues). It’s not good and we shouldn’t rely on it.

Will it contribute to the noise after going through the transceivers, and CDR at the receiver side? Does this noise matter on local RTIO signals (e.g. TTLs on VHDCI)? If not, we are still good.

What we can potentially do is use the FPGA as a mux to select which clocks goes to the Si5324.

How does that help with the problem of getting a clean external clock to a FPGA transceiver clock pin?

[hartytp]

Yes, that’s right, the topology I thought we were implementing had the si534 and dcxo outputs going to a mux, which feeds the transceiver. The ext ref SMA goes to the FPGA where it either supplies the reference for the ddmtd pll or it is routed from the FPGA to the si5324 (as a fallback solution).

[hartytp]

The noise may be okay, I’d have to re-check in-band spurs etc, I’d be more worried about long-term phase drift etc. It’s been a while since I thought about this but iirc the si5324 inbypass mode is still a long way off being just a buffer and shouldn’t be used as such

[sbourdeauducq]

OK, then that needs a hardware change to be a on the safe side.

[gkasprow]

Do I have to change sth in the HW design? In Metlino we have the same direct FPGA input as on Sayma. It's MCX but it's facing the rear MCH module side together with nearby clock output.

[sbourdeauducq]

Yes, there needs to be an external clock input that can clock all the transceivers, with a low-noise low-drift path. According to @hartytp we should not use the Si5324 as a clock mux. The "external clock input" J10 cannot reach the transceivers (other than with the fabric path which is noisy and not recommended by Xilinx). Either mux it into CDR_CLK_CLEAN, make a "capacitor selection" circuit to use it instead of the Si5324, or connect it to quad 226 (moving CLK200 to quad 228 on K5/K6 as I suggested earlier) and we can use the FPGA transceiver circuitry as clock mux. I prefer the latter option. We do not need two copies of CDR_CLK_CLEAN on transceiver pins; GTH clock routing works correctly.

[sbourdeauducq]

In Metlino we have the same direct FPGA input as on Sayma.

Sayma is meant to be used as satellite or repeater and using a CDR clock, whereas Metlino can be the root of the clock tree. How do we define the clock at the root?

[hartytp]

@sbourdeauduqc what about just locking the dcxo to the external ref in master mode and then using the standard fan out to route that to the transceivers?

[hartytp]

That’s how we designed this to operate, under the assumption (based on measurements we’ve taken) that the dcxo/ddmtd doesn’t add significant noise/drift

[sbourdeauducq]

locking the dcxo to the external ref

How?

[hartytp]

Ext ref goes to an FPGA io that’s a DDMTD input. Lock the DCXO to that. DCXO already routes to the transceiver.

In fallback mode the FPGA routes that input to an output which drives the Si5325, which drives the transceivers via a mux

[sbourdeauducq]

And that path through the FPGA won't drift or add noise? One point of that DDMTD flip-flop circuit was to avoid paths through the FPGA fabric...

[hartytp]

Aah right id forgotten the plan to use the external DFF. In that case your approach sounds good.

FWIW on the KC705 the fabric DFFs did perform very well. Weida is preparing to test a KC705 with external DFF but so far we have no reason to think it will improve things (may just be limited to transceiver cdr). The external DFF is mainly a protection against ultra scale specific issues

[sbourdeauducq]

If the external DFF does not improve things then why does the Ultrascale have OBUFDS_GTE3?

[hartytp]

It may improve things on ultra scale (maybe also on other families) needs testing. FWIW though the performance we got on the KC705 with io dffs was already very good so this is a nice to have rather than a strict requirement so long as the ultra scale can produce the same performance as the Kintex using io dffs.

[sbourdeauducq]

I would expect better performance since we're not going through the noisy fabric. There seems to be a significant difference between transceiver and fabric blocks even before Ultrascale.

[hartytp]

Let’s hope so. I’ll let you know what Weida finds on the KC705 but ultra scale will have to wait for metlino

[gkasprow]

I connected clock mux output to bank 226 clk1. CLK200 was moved to bank 228 as @sbourdeauducq suggested.