recovered clock routing

[hartytp]

Discussing plans for WR with @sbourdeauducq https://freenode.irclog.whitequark.org/m-labs/2019-02-22

Once concern with WR is the possibility of adding noise to the recovered clock when it is routed inside the FPGA to the DDMTD FFs. The best option seems to have both DDMTD inputs coming from IOs in the same bank. We can then ensure that the FFs are in the IOBs.

In ultrascale FPGAs, the most direct way of accessing the recovered clock is via a MGTREF pin, which can be configured as an output.

Action:

• @gkasprow can you route a MGTREF output to an IO in bank 66? (@sbourdeauducq any preference as to which MGTREF we use?)

@sbourdeauducq what is the best means of accessing the recovered clock on Artiq FPGAs?

cc @WeiDaZhang

[gkasprow]

ping @sbourdeauducq

[sbourdeauducq]

* @gkasprow can you route a MGTREF output to an IO in bank 66? (@sbourdeauducq any preference as to which MGTREF we use?)

What for? I mentioned routing a OBUFDS_GTE3 to the RTM FPGA (if easy to route and if there are enough pins). Is this for doing DDMTD tests with a locally-clocked circuit all in Ultrascale? Maybe we can add uFL connectors to I/Os instead, then add jumper cables to do this experiment?

@sbourdeauducq what is the best means of accessing the recovered clock on Artiq FPGAs?

https://github.com/sinara-hw/Sayma_AMC/issues/37#issuecomment-466924665

[sbourdeauducq]

I mentioned routing a OBUFDS_GTE3 to the RTM FPGA (if easy to route and if there are enough pins).

Maybe we can just have two uFL that go through the RTM connector and then we can choose to connect them to the MGTREFCLK uFL with jumper cables. The advantage of going through the RTM connector (instead of having uFL cables between AMC and RTM) is the cards can still be inserted easily into the µTCA rack, and it also looks cleaner.

[hartytp]

What for? I mentioned routing a OBUFDS_GTE3 to the RTM FPGA (if easy to route and if there are enough pins). Is this for doing DDMTD tests with a locally-clocked circuit all in Ultrascale?

Yes, this is for testing the DDMTD/DCXO PLL on the AMC. I think this is a really worthwhile test to do. Apart from potentially being useful for Sayma, it is interesting in its own right -- the CERN literature concludes that the performance of WR is limited by the LVDS IO buffers used in FPGAs. Getting the signal out from the (CML) OBUFDS_GET3 and feeding it into a dual ECL DFF might give substantially better results (or it might not if the OBUFDS_GTE3 turns out to suck). Combining that with the higher-order loop filter we're using and the DCXO (which is a fair bit better than the DAC + VCO used in WR) could give some really nice results. Obviously, that's far less important than getting Sayma up and running, but worth leaving the option open for us to play with this.

Maybe we can add uFL connectors to I/Os instead, then add jumper cables to do this experiment?

That would be an acceptable option. However, I'd prefer to add a clock fanout if it can be done easily. While we're playing around with different clocking options, it's nice to be able to switch between them in software (e.g. Creotech have expressed a strong preference for this, as they can do automated testing).

Ideally, I'd like to use a fanout to drive the OBUFDS_GTE3 to the AMC<->RTM connector, the DFF, the SI5324 and a diff IO in the same bank as the other WR signals. That way we keep all options open for testing/evaluation. Currently, the recovered clock is routed from an IO to the Si5324, maybe use a mux for the fanout and use that IO as the second input?

[sbourdeauducq]

Getting the signal out from the (CML) OBUFDS_GET3 and feeding it into a dual ECL DFF might give substantially better results (or it might not if the OBUFDS_GTE3 turns out to suck).

Yeah sure, but why connect MGTREF to to an Ultrascale IO pin?

[hartytp]

Low priority, but might be interesting to compare DDMTD with the external DFF with DDMTD with FFs in the FPGA. For the internal FF case, it's possible that placing both DDMTD FFs in adjacent IO tiles and routing the recovered clock externally from the OBUFDS_GTE3 to the LVDS inputs will give lower noise than routing the recovered clock internally through the FPGA. Essentially, just trying to give us as many different options for implementing this as possible so we can have a play.

[sbourdeauducq]

Let's add uFLs. Also for the Ultrascale I/O pin.

Those are just for experiments, not production testing, and the uFLs are easier to layout/route than the muxes and also we can connect them to anything, not only what we had planned to in the beginning.

[hartytp]

Agreed, we should definitely make sure we connect all critical clocking signals to uFL/MMCX/whatever (e.g. after fanouts). The muxes are low priority, and I'm not overly fussed either way. I'm happy to leave that decision to greg.

[sbourdeauducq]

To clarify what I mean:

• add differential uFLs on a pair of general-purpose Ultrascale I/Os that support LVDS. Probably with some adapter so we can connect CML there directly. Choose a GC pin just in case.
• add (more?) differential uFLs on a GTREFCLK that can output the DRTIO recovered clock
• add differential uFLs on a pair that goes to the RTM connector
• on the RTM side, add some adapter for CML signals and connect to FPGA I/O with LVDS. Choose a GC pin just in case.
• then we do the routing manually using uFL jumper cables when experimenting with the board.

[hartytp]

Probably with some adapter so we can connect CML there directly

Yes, some pads for RC matching elements is a good idea!

[gkasprow]

done