AMC <-> RTM FPGA connectivity

[hartytp]

This has been covered a bit in other threads, but we need to make sure that we have all the AMC FPGA <-> RTM FPGA connectivity that M-Labs want. This includes both allocating lines on the AMC <-> RTM connector and ensuring they are connected to appropriate FPGA pins.

[gkasprow]

at the moment we have 2 LVDS pairs + 1 MGT pair.

[gkasprow]

I managed to route 16 LVDS lines between AMC FPGA and mezzanines + 2 LVDS lines between AMC FPGA and RTM FPGA

[gkasprow]

some of them would be shared with MGT Rx lines. So one can use them either as LVDS or JESD204B without issues, just by replacing capacitors.

[gkasprow]

Let's route all 16 MGT Rx lines as LVDS or MGT. I still have 7 LVDS lines that would be left as they are. In this way one can place on mezzanine even 8-lane JESD204B ADC when needed or decent speed serial ADC, i.e. 4x125MHz which needs 10 LVDS lanes.

[hartytp]

Nice! Do we still have a transceiver to use for DRTIO?

Can you post a shot of the schematic with the AMC FPGA pin mapping for the AMC<->RTM connector, please?

So one can use them either as LVDS or JESD204B without issues, just by replacing capacitors.

Cool. What frequency will that work up to?

[gkasprow]

I will post once I make sure that I implemented changes on AMC, RTM and BaseMod. It will work up to 1.5Gb/s easily, depends on speed grade. I want to remove LVDS connection between FMC DP lane. We won't use it anymore and I will get GC inputs needed for AFE mezzanines. In this way we will get 8 fixed LVDS links + 16 shared with MGT lanes. A lot of options to explore.

[hartytp]

@gkasprow naming for these LVDS/MGT signals. Can we call them LVDS_N, appending _CC for any CC pins, and starting numbering from 0?

[gkasprow]

I already did that.

[hartytp]

@gkasprow since Sayma AMC may be used with other RTMs, let's try to keep the signal names a bit more generic. E.g. on some future RTM the LVDS lanes may not connect to an AFE mezzanine (there may not be an AFE mezzanine). Let's just name them by the signal type, so LVDS_0 etc

[hartytp]

Other than that, looks good!

FWIW, I still suspect that we can scrap AMC_MASTER_AUX_CLK since there is no HMC7043, but @sbourdeauducq should confirm what he wants to do here (is it worth routing a reference clock between the AMC and RTM for debugging, or are we happy to rely on the clock recovery)?

[gkasprow]

We will have REF clock routed to AFEs anyway, so once we install serial LVDS ADC, it will generate its own copy of the clock. We have GTP clocks anyway. The only clock that may make sense is to route clock from Sayma AMC CDR to clock mux on RTM side and for this purpose we may use these signals .

[hartytp]

The only clock that may make sense is to route clock from Sayma AMC CDR to clock mux on RTM side and for this purpose we may use these signals .

Yes, I think that's right.

I don't think there is any point routing the RTM reference clock to the AMC FPGA, since the AMC FPGA will be clocked from the CDR clock.

We could imagine wanting to route the AMC CDR clock to the RTM clock mux in case there is some issue with the WR clock recovery. I suspect that there isn't much point doing this because the cross-talk in the AMC<->RTM connector will be quite bad, so this will not be a low-noise clock.

So, I think we can just scrap the AMC_MASTER_AUX_CLK signal and use it as general purpose LVDS/LVCMOS IO for the AFEs. I'm just giving @sbourdeauducq a chance to object and tell us if he needs a clock routed between the AMC and RTM.

[gkasprow]

Do we care about AMCP2P link? I had free LVDS links so routed them to dedicated links between neighbouring AMC boards (PORTS 12...15). No idea what it could be used for. The same with FatPipe2 which I already removed. LVDS over FatPipe1 (Fabric E) makes some sense because we can assign LVDS links on Metlino and run second DRTIO channel over it.

[hartytp]

Do we care about AMCP2P link?

In principle, one could imagine using that for fast feedback/feedforwards between cards without involving DRTIO. For example, one could imagine a fast ADC AMC being used to modulate Sayma RF.

In practice, I'm not aware of any plans to do that (we will generally keep that kind of feedback local to an AMC/RTM pair) and don't think it's likely that we will do it.

Personally, I think that if we're short of IO then it's more useful to route IO between the AMC and RTM than to route it between separate AMCs, so I'd scrap the AMCP2P links.

[gkasprow]

We are not short, I still have some SSTL15 pins in SDRAM bank. It's matter of routing them later on. Schematic is the easiest part, adding signals to already routed BGA is often tricky.

[gkasprow]

I can leave 2 pairs instead of 8 pairs as it was in rev1.

[hartytp]

ACK. Well, I don't think it's at all likely that we will use these signals, so I wouldn't bother routing them (let's just focus on optimising the SI for the signals we will actually use, as well as PI etc).

[gkasprow]

OK, so remove them all.

[gkasprow]

Anyway, removing connections that are not necessary, makes board far easier to test. In production test suite we test all testable electrical connections so the fewer of them, the better. Development of tests takes time and resources. And also increases the chance that board will fail such test. So keep it simple. In proto version we added a lot of features mainly to see what is really needed.

[gkasprow]

I re-assigned some FMC signals and now we have total 27 of LVDS lanes. 3 of them are CC 19 of them including 2CC are in same bank I routed 3 of them (one CC) to RTM FPGA Remaining 24 lines will be distributed:

• 8 lanes with 2xCC routed to RTM directly and then to AFE
• remaining 16 routed as 0R assembly option to MGT Rx lanes. Populated to support LVDS by default. So even if we decide to go for AFE with 8x JESD204B, we would still have 4 LVDS to implement control signals.

[gkasprow]

This should work

[gkasprow]

we can even move the DAC control lines to banks 47/48 and have all AFE signals in same bank. But I'm not sure we want to move DAC sync lines to the bank without CDR clock... That's why I kept them in this bank

[hartytp]

Thanks @gkasprow we're getting there. There are a few final things that we need to do to complete this:

• [x] Let's not name LVDS/MGT signals by their destination/use, but only by whether they're LVDS of MGT. e.g. RTM_FPGA_LVDS should just be RTM_FPGA_LVDS. The reasoning for this is that Sayma AMC may be used with many different RTMs, so we shouldn't encode the use in the signal names on the AMC. e.g. for the next HW revision, we may want to keep the AMC the same but change how these signals are used on the RTM. Same goes for the transceivers, just name them by signal type and number, not end use.
• [ ] @sbourdeauducq needs to confirm this, but I believe we can remove the GTP_CLK signals from the AMC<->RTM connector. In the next revision there will be no HMC7043, so there is nothing to drive these signals. Let's just make them more LVDS IO.
• [x] There is nothing special about the FPGA_DAC_SYSREF signal, it's just more LVDS, so let's just name it LVDS_N

[hartytp]

• [x] Don't we need the WR SMA reference connected to the same bank as the helper DCXO. I can't see that on the schematic.
• [x] More generally, let's try to keep all critical WR signals as close together as possible and not spread them out on the FPGA.

[hartytp]

we can even move the DAC control lines to banks 47/48 and have all AFE signals in same bank. But I'm not sure we want to move DAC sync lines to the bank without CDR clock... That's why I kept them in this bank

Let's leave this as it is. We now have more than enough LVDS IO, so no need to worry about this.

[hartytp]

Other than that, looks good!

[gkasprow]

OK, the schematic is cut and it doesn't show the WR SMA input. It is connected to GC in bank 47. I can move it to same bank as WR clocks, but it won't be GC. But I think we don't care, do we? I mean that FPGA_DAC_SYSREF is routed to DAC while other signals are user-defined., I don't want to change the signal name because it leads to confusion.

[gkasprow]

Let's not name LVDS/MGT signals by their destination/use, but only by whether they're LVDS of MGT. e.g. RTM_FPGA_LVDS should just be RTM_FPGA_LVDS. The reasoning for this is that Sayma AMC may be used with many different RTMs, so we shouldn't encode the use in the signal names on the AMC. e.g. for the next HW revision, we may want to keep the AMC the same but change how these signals are used on the RTM. Same goes for the transceivers, just name them by signal type and number, not end use.

I'm not sure I follow you here. The signals are called LVDSxx and they enter the RTM connector via resistors. They are grouped into the bus RTM_LVDS_IO. FPGA pins are called LVDSxx. The MGT signals are called GTxxRX/TX and are grouped into the bus RTM_GT. They enter the RTM connectors via DNP capacitors.

[hartytp]

OK, the schematic is cut and it doesn't show the WR SMA input. It is connected to GC in bank 47. I can move it to same bank as WR clocks, but it won't be GC. But I think we don't care, do we?

That SMA doesn't need to be a GC pin or even a CC pin, since it just goes to the DDMTD DFF data input.

Let's prioritise keeping all WR signals in the same bank and as close together as possible.

I mean that FPGA_DAC_SYSREF is routed to DAC while other signals are user-defined., I don't want to change the signal name because it leads to confusion.

On Sayma v2.0 that's not true. (a) because the DAC SYSREF goes to the DFF + delay line and before the DAC and (b) because we will probably use the RTM FPGA to generate the DAC SYSREF signal. From my point of view, it's best to just name these signals by their logic levels so that the schematic does not become incorrect if we change how these signals are used on the RTM.

[hartytp]

I'm not sure I follow you here. The signals are called LVDSxx and they enter the RTM connector via resistors. They are grouped into the bus RTM_LVDS_IO. FPGA pins are called LVDSxx. The MGT signals are called GTxxRX/TX and are grouped into the bus RTM_GT. They enter the RTM connectors via DNP capacitors.

I mean, let's rename the signal RTM_FPGA_LVDS to just LVDS. That way if we reroute these signals on the RTM, the AMC schematic does not need to be changed.

[hartytp]

Similarly, I'd call RTM_FPGA_GTP just GTP

[gkasprow]

oh, you mean the bus name. OK

[gkasprow]

OK, you mean the signal that goes to the Artix chip

[gkasprow]

Schematics with fixes

we still have 2 LVDS pairs on FPGA without RTM assignment.

[hartytp]

@gkasprow Good thanks. I'm happy to close this issue now. Some final comments from me:

• I'm not reviewing things like RTM JTAG, so I assume someone else is checking that (I noticed that one JTAG line does not connect to anything on the RTM connector)
• It's not crucial, but is there a reason not to merge the RTM_GT and RTM_FPGA_LINK busses into a single RTM_GT bus? Again, the point here is that as far as the AMC is concerned it's all just MGT. It's up to the RTM hardware designer to decide how it's used, but that shouldn't be prescribed in bus names on the AMC. If there is some reason that it's hard to merge these busses in the schematic then it's fine to leave them as they are.
• I don't think we need the 2 GTP clock lines anymore since there is no HMC7043. However we now have more than enough IO so it's fine to leave them there if you want.

[sbourdeauducq]

FWIW, I still suspect that we can scrap AMC_MASTER_AUX_CLK since there is no HMC7043

Yes, we can remove it. It is disabled in the current firmware.

As for removing GT clock links from RTM to AMC, there are two things to check:

• is the jitter low enough for JESD204 to work at the maximum clock speed? I suppose those transceivers would be clocked from the Si5324?
• all clock pins (Si5234 and WR) must be able to drive all JESD quads and all DRTIO quads (for AMC backplane DRTIO, RTM DRTIO, and SFP DRTIO if we support that). On Ultrascale, a transceiver clock pin can drive its quad, and the two quads above and below it (max 5 quads from one clock pin).

[gkasprow]

bank 224 has all DRTIO links. It can be driven by Si5324 directly. Or can be driven by GTCLK0 via RTM side and bank 226. I added a connection between Si5324 and bank 226 clk1. In this way, Si5324 would be able to drive JESD as well.