Sayma RTM to AFE board-to-board connectors

[hartytp]

• [ ] Replace SMPs used for RF board-to-board connectors between Sayma RTM and Sayma mezzanines with multi-pin Samtec connectors.

Context:

• Using SMPs costs about $600 per Sayma, which is a bit crazy. Multi-pin RF connectors are much cheaper
• Using SMPs requires an unacceptable level of insertion force to insert AFE mezzanines (cf 360). Insertion force is much lower with multi-pin connectors. Moreover, potential chances for misalignment are much lower (the connectors have precision alignment pins built in)
• The SMPs take up a lot of space on both Sayma RTM and the AFE mezzanines
• RF performance of multi-pin connectors is excellent (see below for details) and they are available with the correct stacking height.
• If we go for 4-channel AFE mezzanines, we need 18 channels per AFE (use 20-pin connectors):
  • DACs: 2 x 4
  • ADCs: 2 x 4 (if we want to maintain support for the JSED ADC)
  • LO: 2

Risks:

• This needs prototyping. Probably no more risky than any other way of solving the mechanical issues we're seeing with the SMPs on the AFE mezzanines.
• To do: @gkasprow has some connectors on order and is planning to build a cheap FR4 PCB to test them

[hartytp]

My feeling here is that we need to make sure that whatever we do actually works from a mechanical standpoint. Let's not go for something that's a bit marginal and wind up with a massive headache later on.

I'm not worried about the RF properties of RA SMAs: they should be absolutely fine to a few GHz. AFAICT, they are only likely to cause issues for work at 10GHz+. At that point, it's probably better to do the mixing on a separate, specialist board, rather than trying to cram it onto Sayma AFE. Or, one could use smaller connectors, such as SMPs, which work well at these frequencies.

The cost is a slight issue, but if we have to decide between slightly more money spent on SMAs or mechanical issues, then I'd definitely spend the money on RA SMAs (the costs are still nothing compared with the overall cost of Sayma).

Edit: and, anyway, we may end up making some of the costs back by switching to a more standard stackup where FPs and other components are cheaper...

[dhslichter]

So is the issue here that we are trying to mash this into a standard FMC front panel? Why are we doing this? It seems if we make a front panel variant we could get around these issues of clearance, no?

I agree that RA SMA is probably alright for this given our frequencies, and that we definitely don't want to have mechanical issues. However, if someone wants to do an upconverting board (or downconverting board), they might start to care about 12 GHz. If there is a way we can modify the front panel geometry/design a bit to use the 8.5 mm stackup with straight connectors and have enough clearance, that would be my preferred solution.

[hartytp]

So is the issue here that we are trying to mash this into a standard FMC front panel? Why are we doing this? It seems if we make a front panel variant we could get around these issues of clearance, no?

@dhslichter if memory serves it's hard to make these stack-ups work with straight SMAs (cf https://github.com/sinara-hw/sinara/issues/402#issuecomment-350362080)

However, if someone wants to do an upconverting board (or downconverting board), they might start to care about 12 GHz. If there is a way we can modify the front panel geometry/design a bit to use the 8.5 mm stackup with straight connectors and have enough clearance, that would be my preferred solution.

Sure.

Taking a step back, I think that part of the problem with Sayma has been that we've done a bunch of non-standard things to try to make it possible to cover every conceivable use-case. IMHO, it's very beneficial to stick with the standard FMC form-factor as far as possible, since it makes it easier to get things right (more prior art to work with) and we can buy front-panels etc more cheaply. In principle, it's not hard to do non-standard things, but in practice it often opens a can of worms.

So, the question is whether 12GHz on the AFE is a use-case that we really need to support. IMHO, the answer should be "no!". For the small number of users who want 12GHz, is it really so bad to build a small up-converting board, rather than doing the up converting directly on Sayma? In any case, that could be preferable since the upconverting board could be right by their trap to minimise the length of cables carrying very high frequency signals...

[hartytp]

Edit: oops clicked on post too early.

Anyway tl;dr: let's focus on building something that works really well, and not try to make something that does every possible function at once. Upconversion can be done using BaseMod to feed an external IQ mixer.

[dhslichter]

I am fine with RA SMA then if it fits in, with whatever FMC stackup height works best.

[hartytp]

Cool, so I think we all agree that:

• The top priority is to get something that works well mechanically: no marginal clearances of odd tricks, please!
• Sticking as close to standard FMC as possible is preferred (standard stack-ups, COTS panels, etc) to doing non-standard things
• Straight SMAs are preferred to vertical ones where possible

With that in mind, I vote that we let @gkasprow pick whichever solution he thinks is best.

[gkasprow]

Finally I'm designing the FMC-like mezzanine for Sayma RTM. I can fit 4 SMA + 4 MMCX on the panel with following arrangement:

I want to do two things with these test boards - make sure that it mechanically fits and that from RF point of view it works. So far I implemented:

• 4 signal paths consisting of SMA -> balun -> searay connector -> balun -> SMA. The last two on the base board.
• 4 signal paths consisting of MMCX -> balun -> searay connector -> balun -> MMCX. The last two on the base board.
• 1 path on mezzanine: SMA-> balun -> diff pair -> balun -> SMA to test the balun and PCB traces
• 1 path on mezzanine SMA-> trace -> SMA for calibration
• I2C tester circuit. It routes all control lines to the IO expander so can be used as a quick tester for Sayma RTM
• resistors simulating load.

In this way with single design we can use it for both connector evaluation and as a test suite for Sayma RTM.

[hartytp]

Looks good.

If there is room, can we have some leds? I've found the ones on Urukul quite useful.

Id we could make the connector pitch constant it might be a bit more clear for the users how the smas and mmcxs relate to each other. But if there isn't room then the current configuration looks good.

Otherwise all looks good to me!

[hartytp]

What about some traces for the digital lines to check diigtal to anlog cross talk? Digital lines will be in a separate part of the connector. For single ended io like ttls we should consider using cmcs to reduce emi issues

[gkasprow]

@hartytp look here:

[gkasprow]

I can add LEDs, but lightpipes are needed (0.5$ each). I'll see what I can fit.

[dhslichter]

I like it @gkasprow, makes it easier to identify the channels if they are evenly spaced out likes this. Is there still enough room for silkscreen labeling that is legible if we add LEDs + light pipes?

@hartytp what are you hoping the LEDs would show? If we have status LEDs, I think it might be easier to have them live on the main Sayma RTM card, in the space in between the AFE front panels.

[gkasprow]

we can place standard THT LEDs and bend prior assembly. I did not manage to find suitable light pipes. Something like this.

Sure, we can add LEDs below but need to reserve some IO lines routed to the AFE boards. There are no spare FPGA lines for that. I2C or SPI extender for LEDs need firmware to be written, so maybe let's stick with bi-colour LEDs on the AFE.

[hartytp]

@hartytp look here:

@gkasprow looks good! So long as those MMCXs won't interfere with getting an SMA spanner in there, it looks great!

@hartytp what are you hoping the LEDs would show?

I quite like that Urukul displays the RF switch state via a front panel LED. I've found that useful a few times during debugging. Similarly, general purpose FP LEDs are often useful for debugging. But, it's definitely a nice-to-have and non critical. If it causes cost/mechanical issues then it can be scrapped.

If we have status LEDs, I think it might be easier to have them live on the main Sayma RTM card, in the space in between the AFE front panels.

I think that having some more LEDs on Sayma RTM/AMC would also be a good idea. e.g. for displaying PLL locked indicators.

Sure, we can add LEDs below but need to reserve some IO lines routed to the AFE boards. There are no spare FPGA lines for that. I2C or SPI extender for LEDs need firmware to be written, so maybe let's stick with bi-colour LEDs on the AFE.

I think there will be a few spare lines on the RTM FPGA once we remove the clock mezzanine. Although, I'd like to route as much IO as possible to the AFE mezzanines for future proofing.

[gkasprow]

3mm LEDs are too massive. I found 2mm ones that fit nicely. Look at this

@hartytp how would you like to use CMCS for LVCMOS lines?

[hartytp]

@hartytp how would you like to use CMCS for LVCMOS lines?

If you think it's redundant then don't bother. Was assuming you'd use a CMC on each end of the FMC con to convert them to differential signals (the n-signal input/output ties to ground at either end).

[gkasprow]

I'm going to connect 32 IO lines to each FMC. That would be a lot of CMCs :) I located all IO lines on the FMC connector edge, far away from signal lines.

[hartytp]

ack.

FWIW, most of those IO will be used for LVDS, so CMCs not needed anyway.

[dhslichter]

Was assuming you'd use a CMC on each end of the FMC con to convert them to differential signals (the n-signal input/output ties to ground at either end).

This just seems like a rather crazy way to do things. If you truly, desperately care about this, then running lower voltage single ended logic, or better yet, running it as LVDS, seems to be the proper way to go. I point out that there is also still already plenty of opportunity for differential-to-differential crosstalk when traces aren't that far apart -- as seen in the Samtec SI test reports.

AFAICT we will have a bunch of LVDS already for the ADC data and clock lines, which would be by far the most active digital signals, and then the single-ended logic is either I2C (hell, you could lowpass-filter that to slow the edges and reduce crosstalk) and then a few GPIO lines for controlling switches, digital attenuators, and so forth that will only be active on a relatively low duty cycle.

I located all IO lines on the FMC connector edge, far away from signal lines.

This sounds like the way to go. And once we have the test boards we can quantify the level of crosstalk between such single-ended digital traces and the differential analog signals.

@gkasprow where will silkscreening go to indicate channel numbers? Can that fit on the main front panel, around the cutout for the AFE front panel? It's important to save room for that.

[gkasprow]

@dhslichter We will fit channel numbering easily, this is similar case:

[dhslichter]

link broken @gkasprow

[gkasprow]

https://www.ohwr.org/projects/afc/wiki/wiki

[gkasprow]

I'm continuing working on test boards. I want to make 2 boards - more or less final FMC-like, let's call it TestMod2 and simple carrier board with baluns and connectors. Here is how it's going to look like:

The schematics are here: TestMod_Base.PDF

TestMod2.PDF

The open questions/comments are:

• shall I add some LVDS traces with headers?
• I will use different matching and routing scenarios in each DAC and ADC channel to test them all at once.
• in one DAC channel I will terminate surrounding SEARAY connector pins to GND via 50R resistors as in Samtec measurements.

[hartytp]

shall I add some LVDS traces with headers?

That sounds like a good idea! Maybe 16 LVDS lines (we probably won't need that many in practice, but it's a good number to test with).

in one DAC channel I will terminate surrounding SEARAY connector pins to GND via 50R resistors as in Samtec measurements.

Sounds good. Let's try to space them as far apart as possible to minimize the cross-talk. Maybe also try different ways of routing the differential pairs to minimize radiation?

Other than that, looks good to me!

[hartytp]

Oh, andI think testmod2 didn't upload properly, can you fix the link, please?

[hartytp]

What are the 50R resistors for on TestMod_base?

[gkasprow]

These are the ones that surround the DAC RF signal pads.

[gkasprow]

I updated the link

[gkasprow]

To test boards with real LVDS signals I used EEM connectors. One can connect i.e. Sampler via stacked boards and pass LVDS signals and then measure how much they interfere with DAC and ADC channels

[gkasprow]

@hartytp please have a look at the board I designed: TestMod2.PDF

[hartytp]

• [x] screw holes for mounting the FmcAdc to the carrier board. looks like a copy paste error
• why are there unconnected lines on the FMC? Shouldn't they be grounded?
• remind me what the 50R terminated lines are for?
• please zero-index everything
• Are ADC_CHx the "calibration" ADC? If so, can we call them 'CAL_ADC_CHx' or similar to avoid confusion with the main ADC.
• are you happy that you've spaced the DAC channels as far apart from each other, and as far from the digital signals as possible? And that there is as much ground around them as possible? It looks like some DAC channels have non-grounded adjacent pins on the FMC, is that okay?
• There are a few different ways of routing differential pairs through the FMC. The cross-talk properties are slightly different. Are you happy that the way you've done the routing is the right way? e.g. would it reduce cross-talk if you rotated adjacent DAC channels by 90deg on the FMC?
• Did you simulate what the cross-talk between those 100ohm differential traces will be? They are quite long, so we should make sure that we are limited by the FMC, not by the traces (maybe route them as stripling on internal layers to reduce cross-talk?)

@dhslichter any comments?

[gkasprow]

screw holes for mounting the FmcAdc to the carrier board. looks like a copy paste error

there are 2 holes close to the FMC connector and 2 at the panel

why are there unconnected lines on the FMC? Shouldn't they be grounded?

I'd like to reserve a few of them for future needs. But the ones close to the DAC signals need to be grounded

remind me what the 50R terminated lines are for?

they were used in SAMTEC measurements to improve crosstalk. They seem to absorb the RF that leaks from one channel to another.

please zero-index everything

OK

Are ADC_CHx the "calibration" ADC? If so, can we call them 'CAL_ADC_CHx' or similar to avoid confusion with the main ADC.

right

are you happy that you've spaced the DAC channels as far apart from each other, and as far from the digital signals as possible? And that there is as much ground around them as possible? It looks like some DAC channels have non-grounded adjacent pins on the FMC, is that okay?

right, I will ground them.

There are a few different ways of routing differential pairs through the FMC. The cross-talk properties are slightly different. Are you happy that the way you've done the routing is the right way? e.g. would it reduce cross-talk if you rotated adjacent DAC channels by 90deg on the FMC?

that was done on purpose to test different scenarios in different channels. That's why on some channels I have 50R resistors and on another not. According to the measurements done by Samtec , the difference is subtle and will cause diff line imbalance if we rotate the pins

Did you simulate what the cross-talk between those 100ohm differential traces will be? They are quite long, so we should make sure that we are limited by the FMC, not by the traces (maybe route them as stripling on internal layers to reduce cross-talk?)

I want to keep the routing on outer layers because such we will do in mezzanines, especially when using Rogers material. I can make another board in same batch with very simple assignment just to measure the crosstalk. I did not simulate it yet but will do.

[hartytp]

there are 2 holes close to the FMC connector and 2 at the panel

I just meant that the comment refers to FmcADC so I wondered if this is copied from an FMC ADC board?

I'd like to reserve a few of them for future needs. But the ones close to the DAC signals need to be grounded

:+1:

they were used in SAMTEC measurements to improve crosstalk. They seem to absorb the RF that leaks from one channel to another.

Okay, makes sense. Sounds like a good idea then.

I can make another board in same batch with very simple assignment just to measure the crosstalk. I did not simulate it yet but will do.

It would be good to have a measurement with a short trace so we can tell the difference between PCB cross-talk and connector cross-talk (no point worrying about small connector cross-talk if it's going to be much worse on the PCB or the DAC itself).

[dhslichter]

I agree with @hartytp that it would be nice to have one test channel with a short PCB trace (to distinguish PCB crosstalk from FMC connector crosstalk). Other thoughts:

• Don't we want to use baluns of the kind we are currently using on the AFE boards, at least for some of the channels? I'd try to duplicate as much of the existing AFE parts as possible. I know there is a 50 vs 100 ohm issue but I have found that generally the components are sufficiently non-ideal (both DAC outputs and baluns) that you're sometimes better hand-optimizing than trusting a particular spec for nominal impedance. In any event, trying out different baluns would be worthwhile because I expect they will have different common-mode rejection levels (and thus different crosstalk would be observed).

• ground f10, f11, f13, g10, g13, h11, h12. This way you can have DAC3 be an aggressor for DAC2 or DAC4 and test whether the 50 ohms helps or hurts (I am guessing it hurts).

• as stated above, ground all other pins near the DAC and ADC signals -- if you want to keep some spare ones open, use pins that are far away (for example in A).

[gkasprow]

@dhslichter on BaseMod we are using MINI-CIRCUITS_TCM2-43X+ I'm using the same baluns on my test board. I forgot to add 0.5pF capacitors

[gkasprow]

I completed PCBs for SAEM/SAEF connectors testing. Unfortunately the ANSYS license server died and the administrator is on holidays. Once I get access to the tools, will do simulations.

Baseboard:

Test board:

And the AFE board:

Documentation: TestMod_FMC_Baseboard.PDF TestMod_FMC.PDF TestFMC_con.PDF

[hartytp]

Nice!

[jbqubit]

Looks good! Any luck with access to simulation?

[gkasprow]

I have all ANSYS suite working. I already did some simulations but for some reason it does not see ports in S-models I got from Samtec. Then I upgraded to newer version, the ports are visible but it crashes when I start simulation. work in progress.

[hartytp]

Responding to https://github.com/sinara-hw/sinara/issues/595#issuecomment-426428759

Great! What tests do we want to do on these boards. I can think of the following:

1. Check that we're happy with the mechanics. @gkasprow do you have an FMC front panel we can try fitting them to. Might also be worth checking that the boards are okay to insert and remove from some carrier with a panel fitter
2. Check S-parameters for microwave lines (S11 and x-talk). Just a quick VNA measurement
3. Check cross-talk between LVDS and RF lines
4. anything else (@dhslichter)? Let's not spend ages overdoing this

@gkasprow how do we want to divide the work?

I can't do (1) since I don't have any suitable board to test fit them to. If you're happy with them from a mechanical stand point then it's good enough for me.

I'm happy to do (2) although it's a quick measurement so probably as quick for you to do it as for you to post it to me. I don't mind either way, just let me know what you want

(3) how do you want to do this? Can we plug the LVDS header on these AFE cards into a BNC DIO card and then drive the BNC at 100MHz and use that to create a signal on the LVDS lines. Then measure the pick up on the RF lines. Again, I am happy to do that if it's easier since it's a quick measurement.

[gkasprow]

I want to ship you one set and keep another. My VNA in the lab works up to 3GHz. To get better one will need some time. I will 3D-print FMC panel to make sure all fits nicely. LVDS crosstalk can be measured using Kasli - I installed EEM connector to inject LVDS and SMA connector to inject single ended signals

[hartytp]

I want to ship you one set and keep another

Sounds good.

My VNA in the lab works up to 3GHz.

Likewise. Mine only goes up to 3 and a bit. I can borrow one, but I'd need to arrange that.

In any case, I don't think this is an issue, measurements up to 3GHz will be fine.

I will 3D-print FMC panel to make sure all fits nicely.

:+1: good idea!

LVDS crosstalk can be measured using Kasli - I installed EEM connector to inject LVDS and SMA connector to inject single ended signals

Great! Can you make that measurement, or do you want me to? The measurement should just be:

• drive the LVDS lines with 100MHz, measure the pick up on a few RF lines with all RF lines 50Ohm terminated

[gkasprow]

I think I can do all measurements. That won't take long. I'd need Kasli firmware that makes noise on LVDS :)

[hartytp]

I think I can do all measurements. That won't take long. I'd need Kasli firmware that makes noise on LVDS :)

Great.

I don't think we need "noise", since measuring with a square wave at a fixed frequency will be enough to tell us the cross-talk. We can infer what the pickup for genuine noise would be from that.

I can see two easy options.

1. We don't use Kasli, but use the BNC_DIO EEM as a TTL to LVDS converter. Then we hook up a singal generator running at 100MHz to the BNC inputs and use that to drive the LVDS lines
2. Either I or @marmeladapk can prepare you a Kasli bitstream which drives all LVDS signals on one EEM from an ARTIQ ClockGen https://github.com/m-labs/artiq/blob/d0ee2c2955e23c835e122b04b915e03c3cecfa46/artiq/gateware/rtio/phy/ttl_simple.py#L135 These are nice configurable clock generators, so you can run experiments that generate signals at whatever frequency you want and reprogram them without rebuilding the bitstream

[gkasprow]

Oh, forgotten about the DIO. Sure, can do that. And they have much higher current LVDS and Kasli does.

[dhslichter]

My VNA in the lab works up to 3GHz.

Likewise. Mine only goes up to 3 and a bit. I can borrow one, but I'd need to arrange that.

Do we really need/want to go higher than that? @hartytp are you hoping to put direct Ca+ microwaves through this thing? I can get access to high-performance VNAs out to 10-20 GHz for some quick measurements if desired.

[gkasprow]

@hartytp I received the boards, designed front panels and 3d-printed them. Here are the results. So far it fits nicely.

[hartytp]

Looks beautiful!

If the s params checks out we're onto a winner.

[gkasprow]

Crosstalk betwen neighbouring adc and dac channels

[gkasprow]

The setup