sinara-hw / sinara

Sayma AMC/RTM issue tracker
Other
42 stars 7 forks source link

analog mezzanine physical/interface specification #8

Closed jordens closed 7 years ago

jordens commented 7 years ago

Here is the final summary, based on comments in the thread below. This post will be sticky at the top, containing the most current specification. Items labeled ACTION: are not finalized yet; @dhslichter will edit this comment with the correct information as it is finalized.

Mezzanine geometry/stackup

image

hartytp commented 7 years ago

@dhslichter @gkasprow I've attached a sketch showing the way I assumed we'd generate + distribute the DAC clock + LO. I've missed some of the recent discussions, so apologies if this is hopelessly out of data/no longer applicable!

Comments:

1.I'd prefer not to generate the LO on the main clock mezzanine: since the range of DAC clocks we need is fairly narrow (say, 500MHz - 2.4GHz), we can make a single DAC clock PLL design, with VCO + loop-filer population options to set the frequency. In contrast, the range of LOs spans ~1GHz to >10GHz. The PLL one uses to generate a 12GHz LO for Ytterbium is likely to look very different to, say, a 3GHz LO for calcium. e.g. the Ytterbium PLL might be a 2-loop design, with a 100MHz=>1GHz loop, followed by a 1GHz=>12GHz loop, while the calcium can probably be done as a population option for the DAC clock board. Separating these jobs onto different mezzanines allows one to use the same DAC clock board, regardless of what LO one needs, which seems nice.

2.We should route the 100MHz reference clock to the analog mezzanine: for LO frequencies up to a few GHz, one can buy IQ modulators with integrated PLL + VCO to generate their own LO from the 100MHz reference clock (e.g. HMC1197). These provide a very convenient, low-noise method of upconversion.

3.We should have the option of installing a LO generation mezzanine (separate to the DAC clock mezzanine): one can put an ultra-low noise PLL on this board to generate the LO from the reference clock. If this is done on the output mezzanines then it has to be duplicated 4 times per Sayma board, which wastes space + money. This is particularly important for LO frequencies above a few GHz, where one would want the option of a using a fancier 2-loop PLL to generate the LO. (note also that the HMC1197 with PLL + VCO is 7mmx7mm & $39 from Digi-Key in singles, while HMC1097 without PLL + VCO is 4mmx4mm & $18).

4.We can minimise routing complexity/SMP count by using clock fan-out + mux ICs: high-quality mux/buffers like the ADCLK948 are extremely convenient and will not degrade the phase noise/stability of the DAC clock or LO.

5.It's nice to have the option of routing the 100MHz reference directly to the HMC7044: that way, we can use the HMC7044's integrated PLL + VCO to generate the DAC clock, without requiring a clock mezzanine or external clock source. This is a cheap, convenient option which will provide adequate performance for many applications, such as driving AOMs.

6.I've assumed that the 100MHz reference reaching the RTM meets the phase noise spec up to >~100kHz: if not, we'd probably want to put a 100MHz clean-up PLL on the DAC clock mezzanine, and route that to the LO mezzanine etc.

7.The clock distribution ICs will not work for 10GHz LOs: the fastest clock distribution ICs I'm aware of don't reach 10GHz. The ones with muxes are generally limited to <5GHz. My suggestion is that we should put mux + fan-out buffer ICs (e.g. ADCLK948) on the Sayma board to cover all LOs up to ~5GHz. Beyond this, I would suggest running coax from the LO Mezzanine to the analog mezzanine (that's about the only practical method at this frequency).

Thoughts?

DAC clk + LO.pdf

dhslichter commented 7 years ago

Thanks for the thoughts, Tom. I would argue that the point of the LO generation on the Sayma (versus using an external LO source piped in on SMAs to the analog mezzanines) is to allow standalone operation/reduce clutter for low-precision applications. No matter how careful we are, I think an onboard LO generation system will never be as high performance as one would desire for e.g. high fidelity operations on hyperfine clock qubits, and that people will probably prefer to use a phase-locked DRO or other source to provide LO to the crate for those sorts of more demanding applications.

With that in mind, simply providing 100 MHz to the analog mezzanines and leaving it up to the mezzanine designer to implement their desired LO on the mezzanine frees us from a great deal of work and hassle on the Sayma, and allows the mezzanine designer to set things up in just their desired manner. Remember there is no restriction on double/triple/quad-width analog mezzanines, so for example it would be possible to design a larger mezzanine card with a multi-stage ultra-low-noise PLL system for ultra-low-noise >10 GHz LO generation (if that is desired by the user), which could be distributed to multiple I/Q modulators on the analog mezzanine card itself. If mezzanine real estate is an issue, one could reduce the channel count slightly to free room.

When talking about clock routing, I was thinking about an application with a really simple mezzanine containing an HMC1197 or ADF5355 + IQ modulator, which would need a 100 MHz reference and could then provide reasonable-quality agile microwave signals for low-precision or medium-precision applications.

Basically what I had thought about would be sending a cleaned 100 MHz LVPECL clock (or equivalent) to each of the mezzanine boards via an additional SMP pair. This would then let the mezzanine designer make whatever necessary design choices to achieve the required LO performance.

@hartytp @jbqubit @gkasprow @jordens thoughts/comments?

hartytp commented 7 years ago

@dhslichter I agree that we should route the 100MHz analog clock to the analog mezzanines.

You're right that one if one wants to generate the LO on the Sayma board, then the PLL could go on the (possibly quad-width) analog mezzanines(s). However, for the sake of modularity/reusability of hardware, I'd still rather have the PLL on a separate mezzanine. That way, for example, we could have two different versions of the LO PLL mezzanine -- a fancy one and a cheap one (e.g. your ADF5355) -- both of which could be used with the same (up-conversion) analog mezzanines. If we put the PLL/synth on the analog mezzanine, then we'd have to remake a lot more hardware if we ever wanted to alter the PLL.

In general, my philosophy is that I'd prefer to have more mezzanines per Sayma board, but with fewer varieties of each mezzanine to design/test/document/stock/maintain.

I do agree that we should have the option of generating the LO externally. My preferred solution for that would be to have a "passive" version of the LO mezzanine, which takes an external LO from a front-panel SMA, and distributes it to the analog mezzanines (via LVPECL buffers/multiplexers for LOs below ~5GHz, as in my diagram, and via a passive splitter and internal SMAs for higher frequencies). This is essentially the same approach we're taking for the DAC clock. Again, this approach allows us to use the same analog mezzanine (IQ modulators + output stage) boards, regardless of whether the LO is generated internally or externally.

FWIW, having run the numbers for gate error versus phase noise, and looked at the phase noise of available microwave VOCs, I'm pretty optimistic that we can make a LO mezzanine with good enough phase noise for even our highest fidelity gates on 43Ca+ hyperfine clock qubits. Obviously, it gets a lot harder for higher frequency qubits.

jordens commented 7 years ago
dhslichter commented 7 years ago

I agree with @jordens here; adding an LO mezzanine, along with LO distribution/fanout, makes the Sayma cards much more complicated and requires us to overengineer at least some parts of the LO distribution system for future-proofing (which we still might screw up). While the modularity argument is nice for motivating an LO mezzanine, I think it's better left to people designing the analog mezzanine cards, with the current front panel SMAs used for providing an external LO for simple upconverting mezzanines, which will work for most simple cases in the near term while people sort out what exactly they might want an analog mezzanine with integrated LO to do.

hartytp commented 7 years ago

@jordens My point was really "we should have a way of bypassing the DAC clock mezzanine and feeding the 100MHz reference directly into the HMC7044". I'm not trying to prescribe exactly how this muxing should be done, and have no objection to using a max inside the HMC7044.

hartytp commented 7 years ago

@jordens @dhslichter My concerns with putting the LO on the analog mezzanine are that

  1. I don't want to have to re-make my carefully-designed LO PLL each time I want to make a slight change to a part of my input/output stage, such as a noise eater, (or, have to re-make the input/output stage if I want to alter my PLL).
  2. While the LO PLL is quite a specialist circuit, which needs to be designed with a great deal of care, the input/output stages are generally quite simple as RF circuits go (a few mixers/amps). It makes sense to keep the PLLs in their own module, so that once they've been designed and tested, people can work on the input/output boards without having to know anything about where their LOs come from.

adding an LO mezzanine, along with LO distribution/fanout, makes the Sayma cards much more complicated and requires us to overengineer at least some parts of the LO distribution system for future-proofing (which we still might screw up)

I don't think the additional complexity is that bad: we're only talking about going from 5 to 6 mezzanines, and adding one extra fanout/mux IC for LO distribution. Still, I agree that if there isn't room for an extra LO mezzanine, then it's not an option.

I would either just leave the LO generation to the ADC/DAC analog mezzanine(s) and only feed them 100 MHz or expect the sampling PLL mezzanine to do it.

If we don't have a separate LO mezzanine, then putting the LO on the DAC clock/sampling PLL mezzanine would be my preferred option for LOs at <~3GHz. We'd still want a mux to allow the option of feeding the 100MHz to the analog mezzanines, but we can incorporate that into the mezzanine if you'd prefer. We'd also need to make sure that the clock mezzanine is large enough to accommodate both PLLs.

gkasprow commented 7 years ago

Take into account that some day we may want to use RF backplane which can not only distribute 100MHz but also 2 RF signals in GHz range. That's why I wanted to place the LO generation in some other place. But in such case we may always re-engineer the Sayma RTM to fit that purpose. In case of 100MHz mux, in order to not deteriorate the signal, we can use just capacitor or resistor jumpers.

hartytp commented 7 years ago

Take into account that some day we may want to use RF backplane which can not only distribute 100MHz but also 2 RF signals in GHz range. That's why I wanted to place the LO generation in some other place.

Okay, that makes sense. FWIW, I'm generally in favour of generating frequencies locally where conveniently possible for maximum flexibility. e.g. I can imagine a few cases where it would be desirable to have more than a single LO frequency in the same rack.

In case of 100MHz mux, in order to not deteriorate the signal, we can use just capacitor or resistor jumpers.

That could work, but it's much more convenient to have a software-programmable switch! For frequencies below ~5GHz, these are readily available with extremely low noise, that's unlikely to degrade even a high-quality LO. If we leave the muxing to the clock board, then users with LOs < 5GHz can build a clock board with an active switch, and users with higher-frequency LOs can go for a different approach.

gkasprow commented 7 years ago

The signal degradation caused by such switch is one thing, another is signal leaking from second, not selected input.

This can seriously affect our reference signal.

And this is a reason where we use passive jumpers to select clock source.

If you know good LVPECL or RF swtiches with high isolation, let me know, at least we will try them. In case of troubles we can always remove serial caps.

hartytp commented 7 years ago

@gkasprow True, I haven't worried too much about the isolation, as low-level 100MHz leakage won't be a problem in my use case (very easy to filter off a 3GHz LO). I have a couple of ADCLK984 ICs in stock, so I can measure this if you'd like. But, you're right, if this is a problem, one can always remove the coupling caps.

Tom

gkasprow commented 7 years ago

Do you mean ADCLK948?

We could use it to deliver clk signal to the clk mezzanine and also to all ADC/DAC mezzanines. In this way the bard could work without the clock mezzanine.

hartytp commented 7 years ago

Yes, sorry for the typo!

We could use it to deliver clk signal to the clk mezzanine and also to all ADC/DAC mezzanines. In this way the bard could work without the clock mezzanine.

I think that's roughly what I was suggesting, but I'd need to see a diagram/schematic to be sure I understand you correctly...

jbqubit commented 7 years ago

From latest RTM_DAC.step posted in #7 it looks like there is insufficient clearance between the SMA. The most compact SMA tool I've found is the following which has tip diameter 0.5".

http://www.minicircuits.com/pdfs/HT-4-SMA.pdf

sma_tool_tip

See drawing. Can the upper row of SMA be shifted up?

dhslichter commented 7 years ago

Agreed here to some degree -- it seems we need a keepout with at least enough room to allow SMA nuts not to interfere with each other as they are rotated (looks in this design like it's not possible). By spacing them out a bit more horizontally we could get more clearance.

gkasprow commented 7 years ago

I modified the mezzanine shape to have them distributed evenly.

dhslichter commented 7 years ago

What is the center-to-center spacing for adjacent SMAs in the latest design?

dtcallcock commented 7 years ago

How about just switching to smaller connectors? For the analog out (upper row) it's hard to beat the SMA performance wise without going to something obscure (SSMA!?), so I would retain them. For the analog in (lower row) connectors where we don't necessarily need the performance there are a few reasonable alternatives like MCX or MMCX (both good to 6GHz). As these connectors sit flush with the panel they will allow much easier access to the remaining SMAs on this card and adjacent cards.

gkasprow commented 7 years ago

we can go for SMBs - they don't require a wrench and are less fragile than MMCX

dhslichter commented 7 years ago

Crosstalk/leakage on SMB is pretty bad, and for the downconverting systems we may have inputs up at 6-10 GHz for superconducting qubit applications. Greg, what is the center-to-center distance on the SMAs in your latest version?

dtcallcock commented 7 years ago
gkasprow commented 7 years ago

The centre to centre distance is 10.26mm. The most right connector is clock input. smas_front

dhslichter commented 7 years ago

I think the benefits of SMA (low leakage, good high-frequency performance, good mechanical stability, many mating cycles, inexpensive connectors and cabling, high-performance cable assemblies available stock, etc) are substantial enough that we should try to stick with SMA if at all possible.

The absolute minimum (no interference) clearance between neighboring SMAs, center to center, is 9.2 mm -- this design will give 1 mm minimum clearance between SMAs as they rotate. When the flats are facing each other, there is 2.3 mm between flats (maximum clearance, as shown in picture). This is enough to get one of those mini-circuits tight-clearance wrenches in, especially if you file off the pointy tips a short distance. In any event, it is possible to fully connect and disconnect the panel with a stock wrench if one of the two neighboring SMAs is not yet connected....you can connect/disconnect everything starting at one end.

For those groups where disconnecting and reconnecting often is in the plan, I would suggest making an external SMA patch panel elsewhere, and the cables from the RTM can be broken out there where access is easier.

I found a different right-angle SMA part that would move the bottom row down another 0.2 mm, if that's worth the extra space: http://www.digikey.com/product-detail/en/linx-technologies-inc/CONSMA002-L/CONSMA002-L-ND/1577206

If it is possible to move the clock connector ever so slightly farther to the side, to increase its spacing with the next connector in, that would be great.

dhslichter commented 7 years ago

That said if it is easy to change panels we could consider going with SMB for the analog inputs, especially for the baseband mezzanine cards, where the frequencies will be lower. Up/downconverting cards would be good to have SMA, but then we could make double-width or quad-width mezzanines with a single LO input (to be shared), which would mean fewer SMAs and thus we could space them out a bit more.

gkasprow commented 7 years ago

The maximum I can do is to shift the clock SMA in horizontal plane by 0.25mm

gkasprow commented 7 years ago

In worst case we can place some 0.5mm distance sheet between SMA THT connector and the mezzanine. This would make assembly a little more difficult but move the SMA row down by 0.5mm.

dhslichter commented 7 years ago

I think we can consider this sheet spacer solution if needed -- it can be implemented late in the process since all it requires is a front panel design change. @dtcallcock opinions?

gkasprow commented 7 years ago

The front panel is not designed yet, so we can implement this idea from very beginning.

To make life easier, I can use laser-cut sheet of thick and rigid cardboard that would be inserted during soldering and later on removed. Anyway, during connector assembly process, we will make some tool to hold them in right position. We can design it in such way that SMA is already elevated.

dhslichter commented 7 years ago

Sounds good to me -- we should probably do some tests (S parameters/TDR) to check whether this creates a problematic impedance discontinuity to have the right-angle connector substantially offset vertically.

gkasprow commented 7 years ago

That's easy to check - one can compare two mezzanines with RA SMAs mounted in standard and elevated way. By elevating the SMA we increase the centre to centre distance to 10.56mm Theoretically we can elevate them even more, but I'm afraid of impedance mismatch. Maybe piece of 1mm FR4 with metallized via would help :)

sbouhabib commented 7 years ago

I agree this has to be checked, the impedance mismatch first, then we have an unshielded small antenna (maybe just a minor issue), the idea with some board piece could be a solution, not sure about FR4 in general, I think Rogers class laminate should be for the RF parts. technically it could cause some issues for assembly, and ground continuity/repeatability of connection

dtcallcock commented 7 years ago

Here's a somewhat more radical idea. I think that a dense field of ~200 SMAs (in a full rack) with sub-mm clearances is going to be basically unusable on a day-to-day basis so I can imagine that most people will take everything to patch panels. If that's the case, then why not use a multi-coax connector on the board and supply an SMA breakout with each board?

There are quite a few solutions out there such as the Smith MDHC. Other suppliers include Times, Carlisle and H&S.. I haven't dug into specs too much but it seems you can get beyond-SMA electrical performance.

Whilst I think this will make a much nicer final product, it will presumably add a lot of cost, increases design complexity and is more likely to be obsoleted than SMA (still going strong at 50+).

jmizrahi commented 7 years ago

@dtcallcock in general I think it's a good idea to avoid obscure, expensive connectors and cables if at all possible. I also don't really see what your solution solves -- it requires an SMA breakout panel, just as before. It's definitely more elegant, but I don't think that's a strong enough argument to move away from SMA.

I would say that so long as it is possible to connect the cables with a wrench, even if it has to be done in sequential order, then it's perfectly usable and I would stick with SMA. You wouldn't even necessarily need a patch panel -- a bunch of 100mm SMA cables could just be permanently attached to the front panel, and then all connections and disconnections happen with the 100mm cables.

ghost commented 7 years ago

I also don't really see what your solution solves -- it requires an SMA breakout panel

Probably several panels. These can then have adequate connector spacing, different connectors depending on requirements, and space for sensible cable management. External up/down-conversion or LO distribution could also be mounted here.

avoid obscure, expensive connectors

Probably no more obscure than the other connector on the mezzanine. As for cost, I'll admit I have no idea (though decent SMAs aren't exactly cheap and you have to buy the cable either way). Anyone have experience here @sbouhabib?

even if it has to be done in sequential order, then it's perfectly usable

Sure, for one or two cards this is fine but my worry is how well it scales to ~200 connections. It's going to be a pig to change cards, debug damaged cables/connectors and could easily be a rats' nest that puts a lot of mechanical force on the cards and connectors. Also a bundle of 100mm cables and i-pieces doesn't seem the best solution for crosstalk, ground loops and maintaining the phase and amplitude stability of microwave signals. I'm sure each lab will come up with a reasonable coping strategy but I thought this might be a good juncture to engineer a nice solution.

It seems like the consensus is that I'm overstating the problem though so I'm happy to drop this.

dhslichter commented 7 years ago

Two different github accounts @dtcallcock @allcockd ? Fancy pants!

The other mezzanine connectors are actually standard SMP (~$6 each), plus the power/io pin connector (also cheap). The spec'ed SMA connectors are ~$20-$25 each. My guess about these sorts of integrated connectors, based on what gets charged for standard Micro-D and other fancy space-hardware stuff, is that they'd be several hundred dollars apiece at a minimum. Factor in that you are buying a mating pair as well, with one end integrated into a cable assembly that likely can only be manufactured by the company that makes the fancy connector. $$$$$$$ :D

That said, I don't think we should be ready to drop this right away. We do need to consider that either way (SMA or multi-connector) one is going to want a sturdily mounted SMA breakout panel, and this panel can be used for routing externally generated LO and mounting the required components, among other tasks. What happens to the godawful ratsnest of cables that would emerge from a fully populated crate is going to have to get figured out by each lab independently -- connector changes have no effect there one way or the other.

dhslichter commented 7 years ago

Also would need the shields of the coax cables not to be grounded together or to the panel if we care about ground loops a la @hartytp

dtcallcock commented 7 years ago

plus the power/io pin connector (also cheap)

I mean isn't this connector an 'obscure' Samtec interface (not that there's anything wrong with that)?

$$$$$$$

But I'd be interested to know actual cost/feasibility from someone who's done something like this.

connector changes have no effect there one way or the other.

The ratsnest can be a lot tidier if it's a smaller number of blind mate connectors and wiring harnesses and not a bunch of cables you have to be able to delve into with your hand and an SMA wrench.

What happens to the godawful ratsnest of cables that would emerge from a fully populated crate is going to have to get figured out by each lab independently

Even if we stick with SMAs it'd be good to share ideas on how to do this. Perhaps something to figure out once we've got prototypes and tried a few things out.

I have no idea who @allcockd is.

jmizrahi commented 7 years ago

@dtcallcock my thinking was along the lines of what @dhslichter said, namely that these connectors would be rather custom and quite expensive, and likely unique to whatever company they came from (that's what I meant by "obscure"). But, you make a strong argument that it would make the front panel quite a bit more manageable. Would you be willing to get some quotes so that we have concrete numbers to discuss? I also hadn't realized just how expensive those SMA connectors are -- at $25 a piece, then a $200 8x connector would be a wash, at least as far as direct connector cost is concerned.

sbourdeauducq commented 7 years ago

I also like Smith MDHC. Quote requested.

gkasprow commented 7 years ago

Samer, you shown us during last meeting at WUT some DSUB-like multipin RF connectors.

Do you remember their PN?

gkasprow commented 7 years ago

These are the fbm connectors.but after 2ghz they become quite "hard" to match properly.theoretically doable but hard and in general narrowbanded

Samer Bou Habib

20.10.2016 10:56 "Grzegorz Kasprowicz" G.Kasprowicz@elka.pw.edu.pl napisał(a):

Samer, you shown us during last meeting at WUT some DSUB-like multipin RF connectors.

Do you remember their PN?

dhslichter commented 7 years ago

It looks like a 4x connector (single-width mezzanine) is 35.6 mm wide, so I don't know if we can really put them all next to each other (mezzanines are 32 mm wide now, not sure what the gap is).

Thus if we go Smith MDHC, we should use double-width mezzanine cards with 8x connectors (53.4 mm width, easily fits). Thus there would be 2 connectors per RTM, covering all 16 DAC/ADC lines.

dhslichter commented 7 years ago

MDHC has -85 to -95 dB crosstalk between connectors, return loss better than -20 dB to 13 GHz (more like -30dB at 3GHz and below)....I'd say performance-wise it's definitely suitable.

dtcallcock commented 7 years ago

I've also asked for quotes on:

Carlisle Core HC Carlisle Core GD Rosenberger Multiport Minicoax The latter system has a few parts on Mouser to give you an idea: link

These might be a bit too fine pitch and fragile but see what you think.

dhslichter commented 7 years ago

Any love on these quotes @dtcallcock @sbourdeauducq ? What pricing/lead times are being given?

sbourdeauducq commented 7 years ago

No reply. BTW we should break down this issue into smaller ones.

dtcallcock commented 7 years ago

I got the Rosenberger quote. Still working on Carlisle.

Rosenberger 23C21E-40ML5 4ch solder pin right angle Qty 100-250 $56.82ea, 5-7wk lead time Datasheet

Rosenberger L99-816-30E Cable set 4ch, 30cm, SMA M termination Qty 100-250 $157.01, 13-15wk lead time Datasheet

As I said, these things might be too compact.

jbqubit commented 7 years ago

@dtcallcock Thank you for getting these quotes. Price is good economy vs individual SMA. @sbourdeauducq Please break this issue into smaller ones.

dhslichter commented 7 years ago

@jbqubit @sbourdeauducq I will break this down into smaller issues.

gkasprow commented 7 years ago

http://uk.rs-online.com/web/p/sma-connectors/7123264/ http://uk.rs-online.com/web/p/sma-connectors/7123254/