sinara-hw / sinara

Sayma AMC/RTM issue tracker
Other
42 stars 7 forks source link

Sayma clock distribution #40

Closed hartytp closed 7 years ago

hartytp commented 7 years ago

@jbqubit @gkasprow Changes to Sayma RTM schematic agreed in discussion today:

  1. Replace HMC7044 with HMC7043, since PLL in HMC7044 cannot be disabled
  2. We will produce a version of the clock mezzanine with a low-cost broad-band PLL on to replace the PLL in the HMC7044
  3. Clock mezzanine board should have 2 x differential (3V3 PECL) RF outputs: DAC_CLK and REF/LO. DAC_CLK is the DAC clock, REF/LO is either the 100MHz reference clock, or a local oscillator generated by the clock mezzanine
  4. DAC_CLK routes directly to HMC7043 input.
  5. REF/LO routes to inputs on the analog mezzanines via suitable fanout buffer
  6. DAC_CLK input (SMA) on front panel routes to clock mezzanine board via internal coax (SMA)
  7. Clock mezzanine with on-board PLLs should also have a mux to allow software switching between internally generated and externally supplied DAC_CLK sources
  8. 100MHz clock will be distributed over the backplane as a PECL square wave. The clock module should have a suitable PECL multi-channel output buffer (chose one with good propagation delay temperature stability and good phase noise). Sayma RTM will have a PECL buffer to receive this signal.
  9. If we decide not to distribute the 100MHz reference clock using the differential clock lines on the RF backplane, we will distribute via a single-ended line using baluns on input and output
  10. Remove Si53307 & EXT_100M_RF_In from the RTM schematic
jordens commented 7 years ago

re 1. Pretty sure it can be disabled. Use CLKIN1/FIN. AFAICT, the lower half of the HMC7044 (figure 30, after the VCO MUX) is exactly the HMC7043 (figure 23).

dhslichter commented 7 years ago

It does appear that you can use FIN/CLOCK1 as an external input and bypass the PLLs. You can also use a reference at CLOCK1 with PLL2 only (not PLL1) if use of the onboard PLL is desired. However, is there an advantage to doing things this way rather than simply putting the HMC7043 on the Sayma and allowing all flexibility in frequency generation (i.e. extra PLLs) to occur on the clock mezzanine?

jordens commented 7 years ago

It would decouple items and provide fallback options. It's a very convenient and simple alternative way to generate a reasonably (very reasonably) good sample clock from the 100 MHz. This would allow good quality operation without the clock/pll mezzanine. That's a "risk mitigation" and "cost" issue.

dhslichter commented 7 years ago

I think this comes down to philosophical notions of where you draw the lines -- do you require some sort of clock mezzanine board (even just a very simple one), or do you want to be able to run without one entirely? Seems to me that our discussions have tended toward the former, with the idea that we make some simple boards to take care of the low-performance-requirement use cases.

That said, using the HMC7044 seems reasonable to me. We would then route 100 MHz from the backplane to CLKIN0, allowing use of the PLLs without a clock mezzanine, and the output of the clock mezzanine is routed to CLKIN1, where it can either be just buffered to the outputs or used as a reference for PLL2. Note that this will require including a 1:2 fanout on the LVPECL 100 MHz clock from the RF backplane, with one output going to the clock mezzanine and the other to the HMC7044 CLKIN0. @hartytp thoughts?

hartytp commented 7 years ago

@dhslichter Thanks for checking up on the HMC7044 properly! So long as the PLLs can be disabled, I have no objection to using a HMC7044.

While I don't feel particularly strongly about this, I'd tend to agree with @jordens: using the HMC7044 is nice, because it gives us a minimal cost/complexity option of running without the clock mezzanine, which will provide adequate performance for many use cases. I think the extra complexity/cost of using the HMC7044 is pretty minimal. But, I'm not overly fussed either way.

dhslichter commented 7 years ago

OK, so consensus seems to be to keep HMC7044 on Sayma. We will need a mux/splitter to route 100 MHz from the RF backplane to CLKIN0 as well as to the inputs of the clock mezzanine.

@gkasprow @jbqubit @hartytp here is the now updated list of tasks for clock distribution, based on first item in this list:

  1. Keep HMC7044 on Sayma. Output of clock mezzanine should be routed to CLKIN1/FIN. 100 MHz clock from RF backplane should be routed to CLKIN0.
  2. A mux or 1:2 buffer needs to be added on the 100 MHz LVPECL signal coming from the backplane to allow it to be sent to HMC7044 CLKIN0 and/or clock mezzanine reference input.
  3. Clock mezzanine board should have 2 x differential (3V3 PECL) RF outputs: DAC_CLK and REF/LO. DAC_CLK is the DAC clock, REF/LO is either the 100MHz reference clock, or a local oscillator generated by the clock mezzanine
  4. DAC_CLK routes directly to HMC7044 CLKIN1/FIN input.
  5. REF/LO routes to inputs on the analog mezzanines via suitable fanout buffer - @sbouhabib suggestions?
  6. DAC_CLK input (SMA) on front panel routes to clock mezzanine board via internal coax (SMA bulkhead on front panel, TBD on clock mezzanine board)
  7. Clock mezzanine with on-board PLLs should also have a mux to allow software switching between internally generated and externally supplied DAC_CLK sources
  8. 100MHz clock will be distributed over the backplane as a PECL square wave. The clock module should have a suitable PECL multi-channel output buffer (chose one with good propagation delay temperature stability and good phase noise). Sayma RTM will have a PECL buffer to receive this signal. This signals needs to be muxed or distributed both to HMC7044 and clock mezzanine.
  9. If we decide not to distribute the 100MHz reference clock using the differential clock lines on the RF backplane, we will distribute via a single-ended line using baluns on input and output
  10. Remove Si53307 & EXT_100M_RF_In from the RTM schematic
jordens commented 7 years ago

@hartytp that was me. @dhslichter I would agree that it was a merely philosophical question if there was a significant downside to using the HMC7044.

re 6.: why not just plug that pigtail it into the (then unused and free) clock mezzanine SMP connector?

hartytp commented 7 years ago

@jordens Yes, it was, sorry! Posting in a hurry and forgot who said what.

@jbqubit Any objections to using the HMC7044?

General question: who is going to take responsibility for looking at the HMC7044 data sheet + reference design carefully and thinking the design through properly? e.g. what should our default population option be for the loop filter passives? Will we be able to cover our desired range of DAC clock frequencies (800MHz - 2.4GHz) with a single loop filter, while keeping acceptable performance? This needs simulation. If this level of performance isn't achievable, we might want to consider using a HMC7043 + different synth IC...

@dhslichter from the data sheet, it looks like the HMC7044 PLL1 is designed to be a low bandwidth (<1kHz) jitter clean up loop. The IC does not provide an internal VCO or loop filter for this PLL, so we'd need an extra VCXO if we want to use it. Since the backplane clock should be pretty good, I suggest we don't bother with PLL1 in the initial design, and just use PLL2 to generate the microwave clock. AFICT this would mean connecting the 100MHz reference clock to OSCIN, rather than CLKIN0.

hartytp commented 7 years ago

@jordens Routing the external DAC clock directly to the SMP via coax is another option (one would also need to use an SMP short on the complementary input of course). However, FWIW, my feeling is that this wouldn't be optimal from a noise perspective: AFAIK the sine-square conversion step is actually one of the hardest parts of clock distribution (when I talked to the guys at SDI, they suggested that it's typical to degrade the phase noise of an ultra-low noise oscillator by 20dB during the sine-square conversion).

So, you'd want to take care with how you do this -- particularly given that, since the HMC7044 already provides a pretty good DAC clock, the only reason to use an external DAC clock would be to attempt to get the lowest possible phase noise. I think that noise-wise, one would do better by either using a balun and going into the HMC7044 differentially, or by using something like an LTC6957-1 to do the conversion. In either case, these components would go on a simple mezzanine.

Another point to consider is that the sine-square conversion is inherently non-linear and thus tends to mix any low-frequency noise onto the clock. So, if one cares about close-in noise and spurs, then breaking ground-loops around the clock input is generally crucial. Again, this is probably best done using a balun on the mezzanine.

In the end, this doesn't affect the RTM design, just how it's used, so we can worry about it later...

jbqubit commented 7 years ago

I object to the use of HMC7043. Broadly, I'm concerned about the tolerance of unnecessary complexity exemplified by the use of HMC7044 over HMC7043 on the RTM motherboard. Let's keep clock generation (PLLs, jitter cleanup, VCXO, etc) on the clock mezzanine and fanout on the RTM motherboard. We're getting feature bloat motivated by speculative cost savings.

Regarding @hartytp observation 6 hours ago on configuration of PLLs, using HMC7043 means the RTM motherboard is simpler and details like PLL component choices (and hopefully modeling) only need to be made once for the analog mezzanine.

I advocate that we should stop referring to the analog mezzanine clocks by their numerical frequencies or possible function (eg "DAC_CLK" and "REF/LO"). Consider: the clock mezzanine generates two frequencies: CMEZ_LF (nominally 100 MHz) and CMEZ_HF (nominally 20*CMEZ_LF). The clock mezzanine is fed an external clock CMEZ_REF (2 GHz or 100 MHz). Call the analog mezzanine LO AMEZ_LO. It's the responsibility of the clock mezzanine to produce CMEZ_LF and CMEZ_HF given CMEZ_REF.

The phase noise in the ADC clock limits its SNR. The use of a single clock distribution chip on the RTM motherboard implies that the ADC clock is derived from CMEZ_HF by dividing by 20 which incurs an unnecessary phase noise penalty. The ADC should have its own HMC7043 with CLKIN driven by MEZ_LF. The distributed signals would then be

The phase noise in the DAC clock limits its performance too. Use a second HMC7043 with CLKIN driven by CMEZ_HF. The distributed signals are then

Even though a different chip with less fanout would work, there's a simplicity in using HMC7043 as a standard clock fanout chip everywhere.

@hartytp (6) DAC_CLK input (SMA) on front panel routes to clock mezzanine board via internal coax (SMA)

Agreed. For low clearance Greg recommended MMCX for clock mezzanine input connector.

@hartytp (7) Clock mezzanine with on-board PLLs should also have a mux to allow software switching between internally generated and externally supplied DAC_CLK sources

Yes, clock source mux is needed on clock mezzanine. From the perspective of clock mezzanine, CMEZ_REF is obtained from a) flying MMCX or b) SMP connector.

RTM motherboard needs a clock mux as well. CMEZ_REF SMP is fed from a) XO on RTM motherboard (default), b) RTM backplane RADIALL single-ended, c) RTM backplane ERNI ErMetZD differential d) clock recovered by Sayma AMC FPGA.

There are two distinct clock mezzanine boards.

@hartytp (8) 100MHz clock will be distributed over the backplane as a PECL square wave.

Agreed.

sbourdeauducq commented 7 years ago

speculative cost savings.

The extra cost of HMC7044 vs. 7043 is very small and it allows operating a Sayma without a clock mezzanine. This does not sound "speculative" to me. The 7044 is essentially 7043+PLL, and if its PLL ends up not working correctly for some reason, we can install a clock mezzanine and use it as a 7043, so it's not risky either. If you are concerned about feature bloat, I would look elsewhere...

jordens commented 7 years ago

@hartytp ack. that external dac clock plan sounds good. and then 100 MHz from the backplane split to both the clock mezzanine and OSCIN. GHz clock from the mezzanine to CLKIN1/FIN.

hartytp commented 7 years ago

My final thoughts re HMC7044 v 7043: I agree with @jordens et al. that using an HMC7044 on the RTM is a nicer way to go. However, I also agree with @jbqubit in the sense that I don't think there's any point putting an ultra-low noise PLL on the RTM unless we're going to put real thought into it (inc details like loop filter stability + noise, power supply noise levels, whether screening from external noise sources is required, etc. etc.). I certainly haven't thought these details through for the HMC7044 (beyond spending an hour skimming the data sheet) and, judging by this thread, I don't get the impression that anyone else has either. Given that we're planning to start design review on the RTM this week, I worry that the temptation will be to rush the PLL design and end up with something useless.

So, unless anyone is volunteering to do a proper design for this, I vote to keep the PLLs on the mezzanine for this design revision (we can always move some of them onto the RTM in the next design revision).

Anyway, as I said, I don't feel strongly about this, but that's my tuppence-worth.... Executive decision someone?

dhslichter commented 7 years ago

I think the "no clock mezzanine" method of running the Sayma seems like an unnecessary complication to the Sayma. You can make an incredibly simple "dumb" mezzanine for standalone operation that takes 100 MHz from the backplane and inputs to a simple HMC830 chip, which outputs to the Sayma. Ship this board by default with each Sayma RTM and you have the capability you're wanting. Then, you can go fancier later as needed. Crucially, there is nothing that needs to be decided/engineered in terms of PLL design/performance on the Sayma RTM motherboard.

The whole point of mezzanine boards, both here and for the analog inputs/outputs, is to allow the hardware to be modular and to rework/test designs of these very critical components without having to redo the Sayma boards themselves. For this reason, I agree with @hartytp that trying to engineer a good PLL in the allotted time for the HMC7044 on the Sayma is not realistic. This sort of thing is best saved for the mezzanines.

That's my two cents. That said, if we just slap something together with an HMC7044 that allows it either to act as a dual PLL system or as a fanout, and we recognize that we may just be making a relatively worthless PLL system if we make some poor design choice in our haste, that doesn't preclude a fix using a clock mezzanine. Either way, I am OK with it.

hartytp commented 7 years ago

@jbqubit The signals I anticipated having for the clock mezzanine are (please comment/criticise):

Inputs:

  1. REF_CLK:
    • source is either RF backplane or on-RTM XO
    • connects to mezzanine via 2xSMP
    • nominally 100MHz PECL square-wave
    • used as reference for mezzanine PLLs, also provides one source for AMEZ_CLK
  2. EXT_CLK
    • from front panel SMA, routed single-ended to MMCX on mezzanine via coax
    • typically 800MHz-2GHz sine, +13dBm
    • TCM2-43X+ balun used to break ground-loops and convert to differential. Differential signal fed into LTC6957-1 for sine-square conversion (nb, as they comment in the data sheet, differential driving generally gives lower noise and, since we have a balun anyway, there's really no reason not to!)
    • Intended for use in cases when on-mezzanine PLLs aren't suitable e.g. because lower noise is needed

Outputs (all PECL square-wave, routed via 2xSMPs):

  1. MEZ_CLK
    • Routes to HMC704x on RTM to provide DAC + ADC clocks + SYS_REF signals
    • Typical frequency range 800MHz-2.4GHz
    • Mux allows this line to be connected either to an on-mezzanine low-noise PLL (referenced to REF_CLK) or to EXT_CLK.
  2. AMEZ_CLK
    • Routes to analog mezzanine boards via fanout buffer NB this signal is intended to be used for LOs, and hence phase stability is absolutely crucial. As a result, the fanout buffer must not be an HMC704x, or any other IC with an integrated phase-shifter, as this will significantly reduce phase stability
    • Mux allows this line to connect to one of: on-board PLL (e.g. to provide a <5GHz LO for mixers on analog mezzanines); or, REF_CLK (e.g. to provide 100MHz reference for PLLs on the analog mezzanines).
jordens commented 7 years ago

The independence of designing/manufacturing/purchasing/debugging another mezzanine IMHO outweigh the "trouble" of adding (1) a fan out for the 100 MHz and (2) three discretes for the loop filter. If you don't care about it, please at least do it to make life for those easier who are building, debugging, and maintaining gateware and firmware. I promise you we will use it. Also the HMC830 does not come for free: needs register maps being written, maintained, debugged, setup and debug code written, tested, debugged, the board (that achieves conceptually the same as the HMC7044 already does) also needs to be designed and built.

hartytp commented 7 years ago

@jbqubit IIUYC, the main differences between your suggestion and my previous post are that:

  1. You would like to have separate outputs on the clock mezzanine for the DAC and ADC clocks
  2. You would like the external SMA input to be used for 100MHz as well as ~2GHz

Comments:

a. With the caveat that I haven't thought this through too carefully/looked at the phase-noise plots in detail: my guess is that the excess phase noise from dividing the DAC clock down won't be a problem.

b. What's the use case for inputting 100MHz via the front-panel SMA? After all, we already have an extremely high-quality reference via the back-plane. While I don't mind having a few muxes in the clock network, I don't want to turn it into a complete rat's nest of muxes that won't ever be used!

hartytp commented 7 years ago

@jbqubit

RTM motherboard needs a clock mux as well. CMEZ_REF SMP is fed from a) XO on RTM motherboard (default), b) RTM backplane RADIALL single-ended, c) RTM backplane ERNI ErMetZD differential d) clock recovered by Sayma AMC FPGA.

That's a lot of muxes!

hartytp commented 7 years ago

@jordens

If you don't care about it, please at least do it to make life for those easier who are building, debugging, and maintaining gateware and firmware.

+1 so long as that's you volunteering to check the data sheet + reference designs carefully to make sure you're thought of all the details!

sbourdeauducq commented 7 years ago

I'm also for 7044 on the RTM card + any fancy low-noise PLL on an optional mezzanine.

+1 so long as that's you volunteering to check the data sheet + reference designs carefully to make sure you're thought of all the details!

The 7044 is barely harder than the 7043 and we need one of those anyway.

jordens commented 7 years ago

@hartytp Sure! This is how we arrived at the HMC7044 recommendation. I assume we all operate like that. ;)

gkasprow commented 7 years ago

I connected 7044 in such way that FIN can be clocked directly from the clock mezzanine, RFSYNC is routed to the RF backplane for future multi-board synchronization purposes (AFAIR @sbourdeauducq wanted this), OSCIN can be connected to 100MHz clock fanout as well as CLKIN2. Optionally, we can solder some VCXO and connect to OSCIN. The clock source for REF/LO and also for 7044 can be either SMA connector on front panel or ERNI/RADIALL from uRFB. The selection of the RF clock from uRFB will be done by capacitor assembly. So we have options:

  1. ability to directly clock 7044 in a'la 7033 mode with all PLLs disabled, driven from clock module
  2. run all board without clock module with single 100MHz input either from SMA front panel or uRFB single ended or diff signal. 7044 uses only second PLL with VCO
  3. as above but with 2 PLLs in 7044, VCXO need to be installed and 0R jumpers removed

Option 1 and 2 do not require HW modifications that would require soldering iron. Moreover in this configuration we can use the boards in MTCA board without or with uRFB and also in stand-alone mode.

Before we build the boards, the loop filters need to be designed.

sbouhabib commented 7 years ago

@gkasprow I think the loop filter should be prepared as a placeholder (or most common case with the possibility to be updated) since with that many options different filters would be needed, we should just decide for the form (active/passive, order etc...) I think the Evaluation module one would be universal as in type and values; http://www.analog.com/media/en/technical-documentation/evaluation-documentation/HMC7044_EVAL_Schematic.pdf , OR preparing the board for the same frequencies as the CLK mezzanine would give with the lowest possible phase noise options using the built in VCOs

gkasprow commented 7 years ago

For the moment I placed values from evaluation module.

jbqubit commented 7 years ago

@jbqubit IIUYC, the main differences between your suggestion and my previous post are that:

You would like to have separate outputs on the clock mezzanine for the DAC and ADC clocks

@hartytp I'm persuaded by your reasoning that the clock dividing is adequately low noise for generating the LO without degraded phase noise. There's also a nice writeup on this by SiLabs that got me comfortable with this. Agreed that there's no need for a second HMC704x.

http://www.silabs.com/Support%20Documents/TechnicalDocs/Clock-Division-WP.pdf

You would like the external SMA input to be used for 100MHz as well as ~2GHz

b. What's the use case for inputting 100MHz via the front-panel SMA?

I'm aiming to support the situation where Sayma is operating stand-alone and there is no backplane-supplied clock. Looks like @gkasprow solved this with configuration (2).

jbqubit commented 7 years ago

HMC7044 includes an integrated VCO pair that span 2400 to 3200 MHz. Use of an external VCO is forseen using CLKIN1_FIN. However, CLKIN1_FIN is already slated for connection to the clock mezzanine borad. So it looks like the HMC7044 can't produce a 2 GHz DAC clock.

sbourdeauducq commented 7 years ago

I'm not sure what your point is. The purpose of the clock mezzanine is to provide that external VCO so yes, FIN should go there. Even with the output dividers, it seems you cannot generate 2GHz without driving the internal VCO out of range (without external VCO, the 7044 has a hole in frequency coverage between 1.6GHz and 2.4GHz). Are 1.6GHz or 2.4GHz unsuitable? You can use a clock mezzanine if that's the case.

hartytp commented 7 years ago

@sbourdeauducq IMHO, the purpose of putting a synth (PLL + VCO) on the RTM is to allow the commonly required DAC clock frequencies to be produced with respectable phase noise, without a clock mezzanine. At a minimum, this means that the RTM synth should be able to produce: 1GHz (max DAC clock for 1x interpolation); 2 GHz (max for x2 interpolation) and 2.4GHz (max DAC clock frequency). If it doesn't cover these frequencies then it's likely that we'll end up having to build and support a broad-band, medium performance version of the clock mezzanine in addition to the the high-performance, narrow-band mezzanine. In my view, if we have to do this anyway, there isn't much point having a synth on the RTM as well.

I think that a good case has been made for having a broad-band synth on the RTM, but I'm not totally convinced that the 7044 is necessarily the best IC for the job. Why not use a HMC7043 in conjunction with a separate synth IC? It would be nice to decouple the JESD204B functionality from the clock synthesis and, adding an extra IC doesn't add much extra complexity/cost of the design.

A good choice of synth might be the HMC830, as this is a relatively high-performance synth with integrated VCO covering 1.5MHz-3GHz in fundamental mode as well as a wide range of output dividers (and hence capable of generating any frequency we require).

sbourdeauducq commented 7 years ago

means that the RTM synth should be able to produce: 1GHz (max DAC clock for 1x interpolation); 2 GHz (max for x2 interpolation) and 2.4GHz (max DAC clock frequency).

The 7044 can do 1 and 2.4 and more generally integer divisions of any frequency in the VCO range.

Why not use a HMC7043 in conjunction with a separate synth IC?

More parts = more complex circuit, more complex BOM, more things that can go wrong.

It would be nice to decouple the JESD204B functionality from the clock synthesis

What do you mean by that and why?

jbqubit commented 7 years ago

I should have added more to my post regarding the constraints of HMC7044. I think it's fine that it doesn't support 2.0 GHz but want everybody to be aware of this limitation in case I'm missing something.

@gkasprow has begun layout #45. The time for adding new features is past. I'll create a new Issue for consideration of this topic for the second prototype round.

dhslichter commented 7 years ago

I agree with @hartytp about the niceties of splitting the PLL functionality from the distribution IC, to allow for more frequencies. That said, I think 1 GHz and 2.4 GHz, with no 2 GHz, will probably suffice for our purposes in the initial round. Of course, if 2 GHz turns out to be needed for some important reason, it can be generated on a clock mezzanine and added after the fact.

For the next round of boards, after prototype, I would definitely support using an HMC830 plus HMC7043/7044 for distribution. I understand that this means more chips/programming/etc, but it definitely makes the system more versatile.

gkasprow commented 7 years ago

Shall I add HMC830 to existing HMC7044? Of course this means additional mux on FIN input

sbourdeauducq commented 7 years ago

add HMC830 to existing HMC7044

If we have a HMC830 then there is no reason to use HMC7044, it should be HMC7043 instead.

hartytp commented 7 years ago

Does anyone still object to adding the HMC830? Speak up if so...

dhslichter commented 7 years ago

I would prefer the HMC830 + HMC7043; @jbqubit seemed to be against it?