Kasli specification

[jbqubit]

@jordens is tracking design proposal/specification for Kasli on the wiki. Let's discuss PCB_kasli specification here.

https://github.com/m-labs/sinara/wiki/Kasli

How Kasli might interface to peripherals is discussed in #99.

[jbqubit]

At teleconference today we discussed freezing Kasli electrical interface so that there is compatiablity with Metlino-VHDCI #99. Did I get this right?

• Kasli uses +12V supply to Kasli
• 8 IDC connectors on Kasli
• each IDC connector has
  • 16 pins for general purpose IO (eg 8 LVDS). Some of them will be dedicated to SPI as secondary function
  • 2 pins for I2C
  • +12V supply, 100 mA max. per IDC. 5V or any other will be generated locally on every IO board
• Artix-7 FGPA

@gkasprow Which student is working on Kasli? Please add to #19 and github. Sebastien already added him (@marmeladapk)

[gkasprow]

I edited specification above.

[dtcallcock]

@jordens Can we have subheadings for each proposed mezzanine on the wiki?

Currently the wiki mentions:

• 16 TTL drivers
• 40 DAC channels
• CameraLink connector

But some other options were mentioned in #99 including SPI interface using LVDS over RJ45 and isolated TTLs.

I would also like to suggest a low cost synthesiser for driving non-critical AOMs/EOMs (ie, where you don't need deterministic phase control or sub-kHz resolution). Would ideally also have pre-amp/switch/attenuators/filters so that only an external power amp is needed. Could be based on the ADF4351 (an $8 synth-on-a-chip) like the SynthNV is.

[jordens]

@dtcallcock sure. Do you want to edit that yourself?

[dtcallcock]

@jordens I have had a go at reorganising the wiki page based on these discussion threads and the work done so far by @gkasprow and @marmeladapk. You guys should fill in any blanks and correct where I've misunderstood what you're up to. I'm also a little unsure when exactly we put things on the wiki. Do we do it right away and then discuss in the issue or suggest in the issue and move to the wiki one we've reached some kind of consensus?

I also have a few questions:

• The /ARTIQ_ALTIUM/PCB_3U_BNC folder contains two .pdf output files, one a few weeks older than the other. Is this a mistake?

• What voltages go over the IDC? Some schematics have +5V and some have +3.3/+12V. Also what are the current specs?

• What is the 5th RJ45 on 3U_RJ45 going to be used for?

• Can we use thread-lock barrel connectors on the boards? These are much more robust in a lab setting and are backwards-compatible with non-locking plugs for those that don't care. Example parts are connector and plug.

• Should the barrel connector input on PCB_3U_VHDCI_Baseboard have some minimal EMI filtering?

[gkasprow]

I deleted wrong pdf file. We use 12V and 3.3 for EEPROM readout. It is available even when the box is not powered. The fifth R45 was meant as I2C link for slow devices, but I don't have strong opinion about it. Yes, we can use such connectors. Just add an issue for Michal. We can install CMC and EMI filter but such should be already in the power supply.

[dtcallcock]

We can install CMC and EMI filter but such should be already in the power supply.

The power supply was originally required to be "just some random wall wart". In my experience these are often horrifically noisy and have poor regulation. Should we add circuitry to deal with that or just change the spec to "reasonably clean isolated +12V supply".

The fifth R45 was meant as I2C link for slow devices, but I don't have strong opinion about it.

I'd probably rather leave this off due to I2Cs lack of robustness to added bus capacitance and possible voltage level incompatibilities. SPI will be better for off-board devices. If remote I2C devices are really desired though, we could utilise bus buffers like these from Intersil.

There is also a high likelihood of accidentally plugging in one of the adjacent LVDS cables or vice-versa, though this could be avoided by using a different connector.

I deleted wrong pdf file.

The .pdf there now is the one that still has the old 5V power rail.

[gkasprow]

Indeed, there is wrong schematic version still with 5V bus.

[gkasprow]

fixed

[jordens]

@dtcallcock looks good. Thanks! I am all on board with the additions/changes and have added a few details.

• I am okay with saying "reasonably clean 12V". But IME there are plenty of examples where standard wall wart supplies have been without problems. E.g. in the USRP. I think we can aim to replicate that.
• Supply-wise I don't care. Greg should probably choose what's best/easiest.
• The thread lock barrel connectors (or similar Bajonett style things) are a good idea.

[hartytp]

Just to check: the current plan is that each Kasili communicates with/receives time from the root Metlino via a fibre link (one fibre per Kasili), and not via a backplane, right?

[jordens]

Yes. The VCO/DDS synthesizer extensions would receive their reference clock individually.

Another issue is experiments where the set of Kasli extensions is sufficient. In those experiments Metlino (plus Rack, clock RTM) seems pointless and too powerful/big and requiring a separate box/piece of hardware just to be able to talk to Kasli is troublesome.

[hartytp]

Yes

@jordens Good, that seems like the correct solution to me. IIRC, the Metlino currently has 3 SFPs, which is extensible to 7 by adding an FMC-SFP board. However, given the number of different functions currently planned for Kasili, I can imagine using these all up pretty quickly. What's the plan for extending this further?

The VCO/DDS synthesizer extensions would receive their reference clock individually.

Sure, if that's useful, although IMO it's a bit overkill: IIRC, the recovered clock should actually be pretty good noise-wise and, I assume that one be better off use Sayma for any ultra-low noise purposes.

Another issue is experiments where the set of Kasli extensions is sufficient. In those experiments Metlino (plus Rack, clock RTM) seems pointless and too powerful/big and requiring a separate box/piece of hardware just to be able to talk to Kasli is troublesome.

What's your plan for this use case?

[dtcallcock]

The VCO/DDS synthesizer extensions would receive their reference clock individually.

Sure, if that's useful, although IMO it's a bit overkill: IIRC, the recovered clock should actually be pretty good noise-wise and, I assume that one be better off use Sayma for any ultra-low noise purposes.

I can see how you might want this as an option for a DDS. For the synth though, can the Kasli FPGA just provide the clock over one of the LVDS pairs? I'm not entirely sure what the Artix clock output capabilities are though.

[jordens]

• You get a bunch of SFPs with each Sayma. You can also hook up another (slave) Metlino to the (master) Metlino. And we can daisy chain Kaslis ad infinitum. All those require DRTIO switch support and consensus about one preferred option that would be implemented.
• Distributing the clock from Kasli to the extension boards is tricky. There is little space on the frontpanel, doing it inside makes assembly even harder than it is now (now you already have to disassemble the crate when you change/add/remove extensions), and there is little space on the PCB for SMA connectors.
• I would be afraid to run a reference clock over uncontrolled impedance, unshielded, IDC connectors, even for a simple Synth. We would also need a clock fan-out as the Artix outputs are very noisy. I am fine with putting an XO on the extension boards as a low complexity alternative to a reference clock.

[hartytp]

You get a bunch of SFPs with each Sayma. You can also hook up another (slave) Metlino to the (master) Metlino. And we can daisy chain Kaslis ad infinitum. All those require DRTIO switch support and consensus about one preferred option that would be implemented.

Good, that works for me. Is the plan for each Kasli to have 2xSFP to allow daisy chaining then?

[jordens]

Right now I don't see a different use for them.

[hartytp]

I would be afraid to run a reference clock over uncontrolled impedance, unshielded, IDC connectors, even for a simple Synth.

Depends on what the reference clock frequency is. For, say, a 100MHz reference clock frequency, routing board-board via pin header will be fine.

IIRC, the AD9912 has a pretty decent clock multiplier (PLL w/ integrated VCO), so there's no need to supply a GHz clock directly, unless one wants to push the noise performance (in which case, the Sayma might be a better choice anyway). Otherwise, one can always use something cheap like an AD9576 to do the same job.

IMO, it'd be great if these Kasli extension boards could be used without needing an external GHz frequency source...

[hartytp]

@jordens My previous post was ambiguous, I meant "Is the plan for each Kasli to have 2xsSFP (which would allow them to be daisy chained)?" Answer: "yes", so I'm happy with this plan.

[hartytp]

We would also need a clock fan-out as the Artix outputs are very noisy.

Won't there be a spare output from the Si5324 we can use to supply the reference clock?

[jordens]

Yes, there are currently two SFP cages on Kasli. Those impedance discontinuities will give us reflections which can quickly lead to double clocking leading to the need for filters etc. I would not want to go through that again. Yes. The SI5324 has one other output. But there are eight extension cards.

[hartytp]

Those impedance discontinuities will give us reflections which can quickly lead to double clocking leading to the need for filters etc. I would not want to go through that again.

IME, this isn't too much of an issue for clocks in the 50-100MHz range. Besides which, aren't you planning to run an SPI bus at something like 50MHz over this pin header?

IIRC, the AD9912 has a max PLL divider ratio of 66, so the reference clock can be as low as 15MHz while still running at max output frequency. I really don't see any issues with reflections at that kind of clock speed.

[hartytp]

Yes. The SI5324 has one other output. But there are eight extension cards.

Sorry, I'd misread a previous post, and was talking cross-purposes here. Of course you need a fanout buffer for this...

[jordens]

SPI/I2C is typically de-glitched AFAIK (at least our SPI slave is de-glitching). And with SPI you can always go slower if it doesn't work. And for a square clock it's not the frequency that determines the reflections. It's the edge speed and the bandwidth of the input buffer at the other end. A 15 MHz clock with the same fast edges as a 1 GHz clock from a good digital fanout will give you the same distorted edges and potential double-clocking at the (fast) input of the AD9912. We've had this at 50 MHz over much shorter connections and better-than-IDC connectors. I'd prefer to be able to choose between standard on-board XO and good external input.

[hartytp]

And for a square clock it's not the frequency that determines the reflections. It's the edge speed and the bandwidth of the input buffer at the other end. A 15 MHz clock with the same fast edges as a 1 GHz clock from a good digital fanout will give you the same distorted edges and potential double-clocking at the (fast) input of the AD9912.

Sure, but if your fundamental is low, you can use a slower fanout buffer (w/ hysteresis if necessary)/lower drive current or add some RC to reduce the edge time without causing problems.

I'd prefer to be able to choose between standard on-board XO and good external input.

IMO, it's nice to have absolute accuracy from a PLL without having to supply an external (SMA) reference. But, I don't feel too strongly about this.

[dtcallcock]

I am fine with putting an XO on the extension boards as a low complexity alternative to a reference clock.

A decent XO will get us <1ppm which will be fine for a >1kHz resolution synth.

The SI5324 has one other output. But there are eight extension cards. Distributing the clock from Kasli to the extension boards is tricky. There is little space on the frontpanel, doing it inside makes assembly even harder than it is now (now you already have to disassemble the crate when you change/add/remove extensions), and there is little space on the PCB for SMA connectors.

Could we run that spare SI5324 output to a single SMA? Then people can use it as their 'external reference' if they so choose. The distribution problem (for >1 extension card) still has to be solved whether they use that or another source they provide.

[dtcallcock]

Another option for clock distribution is to use the backplane. Cards could revert to using the IDC (with no clock) if no backplane is present.

Is there currently a plan for the backplane? The Kasli and VHDCI carriers are currently planned to have a DIN 41612 C, 96 position connector. This would presumably make them mechanically compatible with VME J1 backplanes. Is the idea to somehow use those or just design a custom one? With the 96-position connector it would be trivial to wire the equivalent payload of 4 IDC headers (each 8xLVDS pairs, I2C, power) plus a clock to 4 slots in a star topology.

[hartytp]

A decent XO will get us <1ppm which will be fine for a >1kHz resolution synth.

<1ppm is pretty good for an XO -- most of the ones I've seen are more like 10ppm initial accuracy and then ~1ppm/year ageing. But, I take your point, this won't be an issue.

Another potential advantage of using an XO, rather than distributing the clock via ribbon cable, is that it makes it easier to run these extension boards off a standard microprocessor board like an Arduino.

[gkasprow]

There is no standard backplane on the market, so one would have to design it. In case of VHDCI, we would need dedicated adapter from VHDCI to the backplane. For Kasli we probably have enough pins to have both VHDCI and backplane connected. For the moment we can just add SMA or MMCX clock output for extension boards that need it. DO you really need to connect 8 extension boards with clock input? Of course we can put clock splitter at the SIlabs output to generate 8 copies of the clock.

[hartytp]

@jordens When Kasli is mounted in an enclosure how do you plan to bring out the ribbon cables for the daughter boards? Front panel connectors?

[jordens]

@hartytp they run inside the crate between the cards. See the screenshots on the wiki page.

I am ok with routing the other SI5324 output to a front panel SMA.

I would enable the use of the backplane in the same way as the ribbon cables. I.e. have four extension connectors routed to IDC/ribbon and a pin group on the backplane/DIN. The backplane would then do a star topology to up to four the extension card. But I would not require the use of the backplane as it becomes tedious to design new backplanes for small (single extension) enclosures.

[hartytp]

@jordens The Wiki mentions using Kasli in standalone mode in a frame enclosure, so no rack in this case. Or, is the plan that Kasli never gets used outside (at least a small) small rack?

I assume that when Kasli is in a rack, it just gets a front panel, rather than a full frame enclosure (otherwise, you need a hole in the enclosure to get ribbon cable out, which is inconvenient).

[jordens]

But the enclosures are "similar" to racks in that you can slide in the boards the same way and screw the frontpanels into the front of the enclosure. You would slide Kasli and one (or more) extensions into a small enclosure after having connected them with ribbon cables. The only real difference is the size and the modularity of the enclosure vs the rack.

[hartytp]

@jordens Okay, so is the plan is to design enclosures (or, rather, front panels for enclosures) that can fit Kasli + some (1? 4?) extension boards?

[jordens]

Yes. Frontpanels will be made. They are per extension/Kasli.

[dtcallcock]

I would enable the use of the backplane in the same way as the ribbon cables. I.e. have four extension connectors routed to IDC/ribbon and a pin group on the backplane/DIN. The backplane would then do a star topology to up to four the extension card.

Sounds good. For the VHDCI carrier the 4 IDC/ribbon connectors connected to the backplane should all come from the same VHDCI connector (so Sayma FMC-VHDCI isn't limited to using two backplane slots). For the Kasli carrier, is there enough FPGA IO that the backplane could have its own lines (thus in principle allowing for 12 extensions)?

But I would not require the use of the backplane as it becomes tedious to design new backplanes for small (single extension) enclosures.

Yes, the IDC header should be provided on all extensions too.

How do we feel about adding a clock pair to each backplane pin group as well? On the VHDCI carrier these would be driven by a front panel SMA input. The Kasli would drive them with its recovered clock. Extensions needing a clock would then have a switch to choose between this one and one provided on a (front panel?) coax connector.

[hartytp]

If we do put a CLK_IN SMA on the VHDCI carrier front panel, used to drive the back plane, could we also add a buffered CLK_OUT SMA to allow daisy chaining? If this isn't easy (e.g. due to space considerations) then don't bother.

[gkasprow]

we already have on he front panel: 2xSFP micro USB for UART and JTAG SATA connector we can still fit 2 SMA connectors look here

[jordens]

For a few projects we want to run Kasli stand-alone. I.e. Ethernet over SFP, coredevice and DRTIO master on Kasli. That means there won't be a recovered clock but one that comes from the frontpanel.

If we can have one SMA on the front panel that's great. That would be either an output for the recovered clock (in DRTIO satellite mode) or an input for the RTIO reference clock (in Kasli stand-alone mode). I am ok if that is selected by capacitor placement.

If we really consider it a high value goal to distribute clock (over the backplane), and if we can limit the complexity to a single simple dumb clock fan out that does not need configuration, and if there is space on the panel, I am ok with that second SMA being used as an input to an independent clock distribution from the Kasli board to the backplane. It is getting a bit tight with the pins on the backplane connector (96 [available] = 4 [extensions]*(2*8 [lvds IO] + 2 [12V] + 1 [3.3V] + 2 [I2C] + 2 [lvds clock] + 1 [GND]) is cutting it way too close with GND. We should probably limit the backplane to three extensions.

[gkasprow]

We can install SIlabs chip with fanout IC. Silabs has 2 inputs so one can be routed to the front panel One output goes to the FPGA, another to the fanout IC. Fanout IC then to second SMA on a front panel, to the backplane and to FPGA Second input of the fanout could be connected to input SMA as well. In this way we could use it in daisychain configuration as well, without SIlabs in the loop.

[hartytp]

@jordens To confirm what it says on the Wiki atm: neither of Kasli DDS or VCO will have analogue inputs, so no servos will be possible without additional hardware, right?

[jordens]

@gkasprow That sounds good. @hartytp Yes. But they have means to adjust power. We plan to implement (slow) servos with additional hardware (digital distributed photodetectors).

[hartytp]

We plan to implement (slow) servos with additional hardware (digital distributed photodetectors).

Nice. In a sufficiently compact form-factor, that could be a really great thing to have around the lab.

Would that connect to Kasli using I2C or something like that? Do you have a draft proposal/writeup/specs or an estimated time-line for this?

[jbqubit]

+1 to limited clock distribution over backplane

Seven days ago @jordens said

You get a bunch of SFPs with each Sayma. You can also hook up another (slave) Metlino to the (master) Metlino. And we can daisy chain Kaslis ad infinitum. All those require DRTIO switch support and consensus about one preferred option that would be implemented.

Some questions about daisy chaining. Example daisy-chain: Metlino-Kasli-Kasli-Kasli

• Current plan is for 2 SFP per Kasli. That leaves no free SFP for fiber-based connection to a remote peripheral or to use the transceiver photo-detector for photon detection. Consider removing the front-panel SATA and replace with more SFP(s). Support of SATA, SFP, ribbon and backplane seems an unnecessary proliferation of interconnect options.
• Google says there are commercially available SFP to SATA.
  • https://shop.trenz-electronic.de/en/TE0424-01-SFP-2-SATA-Adapter
• What do you anticipate for latency per Kasli in my example? Will it be deterministic?
• Is there a proposed DRTIO switch implementation?
• Please restate question for which you'd like consensus.

[jordens]

@jbqubit Looking at Sayma AMC again, there are no transcievers connected to the Fayma AMC FMC. There are not enough to do everything. The connections available for designation as downstream DRTIO could be the two SFP, the two SATA and the two FAT_PIPE links.

Using the transcievers for photon detection is tricky (very) because you have a (strong) DC bias and lots of low frequency energy. I.e. they can't be AC coupled, which is what SFP modules and adapters tend to do internally. Apart from that I would not like to squeeze that functionality into the use cases Kasli is targeting. Use the SATA port or the second SFP port, or a port on Sayma (which also has faster transcievers) for that. The SATA connector is convenient and adequate for daisy chaining within the same rack. This is completely orthogonal to the IDC ribbon and backplane connections. I don't understand why you bring them up here.

Latency will be deterministic. Kasli is a DRTIO satellite.

There is a DRTIO switch implementation path that we have discussed, also on IRC and indirectly in that thread. Yes.

When we write the gateware for Sayma and Metlino, we need to decide what's in the FMC slot(s), what that FMC is used for, and what the SATA/SFP ports are used for. TTLs? Transcievers? Photon tagging? DRTIO downlinks? DRTIO uplinks? ADC FMC? DAC FMC? CameraLink?

@hartytp The photodetectors would likely be developed by PTB. I'll try to convince them to go open hardware with them. I2C most likely, otherwise "typical" Ion trap photodetectors, nothing more specific yet. Timeline goal is to be coincident with the rest of Kasli, maybe a bit later, maybe a bit earlier.

[dtcallcock]

It is getting a bit tight with the pins on the backplane connector (96 [available] = 4 [extensions](28 [lvds IO] + 2 [12V] + 1 [3.3V] + 2 [I2C] + 2 [lvds clock] + 1 [GND]) is cutting it way too close with GND. We should probably limit the backplane to three extensions.

There are opportunities for more grounds:

• 3.3V is low current (20mA per extension) so we can use a single pin, giving 3 more grounds.
• With a backplane, the 12V will come directly to the backplane and not via the barrel connector on the carrier, right? If so then 2*12V for Kasli carrier power should be sufficient, giving 6 more grounds.

At this point we'd have 13 grounds. There is also the less desirable option of:

• Clock fanout and/or I2C switch on the backplane. Up to 12 more grounds this way.

[hartytp]

@hartytp The photodetectors would likely be developed by PTB. I'll try to convince them to go open hardware with them. I2C most likely, otherwise "typical" Ion trap photodetectors, nothing more specific yet. Timeline goal is to be coincident with the rest of Kasli, maybe a bit later, maybe a bit earlier.

Thanks for the update. Good luck! IMO, this is quite important, since the DDS boards are significantly less useful to us without some kind of servo...

[dtcallcock]

We plan to implement (slow) servos with additional hardware (digital distributed photodetectors).

I was just thinking of starting a discussion of this.

I2C most likely

One of the problems of these 'slow' servos is that they are often feeding back on few-us pulses. Thus if a relatively slow bus like I2C is being used, a fast acquisition trigger or sample and hold should also be implemented.

otherwise "typical" Ion trap photodetectors

I quite like the way Moglabs have implemented this. They use a firewire cable that contains +/-12V and a shielded twisted pair for the differential photodiode signal. See p84 of this manual.

[jbqubit]

The SATA connector is convenient and adequate for daisy chaining within the same rack.

How does this enable: Metlino-Kasli-Kasli-Kasli-... Two identical ports are required for daisy-chaining.

What's wrong with adding more SFP on the front panel and using a SFP-SATA adapter as required?

In what circumstances do you see SATA as preferred to SFP? SFP transceivers are $20/pair.

When we write the gateware for Sayma and Metlino, we need to decide what's in the FMC slot(s), what that FMC is used for, and what the SATA/SFP ports are used for. TTLs? Transcievers? Photon tagging? DRTIO downlinks? DRTIO uplinks? ADC FMC? DAC FMC? CameraLink?

This broad topic is discussed in https://github.com/m-labs/sinara/issues/139. I created a new Issue for FMC #148. Let's not discuss this here.

[jbqubit]

@dtcallcock comment moved to #148.