search

sinara-hw / sinara

Sayma AMC/RTM issue tracker
Other
42 stars 7 forks source link

Kasli Zotino: Slow DAC features/specs #142

Closed hartytp closed 7 years ago

hartytp commented 7 years ago

Breakout issue to discuss the Kasli/VHDCI slow DAC extension board in more detail.

The following is an initial attempt at a specification for this board:

jordens commented 7 years ago

I would not focus on high update rate and not try to tweak the design for that too much. The scheduling of the individual updates over SPI is really difficult if you want to do streaming. Allocating the SPI bandwidth so that something is guaranteed is complicated. I am fine with "fewer channels". But 32 would be my goal. You can always use more boards and leave the unused channels alone to increase the sample rate. Yes. Connectors are a challenge. They should be easy to assemble, commonly available, mechanically and electrically good. Don't know what's en vogue nowadays. VHDCI again? @hartytp feel free to attach labels to these issues (and make labels where missing). That simplifies searching for issues a lot.

hartytp commented 7 years ago

I would not focus on high update rate and not try to tweak the design for that too much.

Agreed, optimising the bandwidth definitely shouldn't be a priority for this design. But, using a few DAC ICs rather than a single one has a minimal impact on the cost/complexity of the board and gives the user a bit more flexibility.

The scheduling of the individual updates over SPI is really difficult if you want to do streaming. Allocating the SPI bandwidth so that something is guaranteed is complicated.

Could you elaborate on that a little, please?

jordens commented 7 years ago

Yes. I am fine with splitting the extension so that there are as many SPI buses (plus LDAC) as we can fit on the IDC connector and 32 channels in total. But with 8 signals we can only do two SPI buses. And then we don't even have LDAC yet.

Well it's a shared bus. You can not set a channel at some arbitrary time because there might already be traffic for other channels at or around that time. You always have to treat them all as a block of channels if you want to pump lots of data. And you will be dealing with contention all the time.

hartytp commented 7 years ago

In which case, we need a single shared clock line + 2 lines/DAC (MOSI + LDAC), allowing 3 DACs per board. 3 * 16channels = 48 channels, which seems like a good number to me.

Edit: I forgot that we'll probably need a SYNC line as well, meaning we can only run 2DACs per board...

jordens commented 7 years ago

Sharing SPI clock breaks our SPI master. That's currently a no-go for us. Trying to save pins by demoting MISO to "debugging" is a terrible idea.

hartytp commented 7 years ago

Okay, if you feel that strongly about it then we can stick with a single DAC IC.

Trying to save pins by demoting MISO to "debugging" is a terrible idea.

Out of curiosity, what do you plan to use the DAC's MISO for?

jordens commented 7 years ago

Verifying that the chip is there, verifying that writes work, reading overtemperature indicator. Sure it's not much but I don't think saving a pin or two is worth it since even the occasional debugging would require building new bitstreams/runtime, and manual intervention inside the hardware. SYNC is CS. We could share the two LDAC and the two MISO. Then two DACs fit on one IDC.

dtcallcock commented 7 years ago

Or just use 2xIDC for 3x16ch DAC. That still lets you run 192ch off a single Kasli.

Yes. Connectors are a challenge. They should be easy to assemble, commonly available, mechanically and electrically good. Don't know what's en vogue nowadays. VHDCI again?

Individual screening will tricky without going to a custom assembly (like NI does for a lot of its products). One simple option would be HDMI - that gets you 4 individually screened pairs in a cheap, easily available cable. The 6 connectors needed would fit comfortably on the panel (also mini connectors are available). Reasonable breakouts already exist: https://www.adafruit.com/products/3120

dtcallcock commented 7 years ago

Output: +-10V (ideally +-12V using +-15V supply rails -- are these generated on the extension board using switch modes, or supplied by the carrier?)

The carrier only provides +12V.

gkasprow commented 7 years ago

The carrier only provides +12V.

yes, because most extension boards require single voltage. On DAC ones we will add simple DC/DC converter that would produce +/-15V

gkasprow commented 7 years ago

VHDCI are difficult to solder. Usual DB or high density DB are much more common.

hartytp commented 7 years ago

@jordens

Yes. Connectors are a challenge.

Every time we build this kind of thing, we end up going for a standard D-sub and saying "next time we should really find a better solution".

They should be easy to assemble, commonly available, mechanically and electrically good

I'd add to that list that pre-made connectors should be available commercially with a "sensible" lead time/cost (I don't want to waste time because one of us cross-wired 2 channels for a 200 electrode ion trap when they hand assembled the cabling).

@dtcallcock

One simple option would be HDMI - that gets you 4 individually screened pairs in a cheap, easily available cable.

From an electrical perspective, HDMI is pretty nice. But, for routing say 200 channels to a trap, the cabling gets to be a bit of a mess (although, not unworkable). Also, since our signals will be single-ended, routing them as screened pairs is not ideal.

Individual screening will tricky without going to a custom assembly (like NI does for a lot of its products).

AFAICT, none of the cables that NI offers as stock would be suitable for this design, which is a shame. I've never looked into getting custom cable assemblies made, so no feeling for what the costs would be. Might not be too bad if we placed a bulk order for everyone who's interested?

Usual DB or high density DB are much more common.

And, high-quality screened (but not individually screened) Dsub cables are widely available, which is a plus.

Given the relatively low speed of this design, individual screening is probably not required to keep channel-channel cross-talk to an acceptable level for cable runs of a few m (we should make sure the cables are driven from a nice low impedance though).

So, unless anyone (@gkasprow @sbouhabib) can suggest a better alternative, let's stick with DB.

For 48 DAC channels, I'd suggest either going for 2xD25 (1 GND per cable) or, if we want a bit more grounding, for 2xD37. Any preferences?

hartytp commented 7 years ago

Verifying that the chip is there, verifying that writes work, reading overtemperature indicator. Sure it's not much but I don't think saving a pin or two is worth it since even the occasional debugging would require building new bitstreams/runtime, and manual intervention inside the hardware. SYNC is CS. We could share the two LDAC and the two MISO. Then two DACs fit on one IDC.

Playing devil's advocate, while I agree that it's nice to be able to read DAC registers:

I would have marginally preferred to keep the LDAC lines for the two DACs separate to give the user a bit more flexibility in how the DACs are used. However, I don't feel strongly about this, as separate LDAC lines aren't required for any of the uses I have in mind (and, in any case, these boards should be cheap enough that a user can always use more boards to get more flexibility).

@dtcallcock

Or just use 2xIDC for 3x16ch DAC. That still lets you run 192ch off a single Kasli.

That's definitely a possibility. But, it seems nicer to keep it to 1 IDC per extension board if possible: fewer cables to make; and, makes setting up the software configuration easier.

hartytp commented 7 years ago

Edited top-level post to include comments so far.

jordens commented 7 years ago

Verifying that the SPI communication with the DACs works is something we would want to do in CI all the time. There will be changes to the SPI core and the drivers. There will be changes to the other layers involed in getting data to the DACs. Continuously being able to verify that these changes don't break stuff and being able to automatically identify the faulty commit is something I value very much. In all cases I can remember where convenient readback has been omitted it turned out to be the wrong decision.

hartytp commented 7 years ago

@jordens You raise a good point -- I'm not used to thinking in terms of CI and long-term maintainability for a project of this scale. Shared LDAC and MISO it is.

gkasprow commented 7 years ago

We can order such cables from here

http://www.m2cables.us.com/M2-D-Sub-9-Pin-to-SMA-Fanout-Custom-Assembly-Call-p/m2cable-assembly-212.htm

or we could stick with standard DSUB-BNC SVGA cables. Just google VGA BNC

They are low cost and high performance (200MHz) and they have standard pinout. The only drawback is 5 channels.

And they have color marking.

On a front panel we can easily fit several DB15HD connectors

You can have them for 5$ per 2meter cable.

Another option are CCTV cables where DSUB1HD fans out to 8 BNC, this one looks pretty standard one

http://www.dx.com/p/vga-15pin-male-to-8-bnc-female-connectors-cable-for-cctv-blue-25cm-cable-90310#.WIzzNn80968

http://www.dx.com/p/vga-15pin-male-to-8-bnc-female-connectors-cable-for-cctv-blue-white-32cm-cable-110062#.WIzzPH80968

http://www.tradekorea.com/product/detail/P310544/VGA-to-8-bnc-cable,VGA-15Pin-Male-Breakout-Cable-to-8-BNC-Female-Cable.html

http://topcctvcable.com/index.aspx?menuid=12&type=productinfo&lanmuid=48&infoid=148&language=en

and costs only 4$ per piece

They are used in CCTV recorders

we can order higher quantity from Alibaba and distribute them among interested parties.

gkasprow commented 7 years ago

In terms of Kasli and DACs we can use single ended lines for DAC signals. But in case of VHDCI breakout we have much longer pairs with strong coupling between them so single-ended signalling can be used only for slow signals, after lowpass filtering and Shmidt trigger buffer. So each LVDS can drive 8 LVDS pairs of 16 LVTTL signals in Kasli easily. For VHDCI breakout we can also use LVTTL but only for signals that do not need high performance, i.e. DAC selection, resets, status line providing that we still meet setup and hold time constraints. MISO, MOSI, CLK need to be driven by LVDS transceivers.

hartytp commented 7 years ago

@gkasprow

MISO, MOSI, CLK need to be driven by LVDS transceivers.

Agreed, it makes sense for these signals to be LVDS.

Thanks for the suggestions for cables to use with the (single-ended analog) DAC outputs. However, all the cables you suggested are coaxial, with SMA/BNC on one end. Given the space constraints we have in the lab, and the fact that an experiment might require 200+ DAC channels, I think we really need something with a higher density.

In the past, I've hand-assembled cables using individually screened twisted pairs (for single-ended signals, one of the pair is GND) and DB connectors on either end. As @dtcallcock points out, this is essentially what NI do for their DAQ cards. IMO ideally, we'd find a solution where people can use a standard DB cable if they don't care about cross-talk, but also specify a compatible higher quality cable for more sensitive applications...

gkasprow commented 7 years ago

In such case one can use 3M MDR (mini delta ribbon) cables. The cables can be soldered directly to the contacts or one can use IDC version where 2 ribbons are attached simultaneously. One can use twisted pair ribbon cables as well. Special tool is required, don't try to do it at home:) There are shielded versions of such twisted and not twisted ribbon cables from 3M. I used MDR50 due to space constrains, but much more popular is MDR68 used in SCSI-2. One can buy such cables in various lengths. There are popular SCSI cables with MDR68 on one end and VHDCI on another end. National is using VHDCI68 cables but with different diff pair assignment, but for slow analog signals it won't make any difference So we can keep NI signal assignments like here, page 16 http://www.ni.com/datasheet/pdf/en/ds-153 or page 19 http://www.ni.com/pdf/manuals/375140a.pdf In this way one could use National terminal boxes http://sine.ni.com/nips/cds/view/p/lang/pl/nid/10175 It's better to keep any suitable standard than no standard at all. Just to make it difficult to mix the connections, for analog signals let's use MDR68, for digital LVDS - VHDCI. In this way there are several options for DAC outputs:

hartytp commented 7 years ago

@gkasprow Thanks for posting that. I haven't used MDR before, but they look to be exactly what I was after.

Looking on Digi-Key/Mouser for high-quality 3M cables with per-channel screening, the 50pin connectors seem to be the most common. So, what about going for 40ch per DAC board (assuming you'd be happy putting that many DAC channels + active filters etc on a single board) with 2 x MDR50 connectors, each supplying 20 DAC channels? Pin out per 3M 14X50-SZWB-XXX-0NC series cables? Alternately, we could put 48 DAC channels down a single 141A0-SZ5B-XXX-0DC MDR100 cable, if people would prefer that (assuming the cabling doesn't get too bulky/inflexible with that many conductors)? Or, if you have a good source of MDR68 cables, I'd be happy to use them instead.

Digi-key sells high-quality cables with individual screened twisted pairs for ~300-200USD per cable depending on length + conductor count. There are a few of the 50-way assemblies in stock, otherwise lead-time is 9 weeks.

Edit: a cheaper alternative is to use SCSI-2 cables, e.g. http://cable-trader.co.uk/items/scsi-ii-hd50-male-to-hd50-male-1m

On the 3M 14X50-SZWB-XXX-0NC we could consider using some of the unshielded pairs for +-15V to power any low-power active filters etc on the Rx end. For the 141A0-SZ5B-XXX-0DC we could re-purpose some of the 4 auxiliary GND conductors for this (assuming we don't mind breaking the standard slightly).

gkasprow commented 7 years ago

It is possible to assemble ribbon cables manually but you need non-standard ribbon cables (thin ones) With standard ones it's extremely hard to clamp them using vice.

gkasprow commented 7 years ago

The SCSI-2 and MDR68 connectors have the same footprint so one can assemble one or another on the PCB.

hartytp commented 7 years ago

@gkasprow

  1. What pin-out were you thinking of using with the MDR68 connectors? e.g. how many DAC channels/GNDs were you thinking of connecting to the cable?

  2. It would be nice to keep the option open of using cabling which provides screening of individual channels. For MDR50/MDR100 I could find cables like 141A0-SZ5B-XXX-0DC as standard items from Digi-Key/Mouser. But, I couldn't find an equivalent for MDR68. Did I miss one? Otherwise, is there any reason not to go for MDR50/MDR100, as they both seem to be widely available and supported?

hartytp commented 7 years ago

The SCSI-2 and MDR68 connectors have the same footprint so one can assemble one or another on the PCB.

To check I understand the terminology: by "SCSI-2 connector", do you mean MDR50? So, are you suggesting that we design the board to work with either MDR50 or MDR68 connectors? (so in the MDR68 case, one either gets slightly more DAC channels, or we just ground the unused conductors?)

gkasprow commented 7 years ago

I meant only SCSI cables that has connectors which share footprint with MDR68.

They do not have shields, just twisted pairs.

Let’s go for MDR50/MDR100.

MDR100 would be hard to fit on 3U panel and are not handy.

Greg

From: hartytp [mailto:notifications@github.com] Sent: Monday, January 30, 2017 2:08 PM To: m-labs/sinara sinara@noreply.github.com Cc: gkasprow gkasprow@gmail.com; Mention mention@noreply.github.com Subject: Re: [m-labs/sinara] Slow DAC features/specs (#142)

@gkasprow https://github.com/gkasprow

  1. What pin-out were you thinking of using with the MDR68 connectors? e.g. how many DAC channels/GNDs were you thinking of connecting to the cable?

  2. It would be nice to keep the option open of using cabling which provides screening of individual channels. For MDR50/MDR100 I could find cables like 141A0-SZ5B-XXX-0DC http://multimedia.3m.com/mws/media/297587O/3mtm-mdr-cable-assy-050-hispeed-video-data-trans-ts2103.pdf as standard items from Digi-Key/Mouser. But, I couldn't find an equivalent for MDR68. Did I miss one? Otherwise, is there any reason not to go for MDR50/MDR100, as they both seem to be widely available and supported?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/m-labs/sinara/issues/142#issuecomment-276058082 , or mute the thread https://github.com/notifications/unsubscribe-auth/AEH-vg59XxFpLYsm_YTTMkYshTZB_P1nks5rXeCVgaJpZM4LvzLa . https://github.com/notifications/beacon/AEH-vjG57gX1i0CLll5RkC0xr-XgXkOZks5rXeCVgaJpZM4LvzLa.gif

hartytp commented 7 years ago

Okay, so how about we go for:

That gives us 40channels total per extension board, so we can go for:

  1. 1x40 channel DAC
  2. 2x32 channel DACs (wasting 24 DAC channels)
  3. 1x32 channel DAC & 1x8 channel DAC

IMO, we should go for options 1 or 2, since DAC channels are cheap and asymmetry is nasty. I'd generally prefer option 2, but don't feel overwhelmingly strongly about it.

Thoughts? Objections?

hartytp commented 7 years ago

Thoughts on low frequency (<1kHz) noise and thermal drift...

Physicists goal is for a 1MHz trap frequency to be stable to a few Hz. This requires DAC noise/drifts to be <10ppm RMS (1LSB RMS for 16 bits), so:

jordens commented 7 years ago

@hartytp Why not then just do one 40 channel DAC per extension, and have those who need faster channels grab two extension cards and waste the channels there? No adapter board needed. And it makes the design less dense, power hungry and cutting fewer corners (SPI pin sharing etc.). It seems weird that because of the connector choice we are now trying to balance wasting resources. I'd much rather have a less densely populated PCB right now to have some room for all the additional bells and whistles that will be needed as opposed to squeezing every possible use case in there.

hartytp commented 7 years ago

@jordens Okay.

2 DACs is a "nice to have", but I agree that it doesn't make as much sense any more, given our wiring choice.

No adapter board needed.

Which adapter board?

gkasprow commented 7 years ago

SCSI-2 use 68-pin D-type connector like this

http://www.digikey.com/product-detail/en/te-connectivity-amp-connectors/5787190-7/A33515-ND/1121382

which has the same footprint like MDR68

http://www.digikey.com/product-detail/en/3m/N10268-5242PC/N10268-5242PC-ND/1091938

But they are not compatible.

So we can have 2 options – either install MDR68 connector and use custom cables or install D 68 SCSI-style connectors and use standard, low cost SCSI cables which are twisted pairs like VHDCI.

There are standard SCSI cables that have VHDCI on one side and D-68 on the other side.

We can also install MDR50 and use 3M cables

hartytp commented 7 years ago

@gkasprow Thanks for clarifying. Top post edited to summarise these points.

gkasprow commented 7 years ago

@hartytp HD68 (SCSI2) and MDR68 have exactly same footprint and panel cut-out. They both have 68 pins but are not compatible. MDR50 are smaller and have 50 pins

jordens commented 7 years ago

@hartytp

No adapter board needed.

Which adapter board?

My fault. I was thinking wrong.

hartytp commented 7 years ago

HD68 (SCSI2) and MDR68 have exactly same footprint and panel cut-out. They both have 68 pins but are not compatible. MDR50 are smaller and have 50 pins

Yes, sorry, I misread your post (end of the day...).

So we can have 2 options – either install MDR68 connector and use custom cables

How much would this kind of custom cable cost, and who would be a supplier?

gkasprow commented 7 years ago

There are also SCSI cables with MDR50 and MDR68 connectors, called high density or micro- Centronics connector. But they are not very popular. The most popular ones are HD50 and HD68 SCSI cables. So it makes a lot of sense to use MDR50 connectors. We can then use high performance 3M MDR50 cables mentioned before (68pin versions are outdated) or 50pin SCSI cables with HD-50 connectors. I use MDR50 connectors in these projects http://www.ohwr.org/projects/fmc-adc100m14b16cha-dac2m14b4cha/wiki http://www.ohwr.org/projects/fmc-adc100m14b16cha-dac2m14b4cha/wiki

hartytp commented 7 years ago

@gkasprow Okay, thanks for clarifying further.

Summary as I understand it:

I don't have well-calibrated intuitions for how bad the cross-talk would be in, say, a 5M SCSI cable driving high-impedance loads single ended with 100kHz sinusoids. My guess is that it's not going to be a problem, but unless we have some data on this, I'd like to keep the individually screened cables open as a "plan B". I'll have a think about this, and maybe make some measurements in a week or two when I have time...

gkasprow commented 7 years ago

We can always use differential drivers that can be used either as single-eneded or differential. So they can give +/- 20Vpp in diff mode or +/-10V in single ended mode. In this way we can greatly reduce crosstalks. Another option is to use twisted pair with one line grounded on both sides. Yet another idea is to drive diff line in single ended mode and add common mode choke on both ends. In this way for DC we use only one line and ground, but the higher frequency is, the more differential transmission becomes. The CMC on remote side makes signal single-ended again. This is simple and reliable solution but requires grounding of the remote side.

hartytp commented 7 years ago

Another option is to use twisted pair with one line grounded on both sides.

Twisted pair with one ground per DAC channel sounds like a good idea.

I'm happy to connect the grounds on both sides of the cable (ie to connect the Kasli/DAC ground to the Rx/ion trap ground). My reasoning here is that, so long as we keep the Kasli + DAC boards reasonably cheap, ground loops are avoided by driving different grounds from independent Kaslis/DACs.

We can always use differential drivers that can be used either as single-eneded or differential.

Differential drivers are a good approach wrt signal integrity, but can be a bit inconvenient as they require active circuitry on the Rx (ion trap) end for differential to single-ended conversion. Space around the ion trap is generally at a premium, so we normally use only a small passive filter/break out PCB to connect the cables to the ion trap. However, if we produce a sufficiently compact receiver board, this could be okay.

To avoid unnecessarily overcomplicating the design, I'd prefer to avoid using differential drivers unless required. A rough target for cross-talk would be:

If standard SCSI cabling exceeds that*, then this is all a non-issue anyway IMO.

gkasprow commented 7 years ago

We use standard SCSI cables for analog signals up to 100MHz but we send them in differential form and use CMCs.

We can add common mode chokes. IF you don’t want it, just short negative to ground. If you want – just install another CMC on the end.

CMC price is low <1$ https://github.com/notifications/beacon/AEH-vt3stoZzrtgZcroo1Cz1_rMFqeUEks5rX1tIgaJpZM4LvzLa.gif

http://pl.farnell.com/bourns/srf0905-652y/common-mode-choke-6500uh-300ma/dp/1791514

it has 6.5mH for single ended signals, 0.39uH for diff ones and works well up to 2MHz.

Can be easily used for common mode filtering on both sides.

hartytp commented 7 years ago

Okay, let's go for SCSI cables and CMCs.

jordens commented 7 years ago

For the DAC output filtering, I would not like to see much active filtering in on the PCB. A good passive filter to clean up the ~MHz band: absolutely. But active filters are not that good at rejecting above MHz and can themselves easily dominate the noise there. Also the filtering requirements tend to be very different and custom. I am fine with an active filter that can be integrated for free into the gain stage that gets us to +-10V but I would not add extra active components. We do have a frontend power supply on the output connector so that people can apply their own active filtering, right?

gkasprow commented 7 years ago

@jordens CMC is completely passive EMC filtering device, invisible for signal. Later on, user can add additional filter. The most popular CMC you can fin on every DC, HDMI or USB cable. http://www.fair-rite.com/wpapers_anotes/ferrite-cores-for-low-frequency-emi-cable-suppression-2/ we will use discrete version of it.

jordens commented 7 years ago

@gkasprow Yes. The EMC filtering on the cable is all fine and a good idea. I was more criticizing the per-channel active lowpass that Tom had suggested to put on the board between DAC and connector/cable.

hartytp commented 7 years ago

@jordens

I am fine with an active filter that can be integrated for free into the gain stage that gets us to +-10V but I would not add extra active components.

Agreed. My assumption is that we should get active filtering more or less for free using buffer/gain amps that will be required anyway. If the amps are there then we might as well at least have pads so that they can be used as active filters if one wants.

A good passive filter to clean up the ~MHz band: absolutely. But active filters are not that good at rejecting above MHz and can themselves easily dominate the noise there.

Yes, so we'll want pads for a passive filter after the active one.

I'm happy to remove the explicit reference to active filters from the specification if you'd prefer. It's probably better to keep this higher-level anyway: we'll specify the noise we want, including LDAC/CLK/etc spurs. If that can be done passively then I'm happy.

hartytp commented 7 years ago

Broad-band noise

Edit: the Rx (trap) end filters will typically provide 20dB of noise rejection by 1MHz (assuming single-pole, 100kHz cut-off), so if we need to then we can slacken this specification a bit. I've chosen not to do that for now, since the requirements for different traps will be different, so I'd like to keep some slack in the design.

hartytp commented 7 years ago

I've updated the top post to take into account all feedback so far, and filled in the remaining question marks.

This "specification" is now complete in all aspects that I care about. If anyone else has additions/objections/changes, please post them. Otherwise, I'll close this issue and update the Wiki...

hartytp commented 7 years ago

One final thought about this:

Edit:

gkasprow commented 7 years ago

The easiest seem to be simple plastic box with 2 panels (easy to laser-cut) and PCB where HD68, BNCs, VGA, DSUBs and some CMC can be installed. Just specify what you need on the other side.

gkasprow commented 7 years ago

Such PCB or many of them can be also attached to 1U panel. Only mounting fixture need to be foreseen.