[RFC] ASICs for Sinara / ARTIQ

[gkasprow]

There is good microelectronics group at my depatment. They saw our activities and asked if we need some ASIC development. They have some founding, tools and production possibilities. They have also brand new fab worth >1e8 $ . They are also very successful in getting grants. They are looking for interesting scientific projects. So the question is if we need some ASICS for RF, photonics, multi channel DACs, lasers, fast logic etc. The group developed recently i.e. complete GPS/Galileo receivers for satellite mission and micropower SoCs for medical diagnostics.

[sbourdeauducq]

There is also SiFive who seem to have good high-performance SoCs with a RISC-V core: https://www.crowdsupply.com/sifive/hifive-unleashed They do customized ASICs as well.

Maybe for supporting the various EEMs with an ASIC we can have some low-performance open-source FPGA fabric which works with Yosys & co.

[sbourdeauducq]

The most problematic part will be the SAWG... but maybe that one can stay in a regular FPGA, at least for now.

[gkasprow]

Once we find some interesting ASIC to discuss, will open this issue.

[gkasprow]

I might have the opportunity to develop such controller ASIC at the WUT. We may get PhD +funding that would work full time on it. ASIC fabrication would be also funded. For the moment, I need to know what it would look like from an architecture point of view. @sbourdeauducq @hartytp @jordens can we write a very initial specification? I have a meeting on Tuesday so need to gather some ideas.

I assume we need:

• standard SiV RISC core
• large direct access SRAM. How large? A few MB?
• standard peripherals like SPI, fast IO, I2C
• do we need GTP core? I would place it outside in standard Artix FPGA but we can also think about DRTIO implemented entirely in HW. We are losing flexibility than...
• do we need LVDS ports inside of the chip? Or we assume that external companion FPGA would be used anyway.
• if we want to implement EEMs directly, we need some kind of PLD logic or at least fabric that would connect them between peripherals. Something like it is implemented in ARM M0+ CPUs.

[sbourdeauducq]

I think that for a first ASIC project we should try to aim for a minimal but useful chip. IMHO something along those lines:

• two CPU cores (SiFIve should be fine) + mailbox like in the current system
• booting from SPI flash, copying the runtime into internal SRAM
• JTAG debugger
• large SRAM for the kernel CPU. Someone who has ample experience with ASIC implementations should decide what is a good trade-off between SRAM size, execution speed, timing predictability (e.g. is some cache hierarchy acceptable?), and cost.
• no SDRAM
• SPI and TTL RTIO PHYs
• RTIO PHY mux that connects LVDS pins to internal RTIO PHYs in a programmable manner (simpler than eFPGA)
• I/Os are all LVDS for EEM compatibility
• maybe high resolution SERDES I/O for the TTLs (requires PLL etc.) if there is a low-risk path
• if there are enough pins, expansion bus so we can add later a FPGA for things like DRTIO

There should also be funding allocated to port ARTIQ to that chip, and to write the HDL and software support for the RTIO PHY mux.

[gkasprow]

Thanks, We can always connect an external PLL chip to mitigate the risk. Does SiFive have JTAG debugger core?

[marmeladapk]

Shouldn't we first validate ARTIQ with something like Zynq? Perhaps analog chips would be better suited for asics in our case?

[sbourdeauducq]

Shouldn't we first validate ARTIQ with something like Zynq?

How does that help? It's a different CPU, different cores, different process, etc. The main thing is how fast ASIC can be compared to Zynq...

[gkasprow]

Yes, we should. But we know more or less the limitations. Building ASIC is long-term process which needs discussion.

[jordens]

I would try to hit something that is hard to do with cots chips. And not something that tries to compete with them. The area where afaics we have a tough time finding good solutions is high density reasonably fast precision dacs that interface with our preferred interfaces: maybe 8 channels, 14/16 bit, low inl/dnl, 100mhz, drtio or lvds-serdes and some integrated smart sawg-like interpolation. The realistic specs may well make trade offs on those (10 MHz instead of 100 could be good enough etc).

[dtcallcock]

Yes, I was about to suggest the same. It's not immediately clear to me that COTS SoC+FRGA chips won't keep pace with our requirements.

With DACs for trap electrode voltages though, COTS solutions aren't really cutting it even on Shuttler. Eventually we are going to need to go well past 100 channels and so custom DACs are on the cards. This is especially true if we want to move the DACs into cryo/vacuum (or even onto the trap) to make feedthroughs and wiring tractable.

[gkasprow]

What functionality currently delivered by EEM modules would you like to have in the vacuum and which need to be in cryo? And what does cryo mean to you? Let's assume we can make ASICS. We can even make hybrids so multiple dices can be packed together with passives on a tiny ceramic substrate.

• How would you like to have the leads connected?
• What about the control system connection? LVDS via the vacuum feedthroughs or single DRTIO.
• would it be feasible to have the FPGA/ASIC controller in the vacuum as well?
• and which functions are the most needed. I know that DACs, both slow and fast.
• what about fast DACs we use in Sayma?
• what about the Booster. Would we need the same power also in the vacuum? Sooner or later existing and future SInara users may want to commercialize their ideas they develop in labs and will need the control system that fits in the same vacuum package as the quantum stuff. What if we make software-compatible micro-modules that work in a vacuum. They would be scaled down versions of EEM modules produced using hybrid technology. Later on, we can develop ASICS when it makes sense.

[sbourdeauducq]

* would it be feasible to have the FPGA/ASIC controller in the vacuum as well?

Is there a good solution for cooling?

[gkasprow]

You can attach it to the wall of the vacuum chamber

[hartytp]

I had an interesting chat a while back with engineers working on low-voltage CMOS control hardware to go inside dil fridges for silicon qubits. A whole research topic in itself, but quite cool stuff.

[gkasprow]

Actually, CMOS ICs work quite well in cryo temperatures. It's much worse for bipolar circuits. CERN has an internal database of standard chips that were verified in cryo conditions.

[gkasprow]

But the question was if physics community would be interested in miniaturized, vacuum-compatible versions of EEM modules.

[hartytp]

Actually, CMOS ICs work quite well in cryo temperatures.

It's about the power dissipation. Dil fridge will give you less than a mW on the cold stage. So, if you want to integrate your control hardware onto your quantum chip you can't go for standard logic. Anyway, wasn't saying this is something we should worry about, just a comment on some cool technology.

[hartytp]

I'd be interested in a miniaturised version of fastino. So, >100 DAC channels in a package. Maybe 14 bit, >1MSPS.

[jordens]

@gkasprow The answer is generally yes. But what's realistically better than what you can do outside the vacuum? When identifying the killer applications of in-vacuo ASICS, for ion traps it's relatively easy to look at what's going into the vacum and what is coming out of it. That's RF (trap drive), DC (electrodes), microwaves (gates), and photons. Then the applications are: (a) The photon counting electronics could be moved into the vacuum with SNSPDs, (b) a multi-channel trap voltage driver, (c) a trap RF driver, (d) the ~GHz microwave gate drive electronics. My guess is that (b) is the moste needed and most realistic use case.

[hartytp]

My guess is that (b) is the moste needed and most realistic use case.

Yes

[jordens]

To those specs add maybe +-5V or +-10V, whatever you can realistically get through the vacuum feedthrough (~10x100MBit/s LVDS or GTx), pushing some interpolation/AWG intelligence onto the DAC if possible (with power and flexibility constraints). Cooling can typically be arranged. It would need to be pretty good in terms of noise and stability spec, i.e. comparable to what exists already outside.

[dhslichter]

Late to the party, but I agree that making a Kasli-type device with an ASIC is not particularly useful. The challenge with making DACs on an ASIC is that for our purposes one may need a specialized process to get optimized noise/drifts/etc. But I agree that for ion traps, the most pressing/useful ASIC would be one that can go in vacuum or cryo to provide photon counting/timestamping/detector biasing or to provide trap "dc" voltages. For ion traps, "cryo" means ~3-10 K, where there is plenty of cooling power. These temperatures also relax some of the outgassing-related materials constraints.

My worry with ASICs is how expensive/time-consuming they are to debug. One would need to make test devices that are operated at cryogenic temperatures to understand what parameters are different from at room temperature, and then go back to iterate on actual designs...and of course, all the testing becomes more challenging if a cryostat is needed, because you can't just put probes on a circuit anywhere you want to.

[gkasprow]

I invited 2 ASIC guys from WUT to this discussion.

[gkasprow]

The initial idea is to focus on multi-channel fast and precision DAC.

[gkasprow]

I saw a presentation with Artix running in Cryogenic temperatures as a TDC. There were issues with ADC and CDR but logic was working fine, consuming slightly more current H. Homulle, S. Visser, E. Charbon, “A cryogenic 1 GSa/s, soft-core FPGA ADC for quantum computing applications”, IEEE Trans. Circuits and Systems I, 63(11), 1854-1865 (2016).

[sbourdeauducq]

To those specs add maybe +-5V or +-10V,

What current? I suppose that it will drive capacitive loads. How large is the capacitance? The amplifier also needs to be stable with a capacitor at the output.

[sbourdeauducq]

I wrote up a document to summarize the idea and requirements. This is a very early draft, so any feedback appreciated. cheko_wp.pdf

[dhslichter]

A good place to look for some relevant numbers and performance figures for ion trap DACs would be in this paper, which demonstrated chip-integrated DACs for ion trapping. You can see how they implemented things, what they found the issues to be, etc. I have linked to the arXiv version for those who do not have journal access, but the final journal version is here.

[dtcallcock]

You should reference this GTRI paper (preprint). They used AD silicon instead of a custom ASIC and there was a PCB between the chips instead of TWVs and bump bonds, but other than that it's a good proof of principle for your two-die scheme.

The technical reports for Sandia's HOA 2 and Perigrine/Phoenix contain a lot of relevant technical information.

it is desirable that the device be operable at low temperatures (3 to 10K)

I'd say 3 to 15K. 15K is roughly the upper limit for effective cryopumping. Those extra few degrees buy you a lot in cooling power and thermal conductivity, and may help with carrier freeze out.

Permissible output capacitance: 5pF (TBC). The trap electrode and its connection to the driver are modeled by a capacitor.

You need some reasonable capacitance to shunt the RF pickup on the DC electrodes to ground. For reference the Sandia Peregrine/Phoenix traps have an 800pF on-trap trench capacitor connected to each electrode.

Voltage: ±5V to ±10V. This is a requirement from the physics of surface ion traps.

Peregrine/Phoenix traps are rated to ±20V. The higher voltages are presumably required for big old slow 171Yb+. You could insist people use lighter ions or live with lower trap frequencies. Or you could scale the trap dimensions and/or use a more efficient electrode geometry to lower the voltage requirements.

Sample rate: 10MHz to 100MHz

I'd probably consider a topology with N slow, accurate DACs (Where N is number of electrodes) and N/10 fast DACs with multiplexing. From what little I know about DAC design you probably can't get N 100MHz DACs into your target size/power/performance.

A simple interface module, in the EEM form factor, connects the digital signals of one or several Cheko devices to the FPGA of carrier cards such as Kasli or Kasli-SoC

I could Imagine FMC form factor being more suitable depending on how this chip ended up looking.

[dtcallcock]

Relevant: https://github.com/sinara-hw/meta/issues/57#issuecomment-708148657

[sbourdeauducq]

For reference the Sandia Peregrine/Phoenix traps have an 800pF on-trap trench capacitor connected to each electrode.

Ouch. How does that play with a 10-100MHz sample rate?

[dtcallcock]

Ouch. How does that play with a 10-100MHz sample rate?

You generally have an R between the trap C and the DAC, so the impedance is somewhat higher depending on your RC cutoff. But also if every channel has it's own driver in a SOIC-8 on a big PCB in air then you have the envelope to drive what you like.

[dtcallcock]

You generally have an R between the trap C and the DAC

From https://www.nist.gov/system/files/documents/2017/05/09/blakestad2010thesis.pdf (ie. the current NIST 'transporting stuff around' trap).

[dtcallcock]

Relevant: #57 (comment)

The take home from the AD dude was that ion traps fall into an annoying gap between R2R and current steering DACs with regards to pretty much every performance metric you care about. I think this is pretty fundamental so ASICs wouldn’t solve it directly, but they would allow for narrowly targeted hybrid solutions.

An extension of that would be to just off-board the fast DACs (even move them outside of vacuum). If you need a lot less of them then you still make big gains on the wiring problem. As well as time multiplexing, you could simultaneously perform the same shuttling operation in several different trap regions using one set of fast voltage waveforms - if the trap geometries are repeating tiles, so are the voltage wafeforms (note that you probably need a separate offset for each electrode to compensate for charging).

[dhslichter]

Another possibility to consider would be one-bit (or few-bit) DACs with sigma delta modulation for realizing the desired voltages (like a high-performance audio DAC). You could do all the digital side in ASIC hardware so it could be efficient. You would need higher sample rates, but the lowpass filters on the traps mean that you should be in fine shape. This would require you to run with some high-frequency DAC clock to output the samples, but would save a lot of analog design complexity and footprint for the DACs themselves. Basically you do DAC -> lowpass -> amplifier -> (optional switch) -> on-trap lowpass. If you choose the one-bit DAC to be just a high-voltage switch between +/- 10 V rails, then you could dispense with the amplifier altogether, and you could probably achieve fairly low noise too. You would need low-noise rails and would have to think about channel-to-channel crosstalk (depends on the lowpass you use, how much current you draw from the rails each clock cycle), but it would enable a much smaller footprint. This design style also gives you very good linearity for the output voltages. In addition, you are less dependent on the temperature coefficients/performance shifts/etc of various components at cryogenic temperatures.

The idea mentioned by @dtcallcock above about having multiplexed (or external) "fast" DACs still has some problematic points IMHO (thus my interest in Shuttler, for example), but it is one potential way around some of the challenges.

[dhslichter]

A summary of my thoughts on the current cheko whitepaper is that it is asking for something which performs like the Shuttler design spec (more or less), but is on a single die in vacuum. It's hard to see quite how that will work, from a size/power/complexity perspective. I think a better goal would be an in-vacuum Zotino, which would still be useful for many applications.

[sbourdeauducq]

Another possibility to consider would be one-bit (or few-bit) DACs with sigma delta modulation for realizing the desired voltages (like a high-performance audio DAC). You could do all the digital side in ASIC hardware so it could be efficient.

Isn't that basically moving the power dissipation into the lowpass RC filters? And it seems a LC filter cannot work as the L would be too large :(

[sbourdeauducq]

You need some reasonable capacitance to shunt the RF pickup on the DC electrodes to ground. For reference the Sandia Peregrine/Phoenix traps have an 800pF on-trap trench capacitor connected to each electrode.

In https://arxiv.org/pdf/1810.07152.pdf they suggest a more reasonable 10pF to 50pF...

[dhslichter]

@sbourdeauducq in general one wants the following (it is hard to achieve all of these together!):

1. extremely voltage low noise at/near the ion motional frequencies (typically 500 kHz - 10 MHz, but can be outside this range). Noise of this type contributes to heating of the ion motion.
2. very low voltage noise at few kHz to tens of kHz. Noise of this type dephases the ion motion on the typical timescales relevant for experiments (e.g. duration of an entangling gate)
3. low voltage drift/tempco at dc. These drifts cause the ion motional frequencies to drift, which then requires more frequent recalibration.
4. fine dc resolution and good DNL (in practice, ~16 monotonic bits are usually used). This determines how small a step one can make in adjusting the ion position or motional frequency.
5. for non-adiabatic transport, separation, and merging, the ability to change these voltages "rapidly". What this means exactly will depend on the trap design and the ion species, but roughly speaking the voltages on the electrodes, after any filters, should slew across roughly their entire range in ~few us or less.
6. for an on-board DAC, power dissipation should be "manageable". What this means exactly will depend very strongly on the experimental apparatus. For a room temperature apparatus that can be well thermally heat sunk to the vacuum can it's not a big deal. For a cryogenic apparatus, probably the DACs should dissipate less than 500 mW total, certainly less than 1 W, but of course there may be some flexibility here too.

How important each one of the above factors is will depend on the trap, the ion species, the desired science goals, etc. There are many ways one could think about achieving them. Historically, people have usually used RC lowpass filters to achieve (1), and have relied on the DAC and amplifier specs for (2) - (4). This is one way, but not the only way, to try to achieve these. The ability to do (5) depends on the filter choices as well as the DAC/amplifier choices, and as has been pointed out it is kind of silly to make things that have to fight the lowpass for (1) with huge slew rates/current drives in order to achieve (5). Criterion (6) hasn't been tested that much because there are only a handful of results (GTRI, MITLL) that have tried to run in-vacuum and/or cryogenic DACs for traps. Most DAC systems for ion traps these days dissipate a lot of power (much more than given in (6) above), unless they are very slow and/or low voltage.

a more reasonable 10pF to 50pF...

@dtcallcock is right that the capacitance has a dual purpose, both serving to lowpass the DC bias as well as to shunt the RF pickup, so there is again a tradeoff there. Using 10 pF means you will get substantially more RF pickup on the DC electrodes, with corresponding impact on micromotion etc. Whether or not this is a problem for you will depend on the specifics of your experiment and science goals. So these values may or may not be "reasonable", depending...

Isn't that basically moving the power dissipation into the lowpass RC filters? And it seems a LC filter cannot work as the L would be too large :(

If you had a switch and basically ran the trap capacitance like a sample and hold, there would be transient power dissipation in the RC while you got to the right voltage, and then you open the switch and no more current. One could shut down DAC channels when the switch is open too to save power dissipation. I am not suggesting that this is necessarily the way to go but that the problem is not insuperable. There is also a lot of room for more thoughtful/careful design of filters for ion traps to make (1) and (5) be less in conflict. Obviously this would impact DAC design too.

I guess my point is that it is hard to design one DAC that will make everyone happy and meet all needs, which is why for example Zotino, Fastino, and Shuttler are all desired by different people for different applications (sometimes even more than one in the same application!).

[dtcallcock]

@dtcallcock is right

When isn't he.

I also think the flip chip approach is a bit ambitious for a first stab. Mainly because you also need to produce (or convince a trap foundry to) a matching ion trap with thru-wafer vias - no mean feat. If you instead use a PCB between the ASICs and trap you give everyone the option of using existing/future trap designs. I'm guessing the PCB won't be a bottleneck until you get to thousands of electrodes, which already kicks the can a long way down the road on the wiring problem.

or a cryogenic apparatus, probably the DACs should dissipate less than 500 mW total, certainly less than 1 W, but of course there may be some flexibility here too.

You could also imagine a two chip solution where the real power hungry stuff sits on an intermediate temperature stage (40K or so) where you have 10s of Watts of cooling available. Wiring down from there to the trap with stuff like flex PCB is definitely a lot simpler than going from room temp and/or airside.

[dhslichter]

@dtcallcock is right

When isn't he.

Only on rare, delightful occasions ;)

In seriousness, I agree that flip-chip is a big ask for starters. I would begin with a PCB-coupled DAC (you could make it a BGA chip, if that means it's more ready for flip-chip later), which will be much easier for people to integrate into their setups (and thus drive volume to make the development worthwhile).

Not sure how well one would be able to break apart the "power hungry" part from a "low power" part for a high-voltage-output DAC system, but one could potentially put the whole DAC at 40 K and then run just the signals down to base temperature. This would be a compromise between an out-of-vacuum DAC and a trap-integrated DAC that gets some of the advantages of the latter without as many of the challenges.

[gkasprow]

Late to the party, but what about such a solution:

• in one of the papers the guys built a DAC and to lower the noise added an S&H circuit at its output
• In the case of cryopumping, we have two stages. 4K which can extract merely 1...2W of power at 4K and over 10W at 50K. Do we gain much in terms of noise operating the DAC in 4K instead of 50K
• What if we make a combo chip that is a multiplexer with multiple S&H circuits and sits in the 4K area. It has a sequencing logic and analog muxes. We deliver a few channels from 10MHz DACs sitting in 50K zone. Still within the vacuum chamber.
• Did anyone try to run standard AD DACs in 50K? They are often CMOS chips so might work with some parameters degraded... I can do such tests in a few weeks.

I was asked what High-end would I do in terms of quantum. My first thought was ASIC. In 3 months I will have a complete vacuum-cryo-setup in my lab. My colleagues want to characterize the chips they already designed at the WUT in 4K/50K.

[gkasprow]

Update: We applied for two grants. We will characterize the high voltage CMOS process (sth like 110nm) in cryogenic temperatures (4K) In the meantime, we will build a room-temperature demo of the new trap control architecture Then we will build ASIC and test it in cryo temperatures. The cryo test station is under construction. There is going to be fun.

[gkasprow]

Update: the cryocooler, temperature controller, and vacuum system arrived. It's going to be fun to assemble all together.

[gkasprow]

Update: we signed a Quantera grant for cryo-ASIC.

[gkasprow]

Update: the first characterisation ASIC will go for tape out this month.