[RFC] Sinara Servo

[dtcallcock]

The Sinara ecosystem lacks a digital servo capability beyond SU Servo. I am interested to hear what people think might be a good path to creating one. I have laid out a few options.

Voltage output version of SU servo

SU Servo is an 8ch servo for Artiq based on Kasli. It uses the 8ch Sampler ADC as the input and changes the phase/amplitude/frequency of a pair of Urukul 4ch DDS cards as its output. An obvious extension of SU servo would be to have a voltage output instead.

Pros:

• Sampler, Kasli and much of the gateware already exist and would be reused. The required Fastino DAC output board is very simple and may be useful elsewhere.

Cons:

• Latency limited to ~2us
• An 8 channel system requiring 3 or 4 EEMs is somewhat unwieldy for applications that need 1 channel
• Implementing PDH locks would take a further Urukul + a bunch of demodulation electronics.

Fast DAC/ADC EEM

An EEM with 1 or 2 channels of fast DAC and ADC. Servo runs on a Kasli or a Humpback with an FPGA board plugged into it.

Pros:

• Board would be pretty simple.

Cons:

• Requires two cards for standalone operation
• EEM interface is a bandwidth bottleneck and adds power consumption and latency.

Standalone Board

FPGA on the same PCB or a piggyback board like the [MicroZed] (http://zedboard.org/product/microzed) or an FMC carrier card with DAC and ADC on the FMC. Servo loop implemented on the FPGA EEM for optional RTIO control of lock parameters (eg. integrator hold or set point) Ethernet for GUI/logging

Pros:

• Single board solution
• High bandwidth available would enable direct digital demodulation of PDH signals, especially if a DDS is included on the board.

Cons:

• The most complex to design

Reuse an existing design

This wheel has been invented a few times, including at NIST. The NIST one has a few drawbacks - high cost ($1k/channel), delicate and easily damaged front end, no ethernet, Artiq integration is pretty barebones, and Dave L doesn't really have time to support the software/firmware. Eurocard format would also be a plus for a Sinara-based lab.

Does anyone know of anything else out there?

[hartytp]

The way I would probably do this if it were me would be something between "Voltage output version of SU servo" and "Fast DAC/ADC EEM":

• hardware would be a single EEM with 2xDAC and 2xADC. At least 1MSPS, maybe a little more if there is a simple way of doing the digital interface
• connect to either Kasli or, ideally, an FPGA-version of Humpback
• gateware is essentially just the SU-servo with small tweaks. probably do the ethernet interface by running artiq on the FPGA board

[dtcallcock]

ideally, an FPGA-version of Humpback

Is there any reason this would be preferable to using a Kasli? I can see how it might be a bit cheaper but also that cost saving could get eaten up porting-to, debugging and supporting another FPGA board. What is the likely cost of Kasli and Humpback @gkasprow?

[gkasprow]

It would be at least 3x cheaper than Kasli.

[dtcallcock]

Just to be clear, what are we talking about in $$$ for each board?

[gkasprow]

It's roughly 1k for Kasli. For Humpback I don't expect it to be higher than 300$

[dtcallcock]

Copying comment of @hartytp from #23 to here:

Why we need another FPGA?:) Cannot the algorithm be implemented in Kasli?

Maybe that's the right way to go. Some things to consider are:

• We probably want the sample rate for this to be as high as possible when using SPI parts. So, at least 1MSPS and ideally more like a few MSPS
• this can be problematic when the FPGA is not on the same card as the ADC due to meeting timing on the ADC SDO lines, which are have a round-trip delay + some delay from the ADC when compared with the SCK (generated by the FPGA).
• You then have to either limit the ribbon cable to quite short lengths or do something more fancy to allow you to take out the SDO->SCK delay. One option for this is to use an ADC which generates a delayed SCK signal and then latch SDO from that (or add a buffer to loop that signal back to the FPGA). It's not so hard, but it requires thought and also takes up more pins on the EEM connector
• Kasli has 12EEM connectors, which seems like a lot. But, they go fast when one starts using EEMs that take up multiple slots. For a 2-channel board (2 ADCs + 2DACs, so 4 BNCs on the front panel), we need to think about how many EEM connectors would be consumed and make sure it's reasonable
• maybe this is not a good argument, but I felt that it's nice to be able to update the servo card gateware without having to reflash the main Kasli (this could be what's running the experiment, so it's annoying to have to update it). I was thinking that we might want to move more in the direction of "distributed" local control units, rather than making Kasli do everything.
• We also need to think a little about timing etc when shoving too much logic on Kasli. Probably okay with the higher speed grade, but I've certainly found that adding almost anything extra (on top of ARTIQ) to the -2 speed grade can cause timing failures.
• finally do we need an FPGA for this, or can we implement a decent complexity feedback loop with a latency of ~2us using a microprocessor?

Anyway, just some thoughts, so long as this is sensibly implemented I don't mind how it's done.

[dtcallcock]

The more I think about the more I think there could be two boards. As a strawman, how about:

Board 1: High performance daughterboard for Kasli.

• Roughly equivalent to the NIST servo (100MSPS parallel DACs/ADCs, few 100ns latency) and able to do direct digital modulation/demodulation for things like PDH locks.
• To get that bandwidth and latency, the board would need consume most of the EEMs on Kasli. This board could tharefore stack onto Kasli and use the EEMs as board to board connectors to avoid ribbon cable SI issues.
• The back few EEMs would be left open for DIO etc.
• Kasli FPGA would be pretty much dedicated to running the servo.
• High cost, approaching $1k/ch

Board 2: Microcontroller-based

• ~MSPS DACs/ADCs
• Ethernet for standalone operation
• One EEM to allow a Kasli to control lock parameters if desired
• Could also be used as a general lab logger/controller to replace LabJacks (which are great but which have a few too many features I pay for but never use, and a microcontroller that doesn't seem fast enough or flexible enough to run a reasonable servo).
• Low cost, of order $100/ch

I think this approach addresses all of @hartytp 's points above. It also mirrors how we've done things at NIST, where the all-singing, all-dancing FPGA lock is used where the performance is needed and old analog locks we had lying around are used everywhere else to save money.

[dtcallcock]

I'm also surprised that 'Board 2' isn't already available as an Arduino shield or similar. The retired AnalogShield is the closest I've ever seen to an DAC+ADC that doesn't suck on a shield.

[hartytp]

@dtcallcock I think something like "board 2" (the blur naming convention) would be a great addition. Maybe also stick on a D-type or something for some GPIO if there is room.

AFAICT it should be pretty easy to get MSPS update rates using a decent microprocessor even with a moderately complicated loop filter (but this needs a bit of thought).

One other thing I like about the microprocessor approach rather than the "stick everything on Kasli" approach is that it makes logging a bit easier. e.g. one often wants to keep an eye on the locked error signal/control voltage to check things aren't drifting or loosing lock. If the servo is implemented on Kasli then this has to be done through moninj. Having a microprocessor with a really simple ethernet interface allows one to do asynchronous monitoring via a dumb python script, which seems much nicer IMHO.

[sbourdeauducq]

Is the moninj system that bad? It is also accessible by simple Python scripts. There is also little that prevents you from opening another TCP port on Kasli with a different protocol. A better argument would be that the servo could be used standalone without anything from ARTIQ.

[hartytp]

If we do go down that route, it's worth revisiting previous discussions about making the feedback card an AFE mezzanine for Humpback. This will probably push the overall BOM cost up a bit, but allows the project to piggy back off the software/hardware development done on Humpback.

We'd need to make sure that the microprocessor is able to run the MSPS feedback loop as well as the ethernet interface at the same time. Some microprocessors have RTUs that can be used for the loop filter, but they can be a bit buggy and poorly documented. Another option, which we've used on some of our projects, would be to put a microprocessor on the AFE to run the loop filter + handle the DACs + ADCs. This would have a really simple firmware that had a configurable loop filter and exposed a few registers over a serial (UART/SPI) interface to allow Humpback/Kasli to configure it, read ADC + DAC values, etc. Then let humpback do things like the ethernet interface.

Anyway, as always, lots of ways of skinning a cat...

[hartytp]

A better argument would be that the servo could be used standalone without anything from ARTIQ.

Yes. Also, one may not want the servo to be rebooted whenever the ARTIQ master is rebooted.

Is the moninj system that bad?

True, it's not that bad. My biggest issue with it is the lack of documentation, and its reliance on a complex artiq stack. This makes the barrier to entry for users who want to do some simple hacking quite high.

I also wonder how well moninj will scale to large experiments with large numbers of peripherals connected to moninj and many clients receiving updates.

[dtcallcock]

If we do go down that route, it's worth revisiting previous discussions about making the feedback card an AFE mezzanine for Humpback. This will probably push the overall BOM cost up a bit, but allows the project to piggy back off the software/hardware development done on Humpback.

On the other hand, do we want a core Sinara board to be a stack of 3 PCBs, one of which we have no control over bugs and availability? We could still leverage Humpback hardware/software development if we use the same chip and common layout elements (eg. Ethernet) as the main Humpback target (eg. the Nucleo-144 boards, for which Altium files are available).

[dtcallcock]

We'd need to make sure that the microprocessor is able to run the MSPS feedback loop as well as the ethernet interface at the same time. Some microprocessors have RTUs that can be used for the loop filter, but they can be a bit buggy and poorly documented.

Do you mean things like the PRU on the Beaglebone's chip? This seems pretty well documented, but I'm prepared to believe it's buggy.

[hartytp]

On the other hand, do we want a core Sinara board to be a stack of 3 PCBs, one of which we have no control over bugs and availability? We could still leverage Humpback hardware/software development if we use the same chip and common layout elements (eg. Ethernet) as the main Humpback target (eg. the Nucleo-144 boards, for which Altium files are available).

That's a reasonable point.

Do you mean things like the PRU on the Beaglebone's chip? This seems pretty well documented, but I'm prepared to believe it's buggy.

Yes, I was thinking about the Beaglebone in particular. IIRC, it was Tim/@cjbe who played around with that, and would be the people to ask for details.

[cjbe]

The Beaglebone (AM355x) PRUs are pretty horrific to work with - I would avoid them if at all possible: The tool-chain is not good, and the hardware has a plethora of bugs.

[dtcallcock]

It seems like there are quite a few chips out there with two or more cores so one can be dedicated to running ethernet etc and one to running a deterministic control loop (eg. NXP Vybrid). Perhaps someone with more experience could weigh in on whether these kind of things are a good path or just junky and hard to program.

[sbourdeauducq]

You don't necessarily need two cores to run a deterministic control loop; if there is no SDRAM, caches etc. a simple solution that is 100% correct is to run the control loop in a timer (or ADC) interrupt routine, and Ethernet and other things in the main loop, with nothing else using interrupts. More involved solutions include RTOSes, nested interrupts, rtfm, etc.

[gkasprow]

There are fast 100MS/s ADCs with LVDS output. For example LTC2174 family. With Spartan6 and Artix chips you need 10 LVDS to get data from 4 channel ADC (8 data lines, bit clock and frame clock) It adds some latency but it is negligible.. We use them in this design. Then 2 EEM sockets would serve 4 ADC channels. 2 channel versions are also available and would consume 6 LVDS lines. One could use single EEM then. But there are no serial LVDS DACs. Probably due to issues with lane alignment. So they would need to be driven directly. Dual channel LVDS DAC would need 14 + 1 LVDS lines and would consume another 2 EEMs. With 3 EEMs occupied we would be able to fit 2x 125MS/s ADC and 2x DAC on same 3U board. In this way Kasli would be able to run 3 such boards and 6 servo channels.

[hartytp]

@gkasprow that's an option, which could be good depending on the use-case. At least for the things I have in mind, speed is less important and a few MSPS is sufficient. I'd prefer to have a lower-power simpler setup with a microprocessor for localised feedback loops for the reasons discussed before. As always, however, there are other users with other needs so a different board may be needed as well...

[gkasprow]

This is idea for @dtcallcock Board 1 design

[hartytp]

aah, ok, ack. Maybe we should split these two designs into two separate issues.

[gkasprow]

I'm not sure if we need intermediate step between 1 and 100MS/s boards. 100MS/s ADCs can be downgraded to 30MHz. So we can have just 2 assembly variants with different speed grade chips.

[dtcallcock]

Maybe we should split these two designs into two separate issues.

[RFC] New EEM: Fast Kasli Servo #26 [RFC] New EEM: Slow Microcontroller Servo #27