3U DAC (Zotino) tests & rev 1 errata

[gkasprow]

Issues:

• [x] Use two output resistors in series/parallel to improve power handling (200mW short-circuit)
• [x] reverse C143, C145, C147, C84
• [x] mirrored SOT23 diode footprint due to Altium bug - part of component is on top layer, pads are on bottom layer.
• [ ] add Shottky diodes at the LDO outputs
• [ ] 3mm higher EMC shield on PSU
• [x] reverse IDC connector on BNC to IDC board
• [ ] Zero-index all channels
• [x] swap BNC connector GND terminals
• [ ] either add ~2mm plastic washers on BNC barrel behind front panel or move BNC connectors 2mm towards FP. This is to enable fixing the BNC connectors to the panel using the screws.
• [x] Please remove the annotation "This module connects to Kasli ..." and replace it with something like "EEM connector: IO are LVDS, I2C is 3V3 LVCMOS, P3V3_MP up to 20mA, P12V up to 1A."
• [ ] run PI and thermal simulations
• [ ] Can we run the digital interface from 2V5?
• Temperature controller issues:
  • [x] Decrease the thermal resistance between the thermal island and the main PCB/increase the thermal mass of the island by at least 1 order of magnitude -- without active stabilization, the DAC should only be 1-2C hotter than the main PCB. This should also improve the electrical ground connection between the main PCB and the thermal island.
  • [x] Remove current temperature controller and DAC over temperature shutdown circuitry.
  • [x] Add mounting holes/pads for peltier + heat sink controller by external circuit.
  • [x] Leave temperature probe on the thermal island, bring it out to an external connector, which should also provide +12V power, ground and I2C (maybe use I2C-controlled GPIO instead? This way we can do bit-banged SPI if we need to).
  • [ ] Assuming the peltier + heatsink mount on the top side of the PCB, do we want to add a stick-on insulation on the bottom side to screen the DAC from external air currents?
• [x] Silk + schematic + panel should be "v1.x" not "rev1.x"
• [x] Only have one set of +-12V regulators, not two!
• [x] Reduce SMPs outputs slightly, since we're only using +-12 regulators

Tests that can be done without an Artiq driver:

• [ ] Measure step response. This means measuring the 90%-10% rise time for a full-scale signal. If easily possible, also measure the 1% settling time.
• [ ] Measure noise PSD (0.1Hz to 10MHz) with full scale and mid scale output signal
• [ ] Glitches on LDAC, major carry, etc (trigger off the SPI)
• [ ] Confirm that temperature controller works. Measure temperature stability of DAC in lab conditions
• [ ] Measure drifts with temperature controller running. Set one channel to Vmax/0V, and look at drift over ~24 hours to confirm gain and offset drifts. Measure board temp co (e.g. use hot air gun to heat the board up while measuring output drifts).
• [ ] susceptibility to external EMI sources (i.e. neighbouring boards). I will test it with EMC equipment.

Tests that need an Artiq driver:

• [ ] Measure cross-talk (full-scale square wave at ~50kHz on one channel, another channel at 0V)
• [ ] Measure spurs due to SMPSs, digital-cross talk, pickup etc. Have one channel updating as fast as possible and measure noise on another channel
• [ ] Check for thermal issues (drifts) when outputs are slewing at max rate (e.g. testing for issues due to self-heating in the OpAmps). What’s the best way to test this?
• [ ] Measure harmonics with a full-scale sinusoid output at 50kHz.

Configuration for measurements is: DAC->3m SCSI cable->HD68 to IDC->IDC to BNC->scope/DVM/noise analyser. CMCs and a LPF on the IDC-BNC board as previously discussed. LPF chosen to avoid degrading the DAC noise or bandwidth too much.

[gkasprow]

They are stable. Do you think they could case any issue? The output amp is conneted by 2.8k resistor to the DAC so nothing should happen.

[gkasprow]

I connected shutdown pin of VSS to reference. And this is how the opamp rail looks like. blue - positive one:

And negative one (blue)

[gkasprow]

This could be an issue with power sequencing https://ez.analog.com/message/164258 in this discussion DVDD term is used while this chip nas VDD and DVCC pins only..

[hartytp]

hmm...the data sheet doesn't really mention any special sequencing requirements. e.g the digital supply must turn on before we apply signals to the digital pins. But, that's about it.

[hartytp]

@gkasprow To check that I understand you, the issues you see are:

1. When the reference switches on, there is a large current draw on the DAC's negative supply, which causes that rail to sag by a couple of volts. (But no obvious signs of latch up etc).
2. When you heat the DAC above 50deg it draws a large amount of current from the negative rail until it self destructs.

[gkasprow]

@hartytp That's correct. I have a suspicion that the DAC is already on the edge of latchup which gets accelerated when it's hot. And such latchup condition may happen when there is wrong power sequence or supply rises in non monotonic way.

[gkasprow]

Another, standard condition for latchup are digital signals before power supply. But this should be the case only with DVCC rail which in our case is same as other logic.

[sbourdeauducq]

FWIW @mntng also had an episode where the DAC got very hot after powering the board. Did not happen in subsequent power-ups AFAIK.

[gkasprow]

I already damaged 3 DAC chips which are quite costly so would like to find a cause first before I burn another ones.

[gkasprow]

So I will modify the board in such way that opamps supply will go first,then VSS, VDD, DVCC as it is recommended by msamera here.

[gkasprow]

@hartytp this will take a few days, so I can send you Novo boards first. OK?

[hartytp]

@gkasprow Yes, please do send Novo asap.

there are visible oscillations and it is caused by the fact that LC1964 is missing ceramic cap at its output. There is such cap close to the DAC but it seems the equivalent ESR is too high. So I added 1uF 0805 cap and oscillations are gone

Did you check the supplies on Novo? Does this issue with the negative rail oscillating affect Novo as well? (We can check this when the boards arrive if necessary)

[jordens]

@gkasprow Can we also get Novo boards really soon? That could even be the old (afaict current) "Novogorny" version.

[gkasprow]

@hartytp I cheched supplies, anyway we would see such oscillations in the scpectrum

[gkasprow]

@hartytp tomorrow they will be on a way. @jordens wants one piece as well.

[hartytp]

@gkasprow Can you ship @jordens one of the ADC boards we paid for?

[gkasprow]

@hartytp sure.

[hartytp]

Thanks!

[hartytp]

Never mind, these have already been shipped (due for delivery on Monday). @cjbe Would you mind posting one to @jordens when they arrive?

[gkasprow]

@hartytp I know. But this may be yet another undocumented feature. For some reason 3 chips went to silicon heaven.

[hartytp]

:(

[jordens]

IIRC there were some cases with some opamps where due to power supply sequencing they latched/had transients on their inputs to Vcc/Vee and thereby killed what was connected there. Don't remember where I saw that though.

[hartytp]

@gkasprow What are the DAC outputs doing during these transients?

[gkasprow]

@jordens I've heard about such case. It could happen in case of bipolar inputs. But in our case we have resistors which separate DAC outputs from opamps.

[hartytp]

@gkasprow What's the plan for getting these boards up and running?

[sbourdeauducq]

For rest of them I will need ARTIQ test software running on KC705 or Sayma

Do you have a KC705? I'm afraid Sayma will still take some time due to the RGMII issue. KC705 code should be ready within a few days.

[gkasprow]

@sbourdeauducq I have KC705. I used it to test DIO boards

[cjbe]

@gkasprow do you have an estimate for when you will ship Zotinos to Oxford?

[sbourdeauducq]

@gkasprow OK, you can now use it with the FMC DIO 32ch, the VHDCI breakout board 1.0, and a recent ARTIQ 4.0.dev build for nist_clock (8407b2c or newer). @mntng please provide details of connections and help Greg set things up.

[hartytp]

@jordens I've heard about such case. It could happen in case of bipolar inputs. But in our case we have resistors which separate DAC outputs from opamps.

True, this is probably not the culprit in our case. But, FWIW, the isolation resistors are made less effective by the fact that the DAC has to drive 32 of them in parallel. so the total load impedance is much lower. So, it might be worth double checking that there aren't any glitches on the DAC's output during turn on/off.

[gkasprow]

I checked, all voltages including output stage start simultaneously and monotonically. I added Shottky diodes at the LDO outputs and it works. Now working on temperature controller.

[hartytp]

I checked, all voltages including output stage start simultaneously and monotonically. I added Shottky diodes at the LDO outputs and it works. Now working on temperature controller.

So the diodes fixed this problem? That's good (although, I don't understand why from the traces you posted, since the regulators were never reverse biased)!

[gkasprow]

@hartytp I don't know what killed the DACs. I observed supply levels on already dead chips. But since I had such issues with LDO, I added diodes to make sure it does not happen.

[hartytp]

@gkasprow Sounds like a good plan.

FWIW, I always add these diodes on dual supply designs, without waiting to do tests to see if they are needed. The diodes are very cheap, and not having them can cause some really hard to debug issues.

[gkasprow]

@hartytp I fixed the temperature controller. There is a bug so diodes show inverted logic :) It means that diodes show ""not too high temperature" and "not too low". When they are both on it means it is OK. One need to add two pieces of wire to fix it. Now it takes roughly 3 mins to stabilize temperature. I turned it on cold, loaded 0xFFFF to the X2A register and at rougly 30 degrees and voltmeter shows: 11.289V once it reached the temperature, the output is 11.314 when too hot LED is on and 11.307 when too cold LED is on. I've just received brand new 34461A DMM so will publish measurements once I manage to get them out of the multimeter.

[hartytp]

Great!

[dnadlinger]

Great, very much looking forward to using the prototype!

[gkasprow]

So I think I'm redy to send you 2 pcs and leave one for further investigation.

[gkasprow]

@hartytp I observe strange behaviour. Luckily I set the current limit just above nominal so the DAC was not damaged. It worked properly whole day, but when I switched on heating, just after the regulator stabilised at 50 deg, the temperature of the DAC started growing and caused overcurrent event. Here is the plot of the DAC output.

[gkasprow]

So it seems we have to limit the DAC regulator temperature to avoid such situation. No idea why hot DAC behaves like this at 50 degrees. All voltages are stable to that moment, then negative current starts growing and then power supply switches off.

[hartytp]

Odd. This shouldn't cause problems, since the junction temperature should not be much hotter than the ambient, so well below the 130C max specified.

Probably worth posting a question on the analog devices forum about this.

Can you sanity check by adding a second temperature probe as close to the DAC as possible to check that the DAC isn't very hot (e.g. if the temperature controller is actually heating the DAC up more than we mean to).

Otherwise, let's reduce the DAC temperature a bit. How about 35C?

[gkasprow]

It is even stranger. I switched on the DAC board without heating, connected DMM to the temperature sensor and I observe continuous increase of the DAC temperature. After 2 minutes it exceeds 70 degrees. Current consumption gets increased by 100mA @12V. So it looks like the chip was already broken once I heated it last time. The scale is 10mV/K

[gkasprow]

When the chip is getting hot and I touch it by my finger the current consumption gets reduced back to the initial value.

[hartytp]

When the chip is getting hot and I touch it by my finger the current consumption gets reduced back to the initial value.

Loose connection? Something oscillating?

[gkasprow]

No, I simply cool it down :)

[hartytp]

@gkasprow A couple of unrelated thoughts while looking at the schematic:

• The grounding on the DAC's thermal island isn't great. This could lead to worse cross-talk etc. In v1.2 do you think we can increase the grounding here (e.g. make the tabs connecting this to the main PCB a bit wider) without increasing the thermal conductivity to an unacceptable level? Even a small bit of extra ground could make a big difference IMO.
• I was having trouble tracing the layout via the PDFs, so maybe this is already done, but: it would be good to make sure that the heater return current goes directly to the EEM ground pin and without going through the ground plane any more than it has to (a separate ground trace that connects to the EEM pin would be best). This will stop the (relatively large) heater current causing changes in the ground potential seen by the DAC (which could cause changes in the DAC's output voltage). It's particularly important that the heater current doesn't connect to the ground plane on the DAC's island where the ground impedance is quite high.

Note to self: max operating temperature for the DAC is 85C. thermal impedance is 25C/W, static power dissipation (no load) is 1/4W, so expect junction temperature to be approx Ta+6C=56C, so well within safe limits. Supply currents don't change significantly with temperature according to data sheet.

[hartytp]

@gkasprow Are you sure that the temperature controller was working when you first turned it on? Could it have overshot and raised the DAC temperature to over 85C? The DAC disputes a bit of power, so is it possible that the DAC is hotter than the sensor reads (e.g. if the thermal connection between the DAC and the temperature sensor isn't great)? Can you try the heating while monitoring with an external sensor to check that we never exceed the safe limits?

[gkasprow]

The heating return current is at the temperature regulator. There is 0R resistor to make sure it does not touch the ground on the island

[gkasprow]

the sensor is soldered just under the DAC thermal pad. When I touch the DAC with my finger, I see immediate change of the sensor value so thermal contact is good.

[dtcallcock]

Are you enabling the thermal shutdown function on the DAC (bit 1 of the control register)? Whilst it might not stop the problem occurring, you'd hope it would prevent destruction.