v1.0 testing

[gkasprow]

The boards start with an overheating reset chip. There is a bug in pin assignment.VSOT package drawn in datasheet has nothing to do with SOT-23. Use MSOP package pinout instead... It is not critical for board operations so we can remove it for now.

[gkasprow]

POE supply works as expected, at least without load

[gkasprow]

My switch supports 802.3at or 802.3af. I connected 5.1R resistor at the output of the PoE module and it does not work. With 8.R load resistor, it works fine. Idle current from 12V supply is 60mA. The 802.3at delivers up to 25.5W, 12V@5.1R gives already 28V so this could not work. 12@8.1R is 17.7W so it is within the limits of the standard.

[gkasprow]

For some reason, the PoE does not work with dummy injectors. In such case, one must buy also dummy PoE splitter and use DC jack power entry instead. The PoE converter is too smart and does not accept such power source.

[jordens]

That's good. Those passive injectors are also 12 v only typically.

[hartytp]

@gkasprow can we open an issue for each change, rather than one long "errata" issue please? I find that with long issues, it's easy for some of the changes to get missed.

[hartytp]

Also, mixing test reports with change requests makes the problems a little worse. Let's keep testing to a separate issue (ideally, one issue for each test).

[hartytp]

Thinking about PWM resolution. Here is some feedback from @gkasprow:

We are limited by lowpass filter cutoff frequency and PWM counter length. The filters are dual pole ones with Fc=159Hz. It also means that if we want to have an attenuation of the filter of let's say 90dB, we should work at 30kHz. The PWM counter can be even 32-bit wide but this results in very low pulse frequency. In TM4C the PWM modulator can be clocked up to 80MHz. To have 30kHz output we have to use a counter that counts to 2666. It essentially means an 11bit resolution. If we replace the capacitors with 1uF, then 90dB attenuation will be achievable with 3kHz PWM pulses, which means the counter can count to 26,666 and we get almost 15bit resolution.

I'd like the keep the poles roughly where they are now, because I don't want them to impact the loop latency, which is currently set by the 5Hz ADC sample rate. If we want the PWM filter cut-off to be at least 10 times the ADC sample rate the it must be at least 50Hz, which is not much below the current value.

I'll do a simulation with 11 bits resolution. I suspect we will find the PWM resolution is a significant limitation in the overall thermal stability we can achieve (particularly for well-insulated masses where we want to run the TEC driver at relatively low currents).

I can see two ways of getting around this. (1) be a bit cleverer with the PWM to squeeze out another 3-4 bits of resolution (e.g. update it more than once per ADC cylce) (2) add small 16-bit I2C DACs like AD5693R to drive the TEC controller (these can easily be retrofitted to existing boards).

@gkasprow @sbourdeauducq preferences?

[gkasprow]

DAC is usually a better approach because it's not trivial to get very high attenuation from the RC loop filter.

[jordens]

From my experience the relevant cutoff determining the stability will not so much be the reconstruction filter/PWM resolution alone nor the ADC but the PI corner and the overall P gain. And as mentioned in the other issue, there are a bunch of bits to be gained in noise shaping, i.e. not just plain PWM or the interpolating PWM you mention, Also, what's the difference to your existing solution that is achieving 5 ppm stability (18 bit) without issue with the same PWM method?

[hartytp]

Also, what's the difference to your existing solution that is achieving 5 ppm stability (18 bit) without issue with the same PWM method?

Currently, the short-term stability (seconds) is typically quite a bit worse than the longer-term stability. For one setup with a heater current of approx 100mA, we see quantization noise in the from of oscillations of around 400uK pk-pk. The longer term stability (hours) is much better than that. Adding a bypass resistor in parallel with to increase the heater reduces the issue significantly.

[gkasprow]

PWM output stability is determined mainly by the CPU power supply which is affected not only by the supply rail voltage but also CPU load (drop on bonding).

[hartytp]

In our prototype we use a 40MHz SYSCLK with a 10kHz PWM period, updating the PWM once per ADC sample. So, resolution is approx 12 bits. Based on experience (although this is strongly dependent on the heat capacity of the load etc), a few extra bits would be enough to make the TEC current resolution fine in most cases. tl;dr I'd like to shoot for 16 bit resolution in the heater current.

I don't have a strong preference between PWM + interpolation or DACs.

Also, what's the difference to your existing solution that is achieving 5 ppm stability (18 bit) without issue with the same PWM method?

I don't follow you here? The overall stability and the noise induced by the PWM are not directly related.

PWM output stability is determined mainly by the CPU power supply which is affected not only by the supply rail voltage but also CPU load (drop on bonding).

That's true, although I've never directly observed any issues due to this. Over short timescales, noise is removed by the RC filter and over longer timescales it's taken care of by the feedback.

[hartytp]

My inclination right now is that it's probably easier to add 16 bit DACs than to worry about it. Any objections to that?

I'd use a MSOP -> pin header breakout board to hack the DACs onto existing Thermostat prototypes. Thoughts?

[gkasprow]

The board space is not an issue. Such DAC needs a good reference.

[hartytp]

AD5693R has an internal reference.

[hartytp]

Here is a toy simulation of thermostat (needs checking and tidying up!): https://gist.github.com/hartytp/3aa5c0098564771335e6920a95466b6e

The simulations match our observations with prototype hardware well. In particular:

• with 11-bits, the PWM can easily be the dominant limitation. The more I think about it, the more I'm pro having a 16-bit DAC in the next revision.
• with single-precision floats the numerics can easily be the limitation. We should either use double-precision floats (the control loop is simple and slow, so this can be done in software) or need to be careful with numerics

[hartytp]

Linearising the temperature calculation around the set-point to remove the logarithm makes things a little better. But for genuine mK stability, it's probably best to stick with double precision maths unless there is a good reason not to...

[jordens]

I don't follow you here? The overall stability and the noise induced by the PWM are not directly related.

You were worried about just that: "PWM resolution is a significant limitation in the overall thermal stability". That's precisely quantization noise. I'm not so worried about getting another 4 bits out of the PWM but adding the DAC is also fine by me. If you add the DAC, make sure you can PWM its output with the CPU. It's a simple change and gives multiplicative resolution (i.e. just what's needed for large gain dynamic range) while also supporting both approaches.

[hartytp]

You were worried about just that: "PWM resolution is a significant limitation in the overall thermal stability". That's precisely quantization noise.

I think we were talking past each other. Let's forget about it.

If you add the DAC, make sure you can PWM its output with the CPU. It's a simple change and gives multiplicative resolution (i.e. just what's needed for large gain dynamic range) while also supporting both approaches.

Here, you essentially just mean making sure that the DAC can be updated at, say, 10s of kHz and has a proper reconstruction filter before the TEC controller, right?

[jordens]

I'd like the DAC output to be PWM-ed before it hits the reconstruction filter. That's in effect a hybrid multiplying DAC where one (DAC or PWM) sets the range and the other is the PI actuator. In either case it increases dynamic range. You can leave the PWM always on when you are not interested in that dynamic range. And you can keep the PWM frequency high if you are worried by the reconstruction filter.

[hartytp]

@jordens yep, that sounds fine to me. From a hardware perspective, it's what I was planning already. From a software perspective, I probably wouldn't bother to implement DAC PWM in the first instance -- just set it to a fixed value once per ADC reading/control loop iteration. We can play with PWM in firmware down the line when a use-case emerges that needs it.

[jordens]

Just to be clear: I am not referring to muxing the DAC control word but a mux after the DAC that switches between the DAC output and whatever zero TEC current is. Then toggle that mux FET with a CPU PWM output. Feed the mux output into the filter.

[hartytp]

What's the advantage of that over just toggling the DAC control word between two values in software? Modulation of the DAC control word at something like 10kHz should be easy to implement on the microprocessors using interrupts, shouldn't it. Wouldn't that be enough for what you want to achieve?

[jordens]

I think you are underestimating the complexity (race conditions, CPU load, timer setup, SPI/DMA) of modulating the DAC control word. And with that DAC speed at 10 kHz you can only gain 2-3 bits dynamic range.

[hartytp]

I think you are underestimating the complexity ... of modulating the DAC control word

Probably, I've not given it much thought since it's not something I expect to need (16-bit TEC current resolution should be good enough for all the use-cases I have in mind).

I'm not against adding the modulation circuit you describe to the next design revision if you think it will be genuinely useful in real-world applications. As always with these things, if you haven't already done it, I would urge you to do a simulation/get some data for the application you have in mind to make sure that the extra complexity is really worth it. No point increasing circuit complexity if it doesn't help a real use-case.

Also, I don't anticipate contributing to the hardware/firmware development costs for the PWM circuit since we won't use it.

But, with those caveats, and as long as there aren't any issues with board space etc, I don't object to adding this in the next design revision.

[hartytp]

• [ ] If we do add the DACs, we should make sure that we can use solder jumpers/population options to go back to plain old PWM.

[gkasprow]

What about adding to the DAC output the PWM signal, but attenuated below 1 LSB?

[hartytp]

I'm fine with that. It also gives us an easy way of bypsassing the DAC and reverting to PWM if we need to. @jordens?

[jordens]

Maybe I am misunderstanding, but that addition is prone to transients. They would be as large as the PWM quantization that you are worried about. IME precision adding to gain resolution is nasty here. Multiplication isn't since the range factor is fixed.

[hartytp]

Yes, you're right, I hadn't thought that through and it's not a good idea!

In that case I revert to my previous stance that I don't object to adding the resolution multiplication circuit so long as (1) I don't have to do any of the design work (2) it doesn't break other things/cause layout issues/etc (3) someone has done a simulation to show that it actually helps in a real application.

Short of that, I'd suggest we just add the DAC to the next hardware revision and leave fancier modulation schemes open as a future improvement.

[gkasprow]

I'm not saying about the direct addition of PWM signal to the DAC output. To make it work we would need an amplifier. This is of course under the assumption that original DAC has very low DNL. But it would be probably easier to simply use 20bit DAC.

[hartytp]

But it would be probably easier to simply use 20bit DAC.

I'd really like to see a simulation showing the need for >16-bit TEC current resolution before taking this discussion further.

[jordens]

Once the mass gets small (e.g. 10-1g or 20-200 times less than in your example) 16 bits will also become limiting and insufficient. Bypass resistors will be problematic because they turn the driver into a constant voltage source and the temperature, delta-temperature, and power dependence of the TEC resistance show up again. Could be done with a proper current divider but that does look more complicated.

OTOH with e.g. the numbers above (80 MHz PWM, 11 bits, 40 kHz rate) and a simple delta-sigma modulator (40 kHz, 16 bit, digital MASH 1-1-1 with dither, all in software, precomputation and DMA to update the PWM word) we'd have plenty of dynamic range (more like >20 ENOB up to the RC filter cutoff from simulation).

tl;dr I'd really appreciate it if the next version would still support doing PWM. Either by analog modulation of the DAC or directly from the GPIOs.

And probably also switch to some STM32F like Stabilizer. Platform consolidation, wider and more active support in the tools we use and IME fewer silicon problems.

[hartytp]

And probably also switch to some STM32F like Stabilizer. Platform consolidation, wider and more active support in the tools we use and IME fewer silicon problems.

I support this suggestion in principle, particularly as we (Oxford) are going to need both Thermostat and Stabilizer in the near future, so keeping as much of the codebase/tooling common as possible would be helpful.

There are a few prototypes floating around with the TI microprocessor, but the cost of building new units is small compared with firmware development costs, so that shouldn't be a deciding factor for this.

The questions I'd have are:

1. @gkasprow would you be willing to change the microprocessor to one in the same family (or just the same one) as Stabilizer?
2. @sbourdeauducq how will this affect the cost of firmware development?

[hartytp]

Once the mass gets small (e.g. 10-1g or 20-200 times less than in your example) 16 bits will also become limiting and insufficient.

Is this a theoretical concern, or a use-case you have now/are likely to have in the near future? If the latter, are you/the user likely to be able to contribute to development costs?

OTOH with e.g. the numbers above (80 MHz PWM, 11 bits, 40 kHz rate) and a simple delta-sigma modulator (40 kHz, 16 bit, digital MASH 1-1-1 with dither, all in software, precomputation and DMA to update the PWM word) we'd have plenty of dynamic range (more like >20 ENOB up to the RC filter cutoff from simulation).

To check check I understand you, you mean operating the PWM at a fixed frequency of 40kHz, but updating the PWM duty cycle after each iteration (so, at 40kHz).

(more like >20 ENOB up to the RC filter cutoff from simulation).

Won't the noise floor be well above 1LSB at that point, so is this much resolution actually helpful? What level of attenuation are you assuming form the RC filter -- it won't be anything like the theoretical value for frequencies far below cut-off.

Also, what about the point @gkasprow raised earlier that with PWM, one is quite sensitive to noise from the microprocessor? If you're looking for that kind of resolution, you might want to consider an external logic gate operated from a clean supply to clean up the PWM.

[jordens]

Is this a theoretical concern, or a use-case you have now/are likely to have in the near future? If the latter, are you/the user likely to be able to contribute to development costs?

Small thermal masses are not a theoretical thing. They are a use case we have now. You seem to be suspecting that I am making things up.

To check check I understand you, you mean operating the PWM at a fixed frequency of 40kHz, but updating the PWM duty cycle after each iteration (so, at 40kHz).

Yes, for example. The crossover should be tuned but the ballpark between the PWM and the DDSM seemed about right.

(more like >20 ENOB up to the RC filter cutoff from simulation).

Won't the noise floor be well above 1LSB at that point, so is this much resolution actually helpful? What level of attenuation are you assuming form the RC filter -- it won't be anything like the theoretical value for frequencies far below cut-off.

The take home message is not that this will give you an ideal 20 bit DAC straight away (or temperature controller for that matter). It's a simulation, like yours. Yes, components are non-ideal. Yes, there will be some feed-through (but keep in mind that at low frequencies this is quite a bit less of a problem than at RF). And yes, there will be some low-f noise from the power rails. The take home message is that a 11 bit PWM word is nowhere near the resolution limit.

Also, what about the point @gkasprow raised earlier that with PWM, one is quite sensitive to noise from the microprocessor? If you're looking for that kind of resolution, you might want to consider an external logic gate operated from a clean supply to clean up the PWM.

Exactly. If there is too much supply noise in the passband, then it would be better to modulate something stable: PWM the DAC or some other reference output.

[hartytp]

You seem to be suspecting that I am making things up.

No, I'm just trying to understand where you are coming from better and how this is fits into priorities.

The take home message is that a 11 bit PWM word is nowhere near the resolution limit. Exactly. If there is too much supply noise in the passband, then it would be better to modulate something stable: PWM the DAC or some other reference output.

Yes, I agree with all of that.

But, I really don't have a picture of (a) what noise/resolution can be achieved with the current stabilizer hardware (basic PWM) and clever firmware (b) what noise/resolution could be achieved with the MAXIM chip even if the control voltage were perfect -- if an application needs finer control than a 16-bit DAC, does it need an overall fancier design, or is modulating the DAC enough?

Without data on those questions, it's hard to make an informed decision about the right way to do the design.

[hartytp]

Presumably to get a resolution boost by modulating the DAC output, you'd be relying on the DAC having a DNL that's << 1LSB, or am I missing something? If so, that would need to be factored into the design.

Might be simpler to just pick an 18-bit DAC as @gkasprow suggested.

[hartytp]

Without prototyping I'd be quite sceptical about achieving > 16-bits of effective resolution with dithered PWM using a filtered microprocessor output (although I'd be happy to be proved wrong). I'd suspect that something like a discrete logic gate powered from a low noise source (LT3042 for example) would be necessary.

I'd be happy to go for that in the next design revision if someone is likely to fund the design work and firmware development. Otherwise, I'd prefer to stick with the DAC and maybe a solder jumper to allow the DAC to by bypassed and replaced with the microprocessor PWM outputs.

[gkasprow]

Some time ago I wanted to use STM32 and after a long discussion with @sbourdeauducq I switched to TI and used it already in the second design. Fastest STM32 + 1$ PHY is still cheaper than the TI ARM chip so cost is not an issue here. The main argument was support for RUST.

[gkasprow]

@hartytp my student (@wobucho ) finished Thermostat GUI with MQTT support. The firmware supports for now only NTC sensors, but I asked for PT100/PT1000 support. He will publish the source code. I will ask you to test this software.

[hartytp]

Nice! :) I'll be happy to test it.

T

[hartytp]

Some time ago I wanted to use STM32 and after a long discussion with @sbourdeauducq I switched to TI and used it already in the second design. Fastest STM32 + 1$ PHY is still cheaper than the TI ARM chip so cost is not an issue here. The main argument was support for RUST.

I remember that conversation. I think the rust support for STM32 is pretty good. Now that we will have to support the STM32 anyway for stabilizer, it's much better to have the same microprocessor on both boards. Sorry for the change of plans, but will you be able to switch the processor for the next hw revision?

[gkasprow]

Yes, we can change it but somebody will have to port the software. It is based on RTOS so should be easy.

[hartytp]

Cool, thanks!

[hartytp]

@gkasprow how did you student get on with the firmware? Does he have any data he can share with us on the board's performance?

[gkasprow]

He implemented so far only PTC sensor input. I asked him to add PT100/PT1000 option as well. But so far he demonstrated temperature stabilization with TEC and sensor attached. He will also fix the issue with IP address setting that is not saved to EEPROM and implement PID settings from the user interface. So he is very close. MQTT also works. @wobucho wat's the status?

[gkasprow]

btw, I have 3 more Thermostat boards. @jordens would you like to play with it?

[jordens]

Those are the ones with the thorlabs modules or the newer version?

[hartytp]

TI microprocessor + 24-bit ADC I believe.