Channel death (little gain) on v1.4 Booster 003B002F3037470535353239

[dnadlinger]

On Booster 003B002F3037470535353239, which is a v1.4-converted v1.0 chassis, channel 7 has failed.

Interestingly enough, all the diagnostics seem nominal (bias currents, …). The input/output power readings are accurate – measured with an external power meter, the gain (150 MHz input frequency) is about 4 dB, i.e. > 30 dB too low.

PGOOD: 1
FAN SPEED: 20 %
CHANNELS INFO
==============================================================================
                #0      #1      #2      #3      #4      #5      #6      #7
DETECTED        1       1       1       1       1       1       1       1
HWID            91:E5   7B:26   C6:B0   E7:8D   2B:2E   E5:92   91:DF   AE:20
INPWR [V]       0.10    0.00    0.00    0.09    0.00    0.05    0.01    0.98
TXPWR [V]       0.01    0.11    0.01    0.13    0.12    0.14    0.12    0.24
RFLPWR [V]      0.10    0.04    0.11    0.06    0.01    0.15    0.14    0.01
INPWR [dBm]     -19.00  -19.00  -19.00  -19.00  -19.00  -19.00  -19.00  -8.07
TXPWR [dBm]     5.00    5.00    5.00    5.00    5.00    5.00    5.00    5.00
RFLPWR [dBm]    -4.60   -3.94   -4.25   -5.38   -3.25   -3.65   -4.70   -3.67
I30V [A]        0.046   0.047   0.047   0.049   0.049   0.055   0.046   0.052
I5V0 [A]        0.259   0.260   0.234   0.242   0.240   0.254   0.255   0.249
5V0MP [V]       4.942   4.950   4.936   4.954   4.930   4.940   4.960   4.926
ON              1       1       1       1       1       1       1       1
SON             1       1       1       1       1       1       1       1
IINT            0       0       0       0       0       0       0       0
OINT            0       0       0       0       0       0       0       0
SINT            0       0       0       0       0       0       0       0
ADC1            13      188     16      207     202     235     200     397
ADC2            156     63      183     105     24      248     223     12
INTSET [dBm]    31.40   31.00   30.39   28.99   36.80   36.69   35.49   33.68
DAC1            4095    4095    4095    4095    4095    4095    4095    4095
DAC2            3609    3513    3552    3479    4014    2957    3896    3825
SCALE1          86      80      87      85      82      55      82      83
OFFSET1         488     630     565     685     646     571     623     664
BIASCAL         1375    1865    1951    1905    1739    1853    1633    1651
HWIS            85.25   82.25   85.50   85.25   81.75   56.17   83.00   84.33
HWIO            933.25  963.25  953.50  1007.25 1005.75 896.17  950.00  984.33
LTEMP           30.00   31.25   31.50   31.50   30.00   32.50   32.00   32.50
RTEMP           30.50   30.50   30.00   30.00   31.25   31.00   33.25   32.25
==============================================================================
> i2cerr
                #0      #1      #2      #3      #4      #5      #6      #7
I2C ERR         0       0       0       0       0       0       0       0
> status 7
[status] e=1 s=1 r1=397 r2=12 tx=5.000 rf=-3.667 curr=0.052 t=32.25 i=0.98 ip=-8.07
> logstash
[INFO] network client disconnected
[INFO] network client disconnected
[INFO] network client disconnected
[ERROR] Interlock tripped on channel 0, i=0 o=1
[ERROR] Interlock tripped on channel 0, i=0 o=1
[INFO] network client disconnected
[INFO] network client disconnected
[INFO] network client disconnected
[INFO] network client disconnected
[INFO] network client disconnected
[INFO] network client disconnected
[ERROR] Interlock tripped on channel 7, i=0 o=1
[ERROR] Interlock tripped on channel 7, i=0 o=1
[INFO] network client disconnected
[INFO] network client disconnected
[INFO] network client disconnected
[ERROR] Interlock tripped on channel 7, i=0 o=1
[INFO] [tempmgt] Temp 0.000000 (d 20) -> Fan 0
[INFO] [tempmgt] Temp 0.000000 (d 20) -> Fan 20
[INFO] [tempmgt] Temp 32.500000 (d 20) -> Fan 0
[INFO] [tempmgt] Temp 0.000000 (d 20) -> Fan 20
[INFO] [tempmgt] Temp 0.000000 (d 20) -> Fan 0
[INFO] [tempmgt] Temp 0.000000 (d 20) -> Fan 20
[INFO] [tempmgt] Temp 0.000000 (d 20) -> Fan 0
[INFO] [tempmgt] Temp 0.000000 (d 20) -> Fan 20
[INFO] network client disconnected
[INFO] network client disconnected
[INFO] network client disconnected

Power-cycling the Booster doesn't change the behaviour.

If I had to guess, I'd say the power stage is probably still fine, and we are seeing the feedthrough over a broken/inoperative switch.

Any particular things that would be interesting to look at (apart from opening the channel and probing at various points along the signal path)?

[hartytp]

Probing all points in the signal path quickly would be amazing. That would pretty quickly tell us where the issue is. Could even be something dumb like a failing cap in the output stage.

[hartytp]

Discussed with @gkasprow off-line and he's keen to get his hand on as many failing units as possible, so once you've done any preliminary probing you have time for please pass that problematic channel to me and I'll ship it.

[hartytp]

Provisionally, I suspect that this is not the same issue as the "FET death" (which is hopefully now fixed) and, indeed, that this isn't the power FET at all. I'd find it surprising if the FET AC gain changed significantly without the DAC transconductance (and hence quiescent current) changing as well.

We weren't able to easily probe the faulty channel since we don't have a wiring harness to allow us to probe it with the supplemental control PCB in place. I've shipped it back to you @gkasprow it should be with you on Tuesday. Please prioritize this when it arrives! These Booster issues are killing us.

[gkasprow]

OK, so it is some mechanical issue. It did not work until I opened it. btw, @hartytp was it you who removed all the screws inside?

[hartytp]

btw, @hartytp was it you who removed all the screws inside?

Don't think so.

[hartytp]

For us it worked and the bias current was normal, but the gain was quite low.

[hartytp]

@dnadlinger I don't believe we removed any screws from the inside when we inspected this unit (after we found out that it wasn't working), but it's possible I'm wrong. What's your memory?

[gkasprow]

Correct me if I'm wrong. This module has on right side handwritten "OK" on the left side yellow label "CH5" It has an enclosure from the first or second series with a large paper sticker on top. Surprisingly, it has a tiny piece of stainless steel that positions both SMA connectors. It was added recently to Creotech modules. The serial number is 801F125CAE20 the HW version is 1.4, red boards with Technosystem logo. The history of this module seems very intriguing. The board inside was attached to the base with only 2 screws which could cause the power stage overheating.

[dnadlinger]

I definitely didn't open that module, nor did anyone else in our my lab. The Booster went straight into the rack from @hartytp's lab. It is a v1.0 refurbished chassis with the devid from the title.

[dnadlinger]

And regarding taking the channel out of the Booster, we just undid the clamping bar screws and removed the module in one piece, yes. Insides should be unmodified.

[gkasprow]

I'm observing gain of 37dB with -10dBm input signal.

[hartytp]

The board inside was attached to the base with only 2 screws which could cause the power stage overheating.

That's definitely not something we did. This was refurbished by TS and we haven't removed any internal screws since.

[hartytp]

I'm observing gain of 37dB with -10dBm input signal.

That's more like what I'd expect. Try running it for a while and see how stable it is...

[gkasprow]

I'm running it all the time with 32dBm of power. So far no issues observed. But the channel is opened with the upper board installed in a vertical position using RA gold pins. I will assemble it as it was originally and see what happens

[hartytp]

At this stage it's hard to be sure what wen't wrong with this unit. I suspect it was switch glitches/lack of TVS diodes killing an amp stage. Let's close, but keep close track of repaired units with TVSs and see if similar symptoms recur.